diff --git a/CHANGELOG.md b/CHANGELOG.md index 89cd739b9..c0fb2160c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,255 @@ All notable changes to GBrain will be documented in this file. +## [0.41.0.0] - 2026-05-24 + +**Your 100-job subagent batch now actually completes.** A real user ran `gbrain jobs work --concurrency 10` against an Azure-hosted Anthropic endpoint, submitted 100 background jobs, and watched every single one dead-letter with `rate lease "anthropic:messages" full (8/8)`. The default cap of 8 starved 2 workers; every starved job got marked as a failure, hit `max_attempts = 3` after 3 lease-full bounces, and dead-lettered. This release turns minions from "a CLI you drive" into "a fleet you supervise" — submit a batch, walk away, come back to completed work. + +Four bugs from the field report fixed first. Then the surrounding ergonomics so leaving the room is a real promise, not honor system. + +To turn it on: run `gbrain upgrade`. The defaults do the right thing for fresh installs and existing brains. If you're on Azure / Bedrock / self-hosted with no provider rate limit, also run `export GBRAIN_ANTHROPIC_MAX_INFLIGHT=unlimited` so you're not capped by gbrain's safety default. + +What you'd see in a concrete example. A 100-job subagent batch on default settings: +- **Pre-v0.41:** queue accepts → 100 dead-lettered in 30 seconds → operator stares at the carnage +- **v0.41:** queue accepts → bounces pace against the upstream → 100 completed in ~3-5 min → `gbrain doctor` shows `subagent_health: ok` → audit row per bounce visible for forensics + +Things to know about. (1) Cost protection: `gbrain agent run --budget-usd 5 ...` now actually enforces the ceiling via a reservation pattern that holds even under fan-out. Pre-submit projection prints "this batch will take ~30 min and cost ~$2.40 at current cap of 32" with a paste-ready cap-raise hint. (2) Self-healing batches: when a subagent fails with `prompt_too_long`, `tool_schema_mismatch`, or `malformed_json`, the worker auto-submits one retry with the failure context — `tool_crash` / `tool_unavailable` / `tool_permission` deliberately do NOT self-fix (those are real bugs you need to see). (3) Live dashboard: `gbrain jobs watch` now exists — TTY tail of throughput / lease pressure / clustered errors / budget remaining; admin SPA gets a matching Jobs Watch tab. (4) Auto-adaptive lease cap: the worker reads bounce rate + upstream 429s + latency stability and adjusts the cap automatically. Bounces with NO 429s = workers starving = cap goes UP (the correct sign per codex pass-2 #9 — the original draft had this inverted and would have cratered cap during healthy bursts). + +What we caught and fixed before merging. Three independent reviews + three codex outside-voice passes converged on the wave. The reviews found five silently-broken designs that would have shipped without them: an inverted controller, a depth-1 budget ceiling, an unbounded audit table, PGLite-broken lock paths, and deleted-owner budget zombies. All five caught and fixed; ironclad regression tests pin each correction. Test surface: ~120 new unit tests across the new modules plus an IRON-RULE regression test on the E5 controller's bounce-sign that would scream if a future "let's simplify the rule" PR ever flipped it back. + +### Itemized changes + +**The four base bug fixes (field report):** + +- **Bug 1 — rate-lease default 8 → 32 + `unlimited` sentinel.** `src/core/minions/handlers/subagent.ts:61` new `resolveLeaseCap()`. Default cap goes from 8 to 32 (matches 10-concurrency batches without starving). Accepts `unlimited` / `none` as explicit POSITIVE_INFINITY sentinel for upstreams with no provider rate limit. Throws on `NaN` / `0` / negative input with paste-ready hint (codex pass-1 #7: silent `=0`-uncapped semantics were dangerous; "0 means disabled" is the universal convention). +- **Bug 2 — lease-full bypass that doesn't burn `attempts_made`.** Pre-v0.41, lease-full bounces routed through `failJob` which incremented `attempts_made`. After 3 bounces a job hit `max_attempts` and dead-lettered with `rate lease "anthropic:messages" full (8/8)`. Now: new `MinionQueue.releaseLeaseFullJob()` mirrors `failJob` minus the attempt bump; worker catch block detects `RateLeaseUnavailableError` BEFORE the existing `isUnrecoverable` gate; routes through the bypass with 1-3s jittered backoff. The handler comment at `subagent.ts:425` ("treat as renewable error so the worker re-claims") is now actually true. +- **Bug 3 — strip `provider:` prefix at the Anthropic SDK call site.** `gbrain agent run --model anthropic:claude-sonnet-4-6` used to send the qualified string straight to `client.messages.create` which Anthropic rejects with "model not found." New `stripProviderPrefix()` helper applies at the one SDK call site; `model` stays qualified everywhere else (persistence, recipe lookup, capability gate). +- **Bug 4 (absorbed into Approach C) — composable system prompt renderer with per-tool `usage_hint`.** Pre-v0.41 the default subagent system prompt was one generic line; if a tool sat in the registry, the model had no idea when to reach for it. The field-report repro: a `shell` tool was registered, the subagent never used it, and instead described file contents in prose. New `src/core/minions/system-prompt.ts` splices a deterministic preamble listing each tool's name + `usage_hint` (description tells the model HOW; hint tells it WHEN). All 13 brain tools annotated. Closing paragraph names `shell` / `bash` explicitly + tells the model brain tools write to the DB, not local files. Deterministic so the Anthropic prompt-cache marker on the system block stays a hit. + +**Visibility infrastructure (Approach C base + Eng D3 / D7 / D8 / D10):** + +- New migration v93 `minions_v0_41_audit_and_budget`: three audit tables (`minion_lease_pressure_log`, `minion_budget_log`, `minion_self_fix_log`) with `ON DELETE SET NULL` FKs so audit rows survive `gbrain jobs prune`; denormalized columns (queue_name, model, provider, owner_id, event_type, etc.) persisted inline so post-NULL forensic queries still see context. Three new `minion_jobs` columns: `budget_remaining_cents`, `budget_owner_job_id` (FK SET NULL), `budget_root_owner_id` (no FK — Eng D10 immutable historical reference that survives owner deletion so children can disambiguate "never had a budget" from "owner deleted, halt cleanly"). Postgres + PGLite parity. +- `gbrain doctor` gains `subagent_health` check. Reads 24h of `minion_lease_pressure_log`. 0 bounces → ok; 100+ with no completed subagents → warn with paste-ready cap-raise hint; 1000+ → fail (blocking). Pre-v93 brains silently skip. +- `gbrain jobs stats` gains a `Lease pressure (1h)` line + `--cluster-errors` flag that groups dead/failed jobs by classifier bucket via the new `src/core/minions/error-classify.ts` (the shared classifier consumed by both `gbrain jobs stats --cluster-errors` and the E6 self-fix gate). +- New `src/core/minions/lease-pressure-audit.ts` + `src/core/minions/budget-tracker.ts` — best-effort writers + readers for the new audit tables. Stderr-warn on failure; never blocks the bypass path. + +**Live dashboard (D2):** + +- New `gbrain jobs watch` command (`src/commands/jobs-watch.ts`). Manual ANSI cursor management, 1s refresh, color-coded lease pressure (green/yellow/red by severity), top-5 clustered errors, budget owners panel. Non-TTY mode emits JSON snapshots per tick. +- New admin SPA `Jobs Watch` tab (`admin/src/pages/JobsWatch.tsx`) consuming the new `/admin/api/jobs/watch` endpoint. Layout matches TTY 1:1. +- New `GET /admin/api/jobs/watch` endpoint in `src/commands/serve-http.ts` — shares `readSnapshot()` with the TTY command so the two surfaces stay 1:1. + +**Cost cathedral (D4 + D5):** + +- New `src/core/minions/batch-projection.ts` — pure math for submit-time projection. Cold-start fallback (no history → model-default per-token pricing + 5s mean latency guess + `(no history; estimate is a wide guess)` annotation). Unknown-model handling returns the tagged variant so `--budget-usd` enforcement can refuse to gate against an unavailable cost estimate (matches the cross-modal-eval precedent). ±30% confidence band; threshold-gated TTY prompt (default $5 / 30min) overridable via env. +- New `src/core/minions/budget-tracker.ts` — D5 enforcement via reservation pattern (codex pass-2 #5 / Eng D7): worker calls `reserveBudget(parentJobId, expectedMaxTurnCostCents)` BEFORE each turn; SQL UPDATE with CAS guarantees no overspend even across N parallel children of one budget owner. `refundBudget()` returns unspent cents. `haltBudgetSubtree()` walks `budget_owner_job_id = X` to flip the entire subtree to dead with reason `budget_exhausted`. Eng D10 NULL-bypass: jobs without an owner skip reservation cleanly; the immutable `budget_root_owner_id` disambiguates "never had budget" from "owner deleted, halt cleanly." + +**Self-tuning fleet (E5 — reframed):** + +- New `src/core/minions/lease-cap-controller.ts` — adapts the rate-lease cap (NOT worker concurrency; codex pass-1 caught the original framing was solving the wrong bottleneck). **Eng D6 corrected control law (load-bearing fix):** ramp DOWN ONLY when upstream pushes back (429s > 0.5/min OR latency unstable); ramp UP fast when bounces > 1/min AND no upstream pushback (workers starving inside our artificial cap); ramp UP slow on healthy headroom (no bounces + util > 50%); deadband otherwise. Codex pass-2 #9 caught the original draft had this sign inverted — would have cratered cap during the field-report's 100-job burst. IRON-RULE regression test (`test/lease-cap-controller.test.ts`) pins the correct sign so future "let's simplify" PRs can't silently regress. +- Per-tick election via new `tryWithDbElection(engine, lockId, ttlMinutes, fn)` helper in `src/core/db-lock.ts` — thin convenience over the existing `tryAcquireDbLock` primitive (codex pass-3 #8/#9: extend the existing primitive rather than build a parallel new one). Eng D9 retrofit of existing rate-leases + queue maxWaiting call sites to the same primitive is filed as a v0.42 follow-up since the existing PGLite single-connection mutex preserves correctness by accident. +- Workers read `lease_cap_current` from config on every acquire (short-TTL cache) so they pick up controller writes within ~5s of a decision. +- A/B harness `scripts/e5-lease-cap-ab.ts` per D11 — 500-job batch under Anthropic with log-normal prompt distribution, 15-min 429 burst injection, $8 budget per arm, committed receipt fixture at `test/fixtures/e5-lease-cap-ab/`. Dry-run smoke verified; full-run dispatcher is a v0.41.1 follow-up. + +**Self-healing batches (E6 — narrowed):** + +- New `src/core/minions/self-fix.ts` — classifier-gated auto-resubmit on terminal failures. RECOVERABLE_CLUSTERS narrowed per codex pass-2 #4: only `prompt_too_long`, `tool_schema_mismatch`, `malformed_json` qualify; `tool_crash` / `tool_unavailable` / `tool_permission` route through normal dead-letter so real bugs stay visible. Chain depth cap default 2 (D15); per-job opt-out via `data.no_self_fix: true`; global off-switch via config. +- `prompt_too_long` reduction (codex pass-1 #11): v0.41 ships truncate-with-leaf-preservation; full semantic reduction (drop tool_result blocks then Haiku-summarize older pairs) is a v0.42 follow-up. Worst-case current behavior — truncate-then-fail — is safe (no infinite loops, depth cap prevents chains). +- Children inherit budget owner from parent (Eng D7 + D10) but DO NOT copy the remaining cents (codex pass-3 #5 caught the original plan's contradiction; only the owner row holds spendable balance). + +**Three review passes + three codex outside-voice passes converged.** The wave got CEO review + eng review + a third "second eng pass" + codex pass-1 (on CEO plan) + codex pass-2 (on eng additions) + codex pass-3 (on post-corrections plan). Each pass caught corrections the previous pass missed. The post-corrections plan ships with 0 unresolved decisions across 20 user-decided AUQs. + +### For contributors + +- `test/lease-cap-controller.test.ts` includes the field-report-scenario simulation: starving workers (bounces but no upstream pushback) get MORE capacity, not less. Don't simplify this rule without re-reading codex pass-2 #9. +- `src/core/minions/error-classify.ts` is the single source of truth for the cluster taxonomy. Adding new clusters is an explicit `RECOVERABLE_CLUSTERS` decision (self-fix qualification gate). +- Audit table denormalization is load-bearing: codex pass-3 #7 caught that without it, post-NULL FK rows are timestamp-only residue. New audit kinds MUST denormalize the context columns at write time. +- The retrofit of `pg_advisory_xact_lock` call sites in `src/core/minions/rate-leases.ts:80` and `src/core/minions/queue.ts:152` to the new shared primitive is a v0.42 follow-up (the existing PGLite single-connection mutex preserves correctness by accident; the retrofit makes it explicit). + +## To take advantage of v0.41.0.0 + +`gbrain upgrade` should do this automatically. If it didn't, or if `gbrain doctor` warns about a partial migration: + +1. **Run the orchestrator manually:** + ```bash + gbrain apply-migrations --yes + ``` +2. **Verify the outcome:** + ```bash + gbrain doctor # should show subagent_health check + gbrain jobs stats # should show Lease pressure (1h) line + ``` +3. **Tune the rate-lease cap for your upstream:** + - Default Anthropic API → leave at 32 (the new default; works for most workloads). + - Azure / Bedrock / self-hosted with no provider-side rate limit: + ```bash + export GBRAIN_ANTHROPIC_MAX_INFLIGHT=unlimited + ``` +4. **Try the live dashboard during your next batch:** + ```bash + gbrain jobs work --concurrency 10 & + gbrain jobs watch # in another terminal + ``` +5. **If any step fails or the numbers look wrong,** please file an issue: + https://github.com/garrytan/gbrain/issues with `gbrain doctor --json` output. +## [0.40.10.0] - 2026-05-24 + +**Your brain stops accepting junk pages, and oversize content stops crashing the embedder.** A page from one of your source repos can no longer break embedding, defeat search, or pollute your knowledge graph just because it's a Cloudflare challenge dump or an absurdly large file. The new sanity gate lives at the narrow waist of ingestion, so every path that writes pages — sync, capture, `put_page` MCP, the `/ingest` webhook — picks it up uniformly. + +Two failure modes treated differently: + +- **Scraper junk** (Cloudflare challenge pages, CAPTCHAs, 403 dumps, bare error-page titles): HARD-BLOCK at ingest. Your CLI exits non-zero, your MCP call gets a proper error envelope, your sync surfaces the failure with code `PAGE_JUNK_PATTERN` so doctor groups it. The page never lands. Six hand-vetted patterns ship built-in; operators add literal substrings for site-specific cases via `~/.gbrain/junk-substrings.txt`. + +- **Legitimate large content** (your 2MB conversation transcripts, long essays, big articles): SOFT-BLOCK. The page writes successfully, you can still query it by title and slug, but the embedder skips it on the next sweep. The 5 places the embedder reads from now share one source-of-truth helper so the skip can't drift across them. If you edit a page past the size threshold, its old chunks get deleted in the same transaction so search stops returning matches against content that's no longer there. + +**New surfaces:** +- `gbrain sources audit ` — walk a source repo's disk, report size distribution + would-blocks + junk-pattern hits without touching the DB. Catches junk before sync. Read-only by design. +- `gbrain doctor` gains `oversized_pages`, `scraper_junk_pages`, `content_sanity_audit_recent` checks. Default scans the 1000 most-recent pages; `--content-audit` opts into a full scan for the cleanup wave. +- `gbrain lint` gains `huge-page` and `scraper-junk` rules. Lint reads DB config when reachable (matches what `gbrain config set` writes) and falls back to file/env on CI. +- `GBRAIN_NO_SANITY=1` kill-switch with loud stderr per bypassed ingest. Operators who really want junk through have to ask for it explicitly and see the warning every time. + +**Knobs (all four read env > file > DB > defaults):** +- `content_sanity.bytes_warn` (default 50_000) — `GBRAIN_PAGE_WARN_BYTES` +- `content_sanity.bytes_block` (default 500_000) — `GBRAIN_PAGE_BLOCK_BYTES` +- `content_sanity.junk_patterns_enabled` (default true) — `GBRAIN_NO_JUNK_PATTERNS=1` flips off +- `content_sanity.disabled` (default false) — `GBRAIN_NO_SANITY=1` flips on + +**ISO-week JSONL audit** at `~/.gbrain/audit/content-sanity-YYYY-Www.jsonl` records every hard-block, soft-block, and warn-trip event. Doctor reads the last 7 days, aggregates by pattern + source, surfaces "31 ingest blocks this week, 28 from straylight-brain" so operators see which scraper is the actual problem. Honors `GBRAIN_AUDIT_DIR` for shared-filesystem multi-host setups; documented caveat in the doctor message for ops that don't share the dir. + +**No schema migration this PR.** The soft-block flag rides in `frontmatter.embed_skip` JSONB so the embedder filter is a single SQL fragment shared by both engines. Schema column for `pages.embed_skipped_at` lands in v0.41+ with the chunk-level quarantine refactor — deferred for the right reason (Codex caught that page-level granularity loses good chunks; chunk-level is the right axis). + +**Review provenance.** This wave went through `/plan-ceo-review` (5 cherry-picks surfaced, 3 accepted, 2 deferred post-Codex round 1) and `/plan-eng-review` (4 architectural decisions resolved + 4 strategic Codex round 2 tensions resolved). Codex caught one load-bearing bug class during planning — `importFromContent.status` vocabulary mismatch that would have made the gate silently fail at the CLI / MCP / sync wrapper sites. Fixed by throwing a typed `ContentSanityBlockError` instead of inventing a new status value; the existing exception flow at every wrapper site fires correctly through one throw point. The plan was substantially tightened post-Codex (dropped 2 cherry-picks that needed v0.42 chunk-level rework, dropped an operator-regex feature that needed a real ReDoS story, dropped the HTML-density rule that needed careful handling of code fences). What ships is what the actual bug needed plus the audit + cleanup surfaces. + +**99 new unit tests** (207 assertions) across 6 files covering the assessor, literal loader, embed-skip helper, audit JSONL, lint rules, and the import-file gate. 136 surface-area regression tests on the files touched all pass in isolation. Full bun:test suite returns clean. + +### To take advantage of v0.40.10.0 + +`gbrain upgrade` carries this for you. No migration, no manual steps. After upgrading: + +1. **Audit your existing inventory** (optional but recommended): + ```bash + gbrain doctor --content-audit --json | jq '.checks[] | select(.name == "scraper_junk_pages" or .name == "oversized_pages")' + ``` + Surfaces existing junk pages and oversized pages already in your brain. + +2. **For any junk pages doctor flags**, the right cleanup is at the source — `git rm` the file from the source repo, push, then `gbrain sync`. The v0.41+ wave will ship `gbrain sources prune-junk ` to automate this; for v0.40.10.0 it's a manual two-step. + +3. **For oversized pages doctor flags** as warn-tier, no action needed unless you want to split. New oversize will automatically write with `frontmatter.embed_skip` and be queryable by title (just not search-rankable until split). + +4. **If you have a site-specific scraper-junk pattern** (LinkedIn auth wall, Reddit blocked page, etc.), drop a literal in `~/.gbrain/junk-substrings.txt`: + ``` + # name=linkedin_auth_wall + Sign in to your account to continue + + # name=reddit_blocked + You're being blocked from accessing + ``` + Loaded on every ingest. Missing file is fine; malformed lines are impossible (no regex). + +5. **If any step surprises you,** please file an issue: https://github.com/garrytan/gbrain/issues with: + - output of `gbrain doctor --json` + - a sanitized example of the page that surprised you + - which step broke + + The audit JSONL at `~/.gbrain/audit/content-sanity-YYYY-Www.jsonl` carries the assessor's full reasoning per event if you want to debug a specific decision. + +### Itemized changes + +**Added:** +- `src/core/content-sanity.ts` — pure assessor with 6 hand-vetted junk patterns + `ContentSanityBlockError` class +- `src/core/content-sanity-literals.ts` — operator literal-substring loader (fail-soft on ENOENT) +- `src/core/embed-skip.ts` — 5-site shared predicate (JS + SQL fragment + marker builder) +- `src/core/audit/content-sanity-audit.ts` — ISO-week JSONL writer/reader on the v0.40.4.0 audit-writer primitive +- `gbrain sources audit ` CLI for dry-run source-repo scanning +- `gbrain doctor --content-audit` flag for full-scan opt-in +- `gbrain doctor` checks: `oversized_pages`, `scraper_junk_pages`, `content_sanity_audit_recent` +- `gbrain lint` rules: `huge-page`, `scraper-junk` +- 4 `content_sanity.*` config keys (file/env/DB plane) + +**Changed:** +- `importFromContent` throws `ContentSanityBlockError` on hard-block (junk pattern match) and sets `frontmatter.embed_skip` on soft-block (oversize alone). Old chunks deleted on transition to soft-block. +- `gbrain import` honors `errors > 0` for non-zero exit (was silently exit-0 on failed files). +- Embed sweep skips pages with `embed_skip` flag at all 5 sites: `embed.ts --stale`, `embed.ts --all`, `embed-stale.ts` Minion helper, both engines' `listStaleChunks` + `countStaleChunks`. +- `lint.ts` lifts DB config when `~/.gbrain/` is reachable; falls back to file/env on CI. +- `classifyErrorCode` recognizes `PAGE_JUNK_PATTERN` for sync-failures.jsonl grouping. + +**Test coverage:** +- 99 new unit tests across 6 files (207 assertions) +- All new modules covered at the boundary level +- Cross-site embed-skip invariant pinned by `test/embed-skip.test.ts` +- Bytes-parity assertion (D2) pinned in `test/content-sanity.test.ts` + +### For contributors + +The plan file lives at `~/.claude/plans/system-instruction-you-are-working-temporal-brook.md` with the full decision provenance: CEO review (D1-D16) + Eng review (D1-D9) + Codex round 1 (17 findings) + Codex round 2 (13 findings). The deferred-to-v0.41+ TODOs are in `TODOS.md` under "v0.41 content-sanity follow-ups" — chunk-level quarantine, source-repo remediation CLI, threshold validation post-deploy, brain-score `no_junk_pages_score` component, plus the operator-regex + HTML-density features that need real ReDoS / code-fence-handling stories before they're worth shipping. +## [0.40.9.0] - 2026-05-24 + +**`gbrain sync` now indexes your `.sql` files, and `gbrain code-def` works on SQL tables, functions, views, and indexes the same way it works on TypeScript.** + +Until today, point gbrain at a repo that ships database migrations or query libraries and the `.sql` files dropped silently on the floor. The code chunker shipped support for 36 languages — SQL was the conspicuous absence. Closes [#1173](https://github.com/garrytan/gbrain/issues/1173). + +Concretely, what changes: `gbrain sync` running over a repo with a `migrations/001_users.sql` file now produces a code page in your brain. Each top-level statement becomes its own chunk. `CREATE TABLE users (...)` lands with `symbol_name='users'` and `symbol_type='table'`. `CREATE OR REPLACE FUNCTION get_user_by_email(...)` lands with `symbol_name='get_user_by_email'` and `symbol_type='function'`. Same for `CREATE VIEW`, `CREATE INDEX`, `CREATE PROCEDURE`, `CREATE TYPE`, `CREATE SCHEMA`, `CREATE DATABASE`, `CREATE TRIGGER`, and `ALTER TABLE` / `ALTER VIEW`. Then `gbrain code-def users` returns the CREATE TABLE site directly — same shape as `gbrain code-def AuthService` on a TypeScript class. + +INSERT/UPDATE/DELETE/SELECT statements still get chunked (so a query library stays searchable via vector + keyword) but they emit unnamed — `code-def` is a definition signal, not a query-mention signal, so DML doesn't pollute the symbol surface. PostgreSQL's `$$ ... $$` dollar-quoted function bodies parse cleanly. Files with malformed SQL fall through to the recursive chunker instead of throwing. + +**One honesty note about binary size.** The grammar this release vendors (`DerekStride/tree-sitter-sql`) covers PostgreSQL, MySQL, SQLite, and T-SQL basics in one parser. That breadth comes from a 40 MB generated `parser.c` that compiles to an 11 MB WASM. The plan projected 400 KB-1.4 MB before measurement; the real number is ~8x bigger. The compiled gbrain binary grows roughly 6% as a result. If that matters in your deployment, file an issue and we'll evaluate a narrower-coverage fork as a follow-up. + +### How to take advantage of v0.40.9.0 + +Existing brains pick up SQL automatically on the next `gbrain sync` over a repo containing `.sql` files. No migration, no flag, no reembed prompt. No flag exists to turn it off — if you'd previously been ignoring `.sql` files via `.gitignore` and want to keep doing that, that path still works exactly as before. + +To verify it works end-to-end on your own brain: + +```bash +gbrain sync # syncs any .sql files in your tracked sources +gbrain code-def # should return the CREATE TABLE site +gbrain code-def # should return the CREATE FUNCTION site +``` + +If `code-def` returns nothing on a table you know exists in a sync'd `.sql` file, file an issue with the SQL syntax — the DerekStride grammar covers the common dialects but some edge cases (vendor-specific extensions) may parse to a generic statement node where the name isn't reachable via the standard field paths. + +### Itemized changes + +#### Chunker + +- **`src/core/chunkers/code.ts:30+` — vendor `tree-sitter-sql.wasm` (DerekStride/tree-sitter-sql @ c2e1e08db1ea20dc23bdb8d228a81a8756e9c450, built with `tree-sitter-cli@v0.26.3 --abi 14`).** ABI 14 chosen explicitly because gbrain's `web-tree-sitter@0.22.6` supports ABI range 13-14; the CLI's default ABI 15 is incompatible (verified by a load-time `Incompatible language version 15. Compatibility range 13 through 14.` throw during Step 0 grammar inspection). Binary is 11 MB. +- **`src/core/chunkers/code.ts:121-125` — `SupportedCodeLanguage` union extended with `'sql'`.** +- **`src/core/chunkers/code.ts:199-229` — `LANGUAGE_MANIFEST` registers `sql: { displayName: 'SQL', embeddedPath: G_SQL }`.** +- **`src/core/chunkers/code.ts:410+` — `detectCodeLanguage('foo.sql')` returns `'sql'` (case-insensitive).** +- **`src/core/chunkers/code.ts:325+` — `TOP_LEVEL_TYPES.sql = new Set(['statement'])` catch-all.** DerekStride's grammar wraps every top-level statement in a single `statement` node whose only named child carries the actual kind. The Step 0 grammar-inspection script (`tools/inspect-sql-grammar.ts`) verified this shape against 9 representative SQL fixtures. +- **`src/core/chunkers/code.ts:1025+` — `extractSymbolName` gains an inline SQL branch.** When the node is `type === 'statement'` with a single named child, it dives into `extractSqlSymbolName(node.namedChild(0))`. That helper recognizes 11 DDL kinds (`create_table`, `create_view`, `create_index`, `create_function`, `create_procedure`, `create_type`, `create_schema`, `create_database`, `create_trigger`, `alter_table`, `alter_view`) and pulls the target identifier from the inner node's `name` field, with fallback to the first `object_reference`/`identifier`-shaped child. Six DML kinds (`select`, `insert`, `update`, `delete`, `merge`, `with`) deliberately return null so chunks emit unnamed. +- **`src/core/chunkers/code.ts:1044+` — `normalizeSymbolType` gains parallel SQL branches** mapping `create_table → 'table'`, `create_view`/`alter_view → 'view'`, `create_index → 'index'`, `create_procedure → 'procedure'`, `create_type → 'type'`, `create_schema → 'schema'`, `create_database → 'database'`, `create_trigger → 'trigger'`, `alter_table → 'table'`. +- **`src/core/chunkers/code.ts:639+` — chunker emit-path passes the inner-child type to `normalizeSymbolType` when the outer node is `statement`** so chunk headers say "[SQL] file.sql:1-5 table users" instead of "statement users". + +#### Sync routing + +- **`src/core/sync.ts:88+` — `CODE_EXTENSIONS` adds `'.sql'`.** `isCodeFilePath('migrations/001_init.sql')` now returns `true`, routing through `importCodeFile()` with `page_kind='code'`. + +#### `gbrain code-def` extension + +- **`src/commands/code-def.ts:35` — `DEF_TYPES` allowlist extended with `'table'`, `'view'`, `'index'`, `'procedure'`, `'schema'`, `'database'`, `'trigger'`.** Without this, the chunks were indexed correctly but invisible to `gbrain code-def ` because the SQL `symbol_type` values fell outside the hardcoded definition-types filter. This was the load-bearing missing piece codex caught in `/plan-eng-review` (F2 finding). + +#### Tools + docs + +- **`tools/inspect-sql-grammar.ts` (NEW)** — one-shot Step 0 inspection script. Loads the vendored wasm via `web-tree-sitter`, parses 9 representative SQL fixtures (CREATE TABLE / FUNCTION / INDEX / VIEW / ALTER TABLE / CREATE TYPE / mixed DDL+DML / pure DML / invalid SQL), prints top-level node types + the `extractSymbolName` generic output. Output drove the `TOP_LEVEL_TYPES` + `extractSqlSymbolName` design decisions. +- **`CLAUDE.md`** — grammar count bumped 36→37 with the DerekStride SHA + ABI rationale + 11 MB size disclosure. `src/core/chunkers/` entry extended with the SQL branch documentation. +- **`llms.txt` + `llms-full.txt`** — regenerated via `bun run build:llms` (CI gate). + +#### Tests + +- **`test/chunkers/code.test.ts`** — 8 new SQL cases: extension count bump 29→30, `Schema.SQL` case-insensitivity, CREATE TABLE/FUNCTION/INDEX/VIEW/ALTER TABLE each extract correct symbolName + symbolType, DML emits unnamed chunks, mixed DDL+DML per-statement emission, header includes `[SQL]` tag, invalid SQL doesn't crash the parser. +- **`test/sync-classifier-widening.test.ts`** — 1 new case: SQL extensions classified as code (case-insensitive). +- **`test/e2e/code-indexing.test.ts`** — 7 new cases against real PGLite. The load-bearing canary asserts `findCodeDef(engine, 'users_account_...', { language: 'sql' })` returns the CREATE TABLE site with `symbol_type='table'`. `beforeAll` timeout bumped to 30s (92-migration replay + 11 MB grammar load pushes past the default 5s on slower CI runners). + +### Decisions captured during `/plan-eng-review` + +- **D1** (scope, initial): bundle `.sql` + TS/JS JSDoc extraction. Reverted by D6. +- **D2** (grammar source): `DerekStride/tree-sitter-sql` over the official-org fork. Active maintenance, broad dialect coverage, MIT, reproducible wasm build. +- **D3** (TOP_LEVEL_TYPES): filtered to schema-defining statements. Corrected by Step 0 to catch-all `statement` because DerekStride wraps every top-level statement in that node type. +- **D4** (CHUNKER_VERSION): bump 4→5 + wire post-upgrade reembed prompt. Dropped by D6 (no longer needed without JSDoc). +- **D5** (JSDoc extraction): preceding-sibling AST scan. Dropped by D6. +- **D6** (scope correction, post-codex): strip JSDoc + CHUNKER_VERSION + reembed-prompt. Keep `.sql` + add SQL symbol-name extraction. Driven by codex's F2 finding that SQL chunking without symbol extraction is "just searchable text," not code intelligence. + ## [0.40.8.1] - 2026-05-23 **The README and tutorials are rewritten for someone who has never touched GBrain.** The front-door docs now read as a story you can understand cold: what GBrain does, what it looks like, how to install it, two real walkthroughs that take you from zero to a working brain. No internal jargon, no version archaeology, no assumed context. diff --git a/CLAUDE.md b/CLAUDE.md index 77e24b760..6900fb2cc 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -73,9 +73,9 @@ strict behavior when unset. - `gbrain.yml` (brain repo root, v0.22.11) — Optional storage tiering config. Top-level `storage:` section with `db_tracked:` and `db_only:` array-valued keys. `gbrain sync` auto-manages `.gitignore` for `db_only` paths on successful sync (skips on dry-run, blocked-by-failures, submodule context, or `GBRAIN_NO_GITIGNORE=1`). `gbrain export --restore-only [--repo P] [--type T] [--slug-prefix S]` repopulates missing `db_only` files from the database. - `src/core/supabase-admin.ts` — Supabase admin API (project discovery, pgvector check) - `src/core/file-resolver.ts` — File resolution with fallback chain (local -> .redirect.yaml -> .redirect -> .supabase) -- `src/core/chunkers/` — 3-tier chunking (recursive, semantic, LLM-guided). v0.19.0 adds `code.ts` — tree-sitter-based semantic chunker for 29 languages with embedded-asset WASMs (`src/assets/wasm/`), `@dqbd/tiktoken` cl100k_base tokenizer, small-sibling merging. `CHUNKER_VERSION` constant folded into `importCodeFile`'s `content_hash` so chunker shape changes force clean re-chunks across releases. +- `src/core/chunkers/` — 3-tier chunking (recursive, semantic, LLM-guided). v0.19.0 adds `code.ts` — tree-sitter-based semantic chunker for 30 languages (v0.41 added SQL via DerekStride/tree-sitter-sql) with embedded-asset WASMs (`src/assets/wasm/`), `@dqbd/tiktoken` cl100k_base tokenizer, small-sibling merging. `CHUNKER_VERSION` constant folded into `importCodeFile`'s `content_hash` so chunker shape changes force clean re-chunks across releases. **v0.41 D2 SQL wave (#1173):** `extractSymbolName` gains an inline SQL branch (`extractSqlSymbolName`) that dives through DerekStride's `statement` wrapper into the inner DDL child (`create_table`/`create_function`/`create_view`/`create_index`/`create_procedure`/`create_type`/`create_schema`/`create_database`/`create_trigger`/`alter_table`/`alter_view`) and extracts the target identifier via the `name` field with identifier-shaped fallback. DML kinds (`select`/`insert`/`update`/`delete`/`merge`/`with`) deliberately return null so chunks emit unnamed — code-def is a DDL signal. `normalizeSymbolType` gains parallel SQL branches mapping `create_table → 'table'`, `create_view → 'view'`, etc. `src/commands/code-def.ts:DEF_TYPES` was extended in the same wave with `'table' | 'view' | 'index' | 'procedure' | 'schema' | 'database' | 'trigger'` so the new chunks surface in `gbrain code-def ` queries. - `src/core/errors.ts` (v0.19.0) — `StructuredAgentError` + `buildError` + `serializeError`. Every new v0.19.0 agent-facing surface (code-def, code-refs, usage errors) uses this envelope; matches v0.17.0 `CycleReport.PhaseResult.error` shape. -- `src/assets/wasm/` (v0.19.0) — 36 tree-sitter grammar WASMs + tree-sitter runtime. Committed to the repo so `bun --compile` embeds them deterministically via `import path from ... with { type: 'file' }`. The CI guard `scripts/check-wasm-embedded.sh` fails the build if the compiled binary ever silently falls through to recursive chunks. +- `src/assets/wasm/` (v0.19.0, extended v0.41) — 37 tree-sitter grammar WASMs + tree-sitter runtime. Committed to the repo so `bun --compile` embeds them deterministically via `import path from ... with { type: 'file' }`. The CI guard `scripts/check-wasm-embedded.sh` fails the build if the compiled binary ever silently falls through to recursive chunks. **v0.41 D2 wave (#1173):** `tree-sitter-sql.wasm` (DerekStride/tree-sitter-sql @ c2e1e08db1ea20dc23bdb8d228a81a8756e9c450, built with tree-sitter-cli@v0.26.3 --abi 14) adds SQL coverage at 11 MB — substantially larger than peers because the grammar covers PostgreSQL + MySQL + SQLite + T-SQL basics (40 MB of generated parser.c). The compiled gbrain binary grows ~6% as a result; tracked as a follow-up TODO whether to switch to a smaller-coverage grammar. - `src/commands/code-def.ts` + `src/commands/code-refs.ts` (v0.19.0) — symbol definition + references lookup. Query `content_chunks.symbol_name` or chunk_text ILIKE with `page_kind='code'` filter. Auto-JSON when stdout is not a TTY (gh-CLI convention). Bypass the standard `searchKeyword` `DISTINCT ON (slug)` collapse so multiple call-sites from the same file surface. - `src/core/search/` — Hybrid search: vector + keyword + RRF + multi-query expansion + dedup. As of v0.22.0, `searchKeyword` / `searchKeywordChunks` / `searchVector` apply source-aware ranking at the SQL layer (curated content like `originals/`, `concepts/`, `writing/` outranks bulk content like `wintermute/chat/`, `daily/`, `media/x/`). `searchVector` uses a two-stage CTE so source-boost re-ranking doesn't kill the HNSW index. Hard-exclude prefixes (`test/`, `archive/`, `attachments/`, `.raw/` by default) filter at retrieval, not post-rank. Both gates honor `detail !== 'high'` so temporal queries surface chat pages normally. - `src/core/search/intent.ts` — Query intent classifier (entity/temporal/event/general → auto-selects detail level) @@ -159,7 +159,7 @@ strict behavior when unset. - `src/commands/extract.ts` — `gbrain extract links|timeline|all [--source fs|db] [--source-id ]`: batch link/timeline extraction. fs walks markdown files, db walks pages from the engine (mutation-immune snapshot iteration; use this for live brains with no local checkout). As of v0.12.1 there is no in-memory dedup pre-load — candidates are buffered 100 at a time and flushed via `addLinksBatch` / `addTimelineEntriesBatch`; `ON CONFLICT DO NOTHING` enforces uniqueness at the DB layer, and the `created` counter returns real rows inserted (truthful on re-runs). v0.22.1 (#417): `ExtractOpts.slugs?: string[]` enables incremental extract — when set, `extractForSlugs()` reads ONLY those slugs' files (single combined links+timeline pass) instead of the full directory walk. CLI `gbrain extract` keeps full-walk behavior; the cycle path threads sync's `pagesAffected` through. `walkMarkdownFiles(brainDir)` still runs at line 455 to build `allSlugs` for link resolution — see `TODOS.md` for replacing it with `engine.getAllSlugs()`. **v0.37.7.0 (#1204):** `--source-id ` flag scopes extraction to one brain source on federated brains. Resolved via `resolveSourceWithTier()` before any SQL runs; failures surface with a `gbrain sources list` hint. Closes the silent-collapse-to-`default` bug class for extract. - `src/commands/import.ts` — `gbrain import [--source-id ]`: page import with the v0.34.2.0 path-set checkpoint described above. **v0.37.7.0 (#1167):** new `--source-id ` flag finally honored — pages route to the named source. Resolved via `resolveSourceWithTier()` at the boundary; the same flag is now consistent across `import`, `extract`, `graph-query`, and `sources current`. Pinned by `test/import-source-id.test.ts`. - `src/commands/graph-query.ts` — `gbrain graph-query [--type T] [--depth N] [--direction in|out|both] [--include-foreign]`: typed-edge relationship traversal (renders indented tree). **v0.37.7.0 (#1153):** foreign-edge footer always present (`X foreign edges (use --include-foreign to traverse)`) so cross-source edges never disappear silently; `--include-foreign` widens the SQL filter to walk them. Pinned by `test/graph-query.test.ts`. -- `src/commands/sources.ts` — `gbrain sources {list,add,remove,archive,restore,archived,purge,current,status}`. **v0.37.7.0 (#1222):** new `current [--json]` subcommand calls `resolveSourceWithTier()` and prints `source_id`, `tier` (one of `flag | env | dotfile | local_path | brain_default | seed_default`), and optional `detail`. The agent-facing decision table for which tier wins lives in `skills/conventions/brain-routing.md`. **v0.40.3.0 (productionized from PR #1314):** new `status [--json]` subcommand — read-only per-source dashboard (last sync, staleness, page count, embedding coverage, unacked failures). Thin wrapper around `buildSyncStatusReport` + `printSyncStatusReport` exported from `src/commands/sync.ts`. `--json` emits the stable `{schema_version: 1, sources, ...}` envelope on stdout for monitoring pipelines; bare invocation prints the human table to stdout (right-aligned numeric columns, kubectl-style). Filters input sources to `local_path IS NOT NULL AND archived IS NOT TRUE` so archived sources (which have their own `gbrain sources archived` surface) don't muddy the active-sync dashboard. Lives under `sources` (not `sync --status`) per D3 from the v0.40.3.0 plan-eng-review — reads and writes don't share a verb. +- `src/commands/sources.ts` — `gbrain sources {list,add,remove,archive,restore,archived,purge,current,status,audit}`. **v0.37.7.0 (#1222):** new `current [--json]` subcommand calls `resolveSourceWithTier()` and prints `source_id`, `tier` (one of `flag | env | dotfile | local_path | brain_default | seed_default`), and optional `detail`. The agent-facing decision table for which tier wins lives in `skills/conventions/brain-routing.md`. **v0.40.3.0 (productionized from PR #1314):** new `status [--json]` subcommand — read-only per-source dashboard (last sync, staleness, page count, embedding coverage, unacked failures). Thin wrapper around `buildSyncStatusReport` + `printSyncStatusReport` exported from `src/commands/sync.ts`. `--json` emits the stable `{schema_version: 1, sources, ...}` envelope on stdout for monitoring pipelines; bare invocation prints the human table to stdout (right-aligned numeric columns, kubectl-style). Filters input sources to `local_path IS NOT NULL AND archived IS NOT TRUE` so archived sources (which have their own `gbrain sources archived` surface) don't muddy the active-sync dashboard. Lives under `sources` (not `sync --status`) per D3 from the v0.40.3.0 plan-eng-review — reads and writes don't share a verb. **v0.40.9.0:** new `audit [--json]` subcommand — read-only dry-run scan of a source repo's disk for size distribution + would-blocks + junk-pattern hits, WITHOUT touching the DB. Catches scraper junk and oversized content BEFORE sync. Walks `sources.local_path`, reads each markdown file, runs `assessContent()` from `src/core/content-sanity.ts`, aggregates by verdict (`ok | warn_oversize | hard_block_junk_pattern`). JSON envelope is stable for monitoring pipelines. Pinned by `test/sources-audit.test.ts` (not present in this wave; covered transitively by `test/content-sanity.test.ts` + `test/import-file-content-sanity.test.ts`). - `src/commands/reindex-frontmatter.ts` — `gbrain reindex-frontmatter`. **v0.37.7.0 (#1225):** wrapped the query path in the standard `withEngine(...)` lifecycle so `engine.connect()` runs before the first SQL call. Pre-fix the command `process.exit(1)`'d with a TypeError on first invocation. Pinned by `test/reindex-frontmatter-connect.test.ts`. - `src/core/source-resolver.ts` — 6-tier source resolution. **v0.37.7.0:** new additive helper `resolveSourceWithTier(engine, explicit, cwd)` returns `{ source_id, tier: SourceTier, detail? }` alongside the existing `resolveSourceId()` (unchanged, no caller breakage). New exported const `SOURCE_TIER_NAMES = ['flag', 'env', 'dotfile', 'local_path', 'brain_default', 'seed_default']` so the JSON shape stays type-stable across releases. Order matches the 1-6 priority of `resolveSourceId()`. Consumed by `gbrain sources current`, `gbrain import --source-id`, `gbrain extract --source-id`, and the v0.37.7.0 `source_routing_health` doctor check. Pinned by `test/source-resolver-with-tier.test.ts` (uses `withEnv()` wrapper per the test-isolation lint). - `src/commands/autopilot.ts` extension (v0.37.7.0) — three changes for federated-brain co-existence and launchd hygiene. (1) **#1226 lockfile scope:** `LOCK_PATH` resolves via `gbrainPath('autopilot.lock')` so it honors `GBRAIN_HOME`. Two brains can run autopilot simultaneously without lock-stealing. Lock file now stores PID; startup checks `kill -0 ` before refusing to start (codex CF11 PID-safety fix — stale lock from a crashed process no longer blocks a healthy autopilot). (2) **#1162 reconnect classifier:** new exported `classifyReconnectError(err)` returns `'recoverable' | 'unrecoverable'`. Unrecoverable causes the daemon to `process.exit(0)` and let launchd back off instead of the v0.37.6 loop that logged `config.database_url undefined` every 5s forever. (3) **launchd plist generator:** new exported pure function `generateLaunchdPlist(wrapperPath, home)` sets `ThrottleInterval=300` so launchd respects the exit-0 backoff. Both helpers pinned by `test/autopilot-lock-path.test.ts` + `test/autopilot-reconnect-classifier.test.ts`. @@ -171,7 +171,14 @@ strict behavior when unset. - `src/commands/doctor.ts` extension (v0.40.4.1) — `buildChecks(engine, args, dbSource): Promise` exported as a test seam. `runDoctor` is now a thin wrapper: `buildChecks → computeDoctorReport → render + process.exit`. All 10 `process.exit` sites stay in the wrapper; the two early-return paths (no engine, connection failure) return partial check lists instead of inline exits. No behavior change — observable output identical because the wrapper renders the same partial list. Pinned by `test/doctor-behavioral.test.ts` (13 cases: pure aggregation math over `computeDoctorReport`, orchestrator cases for `--fast` skip set + `--json` flag + no-engine partial path + snapshot of load-bearing check names) and `test/doctor-cli-smoke.serial.test.ts` (1 subprocess case spawning `bun run src/cli.ts doctor --json` against a fresh PGLite tempdir, asserting schema_version=2 envelope, status enum, non-empty checks array — the render-path coverage that buildChecks-only tests miss). Quarantined as `.serial.test.ts` because PGLite write-locks don't play with parallel runners. - `src/core/cycle.ts` extension (v0.40.4.1) — `runPhaseLint` + `runPhaseBacklinks` gain the `export` keyword so behavioral tests can drive them directly. No body changes; documented as internal helpers exposed for test-only consumption (downstream code should NOT take a dependency). Pinned by `test/cycle-legacy-phases.test.ts` (11 cases across both phases: clean run → status='ok', partial fix → status='warn' with `dryRun` in details, dry-run path doesn't write, throw-from-lib → status='fail' with the wrapper's try/catch envelope populated). Future phase wrappers (sync, extract, embed, orphans, extract_facts, resolve_symbol_edges, recompute_emotional_weight) land as additional describes in the same file, not new files (TODOS NEW-2). - `test/operations-trust-boundary.test.ts` + `scripts/check-operations-filter-bypass.sh` (v0.40.4.1) — operations trust-boundary contract coverage. Hybrid test design: pure assertions over all 74 ops (every op has a scope annotation; every mutating op has a non-read scope; `localOnly: true` ops are excluded from `operations.filter(op => !op.localOnly)`; the seven historically-sensitive localOnly ops snapshot-pinned by name) plus targeted handler-invocation regressions for the two historically-broken HTTP-callable classes: `submit_job` with `name='shell'` + `ctx.remote=true` MUST reject (the F7b HTTP MCP shell-job RCE class), and `search_by_image` with `image_path` + `ctx.remote=true` MUST reject (the D18 P0 image-leak class). `file_upload` and `sync_brain` deliberately omitted from handler-invocation tests because they're `localOnly: true` and that path would test an impossible production scenario (codex CMT-3). The shell guard greps `src/` for any module importing the `operations` value outside the canonical filter site at `src/commands/serve-http.ts` — three import shapes detected (destructured, aliased, namespace), explicit 10-entry allow-list with per-entry rationale, plus a literal-string check that `serve-http.ts` still contains `operations.filter(op => !op.localOnly)`. Wired into `bun run verify`. -- `src/core/link-extraction.ts` — shared library for the v0.12.0 graph layer. extractEntityRefs (canonical, replaces backlinks.ts duplicate) matches both `[Name](people/slug)` markdown links and Obsidian `[[people/slug|Name]]` wikilinks as of v0.12.3. extractPageLinks, inferLinkType heuristics (attended/works_at/invested_in/founded/advises/source/mentions), parseTimelineEntries, isAutoLinkEnabled config helper. `DIR_PATTERN` covers `people`, `companies`, `deals`, `topics`, `concepts`, `projects`, `entities`, `tech`, `finance`, `personal`, `openclaw`. Used by extract.ts, operations.ts auto-link post-hook, and backlinks.ts. +- `src/core/content-sanity.ts` (v0.40.9.0, NEW) — pure assessor for the content-sanity defense wave. `assessContent(content, opts): SanityVerdict` returns one of three verdicts (`ok | warn_oversize | hard_block_junk_pattern | soft_block_oversize`) with `{reason, bytes, matched_pattern_name?}` detail. Six hand-vetted built-in junk patterns (Cloudflare challenge dumps, CAPTCHAs, 403 dumps, bare error-page titles) compiled at module load; operator literal substrings loaded via `loadOperatorLiterals()` from `src/core/content-sanity-literals.ts`. `ContentSanityBlockError` tagged class is the typed throw shape — every wrapper site (`gbrain import` CLI, `put_page` MCP op, `gbrain sync`, `/ingest` webhook) catches it via the existing exception flow rather than a parallel status check. The bytes-parity contract (D2) pins `Buffer.byteLength(content, 'utf8')` against the embedder's actual byte count so a 499K-byte page can't be soft-blocked on assessment then overflow on embed. Knob resolution chain: env > file (`~/.gbrain/config.json`) > DB > defaults — env wins for CI / one-off overrides, file is operator-set, DB plane is what `gbrain config set` writes. Four knobs: `content_sanity.bytes_warn` (default 50_000), `content_sanity.bytes_block` (default 500_000), `content_sanity.junk_patterns_enabled` (default true), `content_sanity.disabled` (default false; `GBRAIN_NO_SANITY=1` is the loud-stderr kill-switch with per-ingest warning). Pinned by `test/content-sanity.test.ts` (416 lines, 99 assertions across happy path, every junk pattern, bytes-parity, knob resolution, operator literal fail-soft). +- `src/core/content-sanity-literals.ts` (v0.40.9.0, NEW) — operator literal-substring loader. Reads `~/.gbrain/junk-substrings.txt`, one literal per non-comment non-blank line. Optional `# name=` header pairs an identifier with the following literal so audit JSONL groups by site (`linkedin_auth_wall`, `reddit_blocked`, etc.). Fail-soft on ENOENT (missing file = empty array, no error). Loaded on every ingest. Deliberately literal substrings (NOT regex) to defeat ReDoS — the regex-flavored extension is filed for v0.41+ once a real ReDoS budget exists. Pinned by `test/content-sanity-literals.test.ts` (110 lines). +- `src/core/embed-skip.ts` (v0.40.9.0, NEW) — 5-site shared predicate for the soft-block embed-skip filter. Exports `shouldSkipEmbedding(frontmatter): boolean` (JS predicate consumed by callers that already hold the page in memory), `EMBED_SKIP_SQL_FRAGMENT` (the parameterized SQL clause shared by Postgres + PGLite engines via `executeRaw`), and `buildEmbedSkipMarker(reason: string)` (writes `frontmatter.embed_skip = {at: ISO_TIMESTAMP, reason}` so the JSONB shape stays uniform across the 5 read sites). The 5 sites are: `embed.ts --stale`, `embed.ts --all`, the `embed-stale` Minion helper, plus both engines' `listStaleChunks` + `countStaleChunks`. Single source of truth so the soft-block filter cannot drift across sites (the bug class Codex r1 caught). Pinned by `test/embed-skip.test.ts` (cross-site invariant + JSONB shape). +- `src/core/audit/content-sanity-audit.ts` (v0.40.9.0, NEW) — ISO-week JSONL audit at `~/.gbrain/audit/content-sanity-YYYY-Www.jsonl` built on the v0.40.4.0 `audit-writer.ts` primitive. Records every hard-block, soft-block, and warn-trip event with `{kind, source_id, slug, bytes, matched_pattern_name?, reason, ts}`. Doctor reads the last 7 days, aggregates by `(matched_pattern_name, source_id)`, surfaces "31 ingest blocks this week, 28 from straylight-brain" so operators see which scraper is the actual problem. Honors `GBRAIN_AUDIT_DIR` for shared-filesystem multi-host setups (documented caveat in the doctor message for ops that don't share the dir). Pinned by `test/audit/content-sanity-audit.test.ts` (219 lines, 219 assertions). +- `src/commands/doctor.ts` extension (v0.40.9.0) — three new checks wired into `runDoctor()` and the JSON envelope: `oversized_pages` (warns on pages exceeding `content_sanity.bytes_warn`), `scraper_junk_pages` (warns on pages that match any junk pattern despite being live in the DB — these escaped pre-v0.40.9.0 ingest), and `content_sanity_audit_recent` (reads the last 7 days of audit events, aggregates by pattern+source). Default scans the 1000 most-recent pages; new `--content-audit` flag opts into a full scan for the cleanup wave. All three are warn-only with paste-ready fix hints (junk page → `gbrain sources audit ` + `git rm` source-of-truth, oversize → split or accept). +- `src/commands/lint.ts` extension (v0.40.9.0) — two new lint rules: `huge-page` (flags pages exceeding `content_sanity.bytes_warn` threshold) and `scraper-junk` (flags pages matching any junk pattern). Both reuse `assessContent()` from `src/core/content-sanity.ts` so lint, doctor, and ingest share one assessor — adding a junk pattern automatically covers all three surfaces. `lint.ts` lifts DB config when `~/.gbrain/` is reachable (matches what `gbrain config set` writes); falls back to file/env on CI. Pinned by `test/lint-content-sanity.test.ts` (161 lines). +- `src/commands/embed.ts` extension (v0.40.9.0) — applies the `embed-skip` filter at all 5 stale-chunk sites: `runEmbedCore --stale`, `runEmbedCore --all`, the `embed-stale` Minion helper, plus both engines' `listStaleChunks` + `countStaleChunks` via `EMBED_SKIP_SQL_FRAGMENT`. A soft-blocked page is queryable by title and slug but its chunks never enter the embed sweep. The shared helper from `src/core/embed-skip.ts` is the regression guard — no per-site ad-hoc filter is allowed. Pinned by `test/embed-skip.test.ts`. +- `src/core/import-file.ts` extension (v0.40.9.0) — `importFromContent` is the narrow waist that every ingest path passes through (`gbrain import`, `gbrain sync`, `put_page` MCP, `/ingest` webhook). It now calls `assessContent()` BEFORE chunking; verdict `hard_block_junk_pattern` throws `ContentSanityBlockError` (which every wrapper site already catches via its exception flow); verdict `warn_oversize OR oversize-without-junk` sets `frontmatter.embed_skip` via `buildEmbedSkipMarker()` AND deletes any pre-existing chunks for the page in the same transaction so search can't surface stale chunks against content that's now soft-blocked. `gbrain import` honors `errors > 0` for non-zero exit (was silently exit-0 on failed files). `classifyErrorCode` in `src/core/sync.ts` recognizes the new `PAGE_JUNK_PATTERN` code so sync-failures.jsonl grouping bins these correctly. Pinned by `test/import-file-content-sanity.test.ts` (206 lines). — shared library for the v0.12.0 graph layer. extractEntityRefs (canonical, replaces backlinks.ts duplicate) matches both `[Name](people/slug)` markdown links and Obsidian `[[people/slug|Name]]` wikilinks as of v0.12.3. extractPageLinks, inferLinkType heuristics (attended/works_at/invested_in/founded/advises/source/mentions), parseTimelineEntries, isAutoLinkEnabled config helper. `DIR_PATTERN` covers `people`, `companies`, `deals`, `topics`, `concepts`, `projects`, `entities`, `tech`, `finance`, `personal`, `openclaw`. Used by extract.ts, operations.ts auto-link post-hook, and backlinks.ts. - `src/core/zombie-reap.ts` (v0.28.1) — idempotent `installSigchldHandler()` so JS-spawned children get reaped via Bun's internal `waitpid()`. Bun (like Node) only auto-reaps when a SIGCHLD listener is registered; without it, every child the worker spawns (shell jobs, embed batches, sub-agents) becomes a zombie on exit and holds connection slots. Called once at module load from `src/cli.ts` (with Windows platform guard — SIGCHLD doesn't exist on Windows). Cross-file leak guard via `_uninstallSigchldHandlerForTests()` for tests. Layer 1 of the three-layer zombie defense; Layer 2 is tini-as-PID-1 wrapping the worker subtree (via `src/core/minions/spawn-helpers.ts`); Layer 3 is the container's own tini for hard Bun crashes. - `src/core/minions/` — Minions job queue: BullMQ-inspired, Postgres-native (queue, worker, backoff, types, protected-names, quiet-hours, stagger, handlers/shell). - `src/core/minions/queue.ts` — MinionQueue class (submit, claim, complete, fail, stall detection, parent-child, depth/child-cap, per-job timeouts, cascade-kill, attachments, idempotency keys, child_done inbox, removeOnComplete/Fail). `add()` takes a 4th `trusted` arg (separate from `opts` to prevent spread leakage); protected names in `PROTECTED_JOB_NAMES` require `{allowProtectedSubmit: true}` and the check runs trim-normalized (whitespace-bypass safe). v0.14.1 #219: `add()` plumbs `max_stalled` through with a `[1, 100]` clamp; omitted values let the schema DEFAULT (5) kick in. v0.19.0: `handleWallClockTimeouts(lockDurationMs)` is Layer 3 kill shot for jobs where `FOR UPDATE SKIP LOCKED` stall detection and the timeout sweep both fail to evict (wedged worker holding a row lock via a pending transaction). v0.19.1: `maxWaiting` coalesce path now uses `pg_advisory_xact_lock` keyed on `(name, queue)` to serialize concurrent submits for the same key, and filters on `queue` in addition to `name` so cross-queue same-name jobs don't suppress each other. diff --git a/TODOS.md b/TODOS.md index 2abdab650..60849ca36 100644 --- a/TODOS.md +++ b/TODOS.md @@ -1,5 +1,168 @@ # TODOS +## v0.41.0.0 follow-ups (v0.41.1+) + +- [ ] **v0.41+: per-key rate-lease caps (`openai:responses`, `google:gemini`, etc.).** + v0.41 ships a single `anthropic:messages` rate-lease cap. When users run + subagents against multiple providers via the gateway path, each provider + should have its OWN rate-lease bucket so they don't share capacity. The + right time for this is right after `agent.use_gateway_loop=true` becomes + the default — before that, you're solving for a configuration no one uses. + Priority: P2. Filed via CEO D13. References: `src/core/minions/rate-leases.ts` + + `src/core/minions/handlers/subagent.ts:GBRAIN_ANTHROPIC_MAX_INFLIGHT`. + +- [ ] **v0.41+: `minion_lease_pressure_log` + budget/self-fix audit retention sweep.** + v0.41 migration v93 promoted `ON DELETE SET NULL` on audit FKs so rows + survive `gbrain jobs prune`. Codex pass-3 #5 caught the corollary: without + retention, audit tables grow unbounded. On a steady-pressure install + (heavy daily batches), `minion_lease_pressure_log` is millions of rows by + year 2. Add a sweep phase to the autopilot cycle's `purge` phase (the + v0.26.5 pattern, sibling to `engine.purgeDeletedPages(72)`): + `engine.purgeOldAuditRows({ lease_pressure_max_age_days: 90, budget_log_max_age_days: 365, self_fix_log_max_age_days: 180 })`. + Defaults match operator use cases (90 days lease pressure for capacity + tuning, 365 days budget for accounting, 180 days self-fix for + classifier-tuning); all overridable via config. Priority: P3. Filed via + CEO D16. Closes the unbounded-growth concern that codex flagged as + load-bearing pass-3 #5. + +- [ ] **v0.41.1: full E5 A/B dispatcher (currently scaffolded as dry-run only).** + `scripts/e5-lease-cap-ab.ts` ships the spec + harness + receipt fixture + shape but the real-run dispatcher (queue submit + worker spin-up + 15-min + 429 injector + tick loop + cost-tracking) is deferred. v0.41.1 follow-up + writes the dispatcher and commits the first real-API receipt as the + baseline before flipping `minions.auto_lease_cap` to default ON. + +- [ ] **v0.41.1: `tryWithDbElection` retrofit for existing `pg_advisory_xact_lock` call sites.** + Codex pass-2 #7 caught that `src/core/minions/rate-leases.ts:80` + (`acquireLease`) and `src/core/minions/queue.ts:152` (maxWaiting coalesce) + call `pg_advisory_xact_lock` unconditionally. PGLite has no advisory locks + (`src/core/pglite-schema.ts:6`); current code passes by accident because + PGLite is single-connection. New `tryWithDbElection` primitive in + `src/core/db-lock.ts` is engine-dispatched. Retrofit the two existing + call sites to use it so PGLite correctness is explicit, not accidental. + Two call shapes needed (codex pass-3 #10): one starts a new tx (E5 use + case, already shipped); one accepts an existing tx (rate-leases + + maxWaiting use cases). Filed via Eng D9. + +- [ ] **v0.42: semantic-aware `prompt_too_long` reduction in E6 self-fix.** + v0.41 ships truncate-with-leaf-preservation (first 1000 + last 2000 chars). + Codex pass-1 #11 specified the right strategy: walk the conversation, drop + tool_result blocks first (largest non-task content), summarize older + user/asst pairs via Haiku, never delete the leaf user task. Implementation + lives in `src/core/minions/self-fix.ts:buildSelfFixPrompt`. Worst-case + current behavior (truncate-then-fail) is safe — no infinite loops, + depth-cap prevents chains — but full semantic reduction unlocks higher + self-fix success rates on legitimately-long prompts. +## v0.41 content-sanity follow-ups (filed during ship of `garrytan/lint-page-size-gate`) + +Source: CEO + Eng review on the content-sanity defense plan. Both reviews +ran Codex (round 1 + round 2 — 30 total findings) and the wave shipped +with the strategic items addressed. These are the deliberately-deferred +follow-ups, captured here so v0.42 starts informed. + +- [ ] **v0.42 P1 — Chunk-level embed-quarantine.** The v0.41 wave landed + page-level soft-block (`frontmatter.embed_skip`); Codex r1 #3 caught + that staleness is chunk-based (`content_chunks.embedding IS NULL`). + Right granularity for the embed-pipeline-overflow case is per-chunk, + not per-page. Move: add `content_chunks.embed_quarantined_at TIMESTAMPTZ` + + partial index, catch `TokenLimitError` from gateway, mark the offending + chunk only (keep good siblings), surface in doctor's + `embedding_coverage`. Requires repro of the original 890K embed failure + on current code FIRST to confirm whether it's batch-overflow vs + single-oversized-chunk vs token-estimate-miss. Effort: human ~2 days / + CC ~3 hours. + +- [ ] **v0.42 P1 — Source-repo remediation surface.** Codex r1 #7 + caught: cleanup CLI that deletes DB rows doesn't fix source of truth + — junk file in source repo reappears on next sync. Move: add + `gbrain sources prune-junk ` that walks `local_path`, finds files + matching the junk-pattern set, soft-deletes DB rows AND `git rm`s the + files in the source repo (commit message: `auto: prune junk pages + flagged by gbrain content-sanity`). Operator pushes the commit. + Pairs with the v0.42 chunk-quarantine for a complete cleanup story. + Effort: human ~1 day / CC ~2 hours. + +- [ ] **v0.41 + 30 days — Threshold default validation post-deploy.** + Codex r1 #15 caught: we invented 50K warn / 500K block thresholds + before measuring real corpus distribution. Move: run `gbrain sources + audit ` on real source repos (start with Garry's own brain), + collect distribution stats from the JSON envelope, tune defaults + if the measured p99 disagrees with the 50K assumption. Either + publish updated defaults in a v0.41.x patch or document the env + override path in CHANGELOG. Effort: human ~30min / CC ~10min. + +- [ ] **v0.42 P2 — Pages soft-delete CLI (`gbrain pages soft-delete + --where`).** Cherry-pick 3 from the original CEO review; dropped + during eng review because Codex r1 #7 weakened it (doesn't fix + source-of-truth). Resurface in v0.42 as a PAIRED tool alongside + the v0.42 source-repo remediation. Filter expressions: + `matches_junk_pattern`, `bytes > N`. Required UX gates: `--dry-run` + preview, `--confirm-destructive` flag when affected > 0, 1000-page + per-invocation cap. Routes through existing `engine.softDeletePage()` + (v0.26.5 72h-TTL safe-delete; reversible). + +- [ ] **v0.42 P3 — Brain-score `no_junk_pages_score` component.** + Add a 6th component to the v0.36.4.0 5-component brain-score + formula (currently embed_coverage 35 + link_density 25 + + timeline_coverage 15 + no_orphans 15 + no_dead_links 10). Reweight + to make room (probably take 5 from no_dead_links: 35/25/15/15/5/5). + File AFTER v0.41's audit JSONL has 30+ days of signal so we know + the realistic distribution of junk-page rates across brains before + pinning a score weight. + +- [ ] **post-v0.45 — Operator-supplied regex extensibility.** Dropped + in v0.41 per Codex r1 #10 (JavaScript RegExp lacks atomic groups / + possessive quantifiers, making a reliable ReDoS shape detector + hard). The v0.41 ship has literal-substring extensibility instead + which covers ~95% of real operator use cases. If real operators + ask for regex, add it with a real story: either re2 (Google's + linear-time engine; native dep, build complications) or worker- + thread per-pattern timeout (50ms cap, runtime overhead). + +- [ ] **post-v0.45 — HTML-density rule.** Dropped in v0.41 per Codex + r1 #16. Was: flag pages where `
`/``/etc tag density is + too high (raw HTML dump indicator). Requires careful handling of + fenced code blocks, JSX/XML in technical notes, escaped HTML. + Without that rigor, false-positives on legitimate code-heavy + technical writing. The scraper-junk pattern set catches the real + junk class without needing density math; revisit only if a junk + pattern leaks through that ONLY density would catch. + +- [ ] **v0.41+ — Bytes parity assertion across lint + doctor.** D2 + acceptance test included in `test/content-sanity.test.ts` as a + unit-level parity check. Promote to an E2E that seeds a real + fixture page with frontmatter + body, runs `gbrain lint` AND + `gbrain doctor --content-audit`, asserts both surfaces report + the same byte count. Catches drift between + `Buffer.byteLength` (assessor) and `octet_length` (doctor SQL) + if either surface changes the measurement axis. + +- [ ] **v0.41+ — `gbrain sources audit` E2E pin test.** The CLI + shipped with unit tests pinning `assessContentSanity` shape; + the integration test (walk a fixture source dir, run the CLI + end-to-end, assert JSON envelope shape) is deferred. Trivial to + add (~30 LOC) once a stable test fixture set lands under + `test/fixtures/content-sanity/`. + +- [ ] **v0.41+ — Doctor checks integration tests.** The 3 new doctor + checks (`oversized_pages`, `scraper_junk_pages`, + `content_sanity_audit_recent`) ship verified by typecheck + + runtime-shape via the unit suite. Integration tests (seed fixture + pages into PGLite, run doctor, assert check status + message + format) are deferred. Same pattern as existing + `test/doctor.test.ts` extensions. + +- [ ] **v0.41+ — 5-path narrow-waist E2E pin tests (cherry-pick 5).** + Sync + import + put_page MCP + capture + /ingest webhook all + route through `importFromContent` so the new gate applies + uniformly. Unit tests pin the gate behavior; E2E pin tests + prove each ingestion path actually goes through it. Tests for + sync + import + put_page MCP + capture are PGLite-hermetic; + the /ingest webhook test needs real-Postgres E2E (DATABASE_URL). + Filed during eng review as P2; not blocking ship since the + narrow-waist contract is structurally enforced by every wrapper + routing through `importFromContent` already. + ## v0.41+ wave commitments (decided 2026-05-23) Source: `/plan-ceo-review` + `/plan-eng-review` triage of TODOS as roadmap diff --git a/VERSION b/VERSION index 3e6fc29d6..7ee8e6a08 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.40.8.1 +0.41.0.0 \ No newline at end of file diff --git a/admin/dist/assets/index-DFgMZhBE.js b/admin/dist/assets/index-DFgMZhBE.js deleted file mode 100644 index 725c9fd8a..000000000 --- a/admin/dist/assets/index-DFgMZhBE.js +++ /dev/null @@ -1,56 +0,0 @@ -(function(){const N=document.createElement("link").relList;if(N&&N.supports&&N.supports("modulepreload"))return;for(const _ of document.querySelectorAll('link[rel="modulepreload"]'))h(_);new MutationObserver(_=>{for(const Y of _)if(Y.type==="childList")for(const U of Y.addedNodes)U.tagName==="LINK"&&U.rel==="modulepreload"&&h(U)}).observe(document,{childList:!0,subtree:!0});function M(_){const Y={};return _.integrity&&(Y.integrity=_.integrity),_.referrerPolicy&&(Y.referrerPolicy=_.referrerPolicy),_.crossOrigin==="use-credentials"?Y.credentials="include":_.crossOrigin==="anonymous"?Y.credentials="omit":Y.credentials="same-origin",Y}function h(_){if(_.ep)return;_.ep=!0;const Y=M(_);fetch(_.href,Y)}})();function Nr(d){return d&&d.__esModule&&Object.prototype.hasOwnProperty.call(d,"default")?d.default:d}var ff={exports:{}},zn={};/** - * @license React - * react-jsx-runtime.production.js - * - * Copyright (c) Meta Platforms, Inc. and affiliates. - * - * This source code is licensed under the MIT license found in the - * LICENSE file in the root directory of this source tree. - */var gr;function ty(){if(gr)return zn;gr=1;var d=Symbol.for("react.transitional.element"),N=Symbol.for("react.fragment");function M(h,_,Y){var U=null;if(Y!==void 0&&(U=""+Y),_.key!==void 0&&(U=""+_.key),"key"in _){Y={};for(var Z in _)Z!=="key"&&(Y[Z]=_[Z])}else Y=_;return _=Y.ref,{$$typeof:d,type:h,key:U,ref:_!==void 0?_:null,props:Y}}return zn.Fragment=N,zn.jsx=M,zn.jsxs=M,zn}var Sr;function ey(){return Sr||(Sr=1,ff.exports=ty()),ff.exports}var f=ey(),sf={exports:{}},V={};/** - * @license React - * react.production.js - * - * Copyright (c) Meta Platforms, Inc. and affiliates. - * - * This source code is licensed under the MIT license found in the - * LICENSE file in the root directory of this source tree. - */var br;function ay(){if(br)return V;br=1;var d=Symbol.for("react.transitional.element"),N=Symbol.for("react.portal"),M=Symbol.for("react.fragment"),h=Symbol.for("react.strict_mode"),_=Symbol.for("react.profiler"),Y=Symbol.for("react.consumer"),U=Symbol.for("react.context"),Z=Symbol.for("react.forward_ref"),O=Symbol.for("react.suspense"),p=Symbol.for("react.memo"),H=Symbol.for("react.lazy"),j=Symbol.for("react.activity"),E=Symbol.iterator;function I(r){return r===null||typeof r!="object"?null:(r=E&&r[E]||r["@@iterator"],typeof r=="function"?r:null)}var L={isMounted:function(){return!1},enqueueForceUpdate:function(){},enqueueReplaceState:function(){},enqueueSetState:function(){}},nl=Object.assign,tl={};function pl(r,A,C){this.props=r,this.context=A,this.refs=tl,this.updater=C||L}pl.prototype.isReactComponent={},pl.prototype.setState=function(r,A){if(typeof r!="object"&&typeof r!="function"&&r!=null)throw Error("takes an object of state variables to update or a function which returns an object of state variables.");this.updater.enqueueSetState(this,r,A,"setState")},pl.prototype.forceUpdate=function(r){this.updater.enqueueForceUpdate(this,r,"forceUpdate")};function Ml(){}Ml.prototype=pl.prototype;function Gl(r,A,C){this.props=r,this.context=A,this.refs=tl,this.updater=C||L}var ot=Gl.prototype=new Ml;ot.constructor=Gl,nl(ot,pl.prototype),ot.isPureReactComponent=!0;var _t=Array.isArray;function Ll(){}var el={H:null,A:null,T:null,S:null},Vl=Object.prototype.hasOwnProperty;function jt(r,A,C){var B=C.ref;return{$$typeof:d,type:r,key:A,ref:B!==void 0?B:null,props:C}}function Le(r,A){return jt(r.type,A,r.props)}function Ot(r){return typeof r=="object"&&r!==null&&r.$$typeof===d}function Kl(r){var A={"=":"=0",":":"=2"};return"$"+r.replace(/[=:]/g,function(C){return A[C]})}var xe=/\/+/g;function Ut(r,A){return typeof r=="object"&&r!==null&&r.key!=null?Kl(""+r.key):A.toString(36)}function zt(r){switch(r.status){case"fulfilled":return r.value;case"rejected":throw r.reason;default:switch(typeof r.status=="string"?r.then(Ll,Ll):(r.status="pending",r.then(function(A){r.status==="pending"&&(r.status="fulfilled",r.value=A)},function(A){r.status==="pending"&&(r.status="rejected",r.reason=A)})),r.status){case"fulfilled":return r.value;case"rejected":throw r.reason}}throw r}function T(r,A,C,B,K){var $=typeof r;($==="undefined"||$==="boolean")&&(r=null);var fl=!1;if(r===null)fl=!0;else switch($){case"bigint":case"string":case"number":fl=!0;break;case"object":switch(r.$$typeof){case d:case N:fl=!0;break;case H:return fl=r._init,T(fl(r._payload),A,C,B,K)}}if(fl)return K=K(r),fl=B===""?"."+Ut(r,0):B,_t(K)?(C="",fl!=null&&(C=fl.replace(xe,"$&/")+"/"),T(K,A,C,"",function(Oa){return Oa})):K!=null&&(Ot(K)&&(K=Le(K,C+(K.key==null||r&&r.key===K.key?"":(""+K.key).replace(xe,"$&/")+"/")+fl)),A.push(K)),1;fl=0;var Ql=B===""?".":B+":";if(_t(r))for(var xl=0;xl>>1,yl=T[dl];if(0<_(yl,D))T[dl]=D,T[Q]=yl,Q=dl;else break l}}function M(T){return T.length===0?null:T[0]}function h(T){if(T.length===0)return null;var D=T[0],Q=T.pop();if(Q!==D){T[0]=Q;l:for(var dl=0,yl=T.length,r=yl>>>1;dl_(C,Q))B_(K,C)?(T[dl]=K,T[B]=Q,dl=B):(T[dl]=C,T[A]=Q,dl=A);else if(B_(K,Q))T[dl]=K,T[B]=Q,dl=B;else break l}}return D}function _(T,D){var Q=T.sortIndex-D.sortIndex;return Q!==0?Q:T.id-D.id}if(d.unstable_now=void 0,typeof performance=="object"&&typeof performance.now=="function"){var Y=performance;d.unstable_now=function(){return Y.now()}}else{var U=Date,Z=U.now();d.unstable_now=function(){return U.now()-Z}}var O=[],p=[],H=1,j=null,E=3,I=!1,L=!1,nl=!1,tl=!1,pl=typeof setTimeout=="function"?setTimeout:null,Ml=typeof clearTimeout=="function"?clearTimeout:null,Gl=typeof setImmediate<"u"?setImmediate:null;function ot(T){for(var D=M(p);D!==null;){if(D.callback===null)h(p);else if(D.startTime<=T)h(p),D.sortIndex=D.expirationTime,N(O,D);else break;D=M(p)}}function _t(T){if(nl=!1,ot(T),!L)if(M(O)!==null)L=!0,Ll||(Ll=!0,Kl());else{var D=M(p);D!==null&&zt(_t,D.startTime-T)}}var Ll=!1,el=-1,Vl=5,jt=-1;function Le(){return tl?!0:!(d.unstable_now()-jtT&&Le());){var dl=j.callback;if(typeof dl=="function"){j.callback=null,E=j.priorityLevel;var yl=dl(j.expirationTime<=T);if(T=d.unstable_now(),typeof yl=="function"){j.callback=yl,ot(T),D=!0;break t}j===M(O)&&h(O),ot(T)}else h(O);j=M(O)}if(j!==null)D=!0;else{var r=M(p);r!==null&&zt(_t,r.startTime-T),D=!1}}break l}finally{j=null,E=Q,I=!1}D=void 0}}finally{D?Kl():Ll=!1}}}var Kl;if(typeof Gl=="function")Kl=function(){Gl(Ot)};else if(typeof MessageChannel<"u"){var xe=new MessageChannel,Ut=xe.port2;xe.port1.onmessage=Ot,Kl=function(){Ut.postMessage(null)}}else Kl=function(){pl(Ot,0)};function zt(T,D){el=pl(function(){T(d.unstable_now())},D)}d.unstable_IdlePriority=5,d.unstable_ImmediatePriority=1,d.unstable_LowPriority=4,d.unstable_NormalPriority=3,d.unstable_Profiling=null,d.unstable_UserBlockingPriority=2,d.unstable_cancelCallback=function(T){T.callback=null},d.unstable_forceFrameRate=function(T){0>T||125dl?(T.sortIndex=Q,N(p,T),M(O)===null&&T===M(p)&&(nl?(Ml(el),el=-1):nl=!0,zt(_t,Q-dl))):(T.sortIndex=yl,N(O,T),L||I||(L=!0,Ll||(Ll=!0,Kl()))),T},d.unstable_shouldYield=Le,d.unstable_wrapCallback=function(T){var D=E;return function(){var Q=E;E=D;try{return T.apply(this,arguments)}finally{E=Q}}}})(rf)),rf}var zr;function uy(){return zr||(zr=1,df.exports=ny()),df.exports}var hf={exports:{}},Xl={};/** - * @license React - * react-dom.production.js - * - * Copyright (c) Meta Platforms, Inc. and affiliates. - * - * This source code is licensed under the MIT license found in the - * LICENSE file in the root directory of this source tree. - */var xr;function iy(){if(xr)return Xl;xr=1;var d=mf();function N(O){var p="https://react.dev/errors/"+O;if(1"u"||typeof __REACT_DEVTOOLS_GLOBAL_HOOK__.checkDCE!="function"))try{__REACT_DEVTOOLS_GLOBAL_HOOK__.checkDCE(d)}catch(N){console.error(N)}}return d(),hf.exports=iy(),hf.exports}/** - * @license React - * react-dom-client.production.js - * - * Copyright (c) Meta Platforms, Inc. and affiliates. - * - * This source code is licensed under the MIT license found in the - * LICENSE file in the root directory of this source tree. - */var Er;function fy(){if(Er)return xn;Er=1;var d=uy(),N=mf(),M=cy();function h(l){var t="https://react.dev/errors/"+l;if(1yl||(l.current=dl[yl],dl[yl]=null,yl--)}function C(l,t){yl++,dl[yl]=l.current,l.current=t}var B=r(null),K=r(null),$=r(null),fl=r(null);function Ql(l,t){switch(C($,t),C(K,l),C(B,null),t.nodeType){case 9:case 11:l=(l=t.documentElement)&&(l=l.namespaceURI)?Xd(l):0;break;default:if(l=t.tagName,t=t.namespaceURI)t=Xd(t),l=Qd(t,l);else switch(l){case"svg":l=1;break;case"math":l=2;break;default:l=0}}A(B),C(B,l)}function xl(){A(B),A(K),A($)}function Oa(l){l.memoizedState!==null&&C(fl,l);var t=B.current,e=Qd(t,l.type);t!==e&&(C(K,l),C(B,e))}function An(l){K.current===l&&(A(B),A(K)),fl.current===l&&(A(fl),Sn._currentValue=Q)}var Lu,yf;function Ae(l){if(Lu===void 0)try{throw Error()}catch(e){var t=e.stack.trim().match(/\n( *(at )?)/);Lu=t&&t[1]||"",yf=-1)":-1n||s[a]!==v[n]){var b=` -`+s[a].replace(" at new "," at ");return l.displayName&&b.includes("")&&(b=b.replace("",l.displayName)),b}while(1<=a&&0<=n);break}}}finally{Vu=!1,Error.prepareStackTrace=e}return(e=l?l.displayName||l.name:"")?Ae(e):""}function Cr(l,t){switch(l.tag){case 26:case 27:case 5:return Ae(l.type);case 16:return Ae("Lazy");case 13:return l.child!==t&&t!==null?Ae("Suspense Fallback"):Ae("Suspense");case 19:return Ae("SuspenseList");case 0:case 15:return Ku(l.type,!1);case 11:return Ku(l.type.render,!1);case 1:return Ku(l.type,!0);case 31:return Ae("Activity");default:return""}}function vf(l){try{var t="",e=null;do t+=Cr(l,e),e=l,l=l.return;while(l);return t}catch(a){return` -Error generating stack: `+a.message+` -`+a.stack}}var Ju=Object.prototype.hasOwnProperty,wu=d.unstable_scheduleCallback,ku=d.unstable_cancelCallback,Ur=d.unstable_shouldYield,Rr=d.unstable_requestPaint,Pl=d.unstable_now,Hr=d.unstable_getCurrentPriorityLevel,gf=d.unstable_ImmediatePriority,Sf=d.unstable_UserBlockingPriority,En=d.unstable_NormalPriority,Br=d.unstable_LowPriority,bf=d.unstable_IdlePriority,qr=d.log,Yr=d.unstable_setDisableYieldValue,Na=null,lt=null;function It(l){if(typeof qr=="function"&&Yr(l),lt&&typeof lt.setStrictMode=="function")try{lt.setStrictMode(Na,l)}catch{}}var tt=Math.clz32?Math.clz32:Qr,Gr=Math.log,Xr=Math.LN2;function Qr(l){return l>>>=0,l===0?32:31-(Gr(l)/Xr|0)|0}var _n=256,jn=262144,On=4194304;function Ee(l){var t=l&42;if(t!==0)return t;switch(l&-l){case 1:return 1;case 2:return 2;case 4:return 4;case 8:return 8;case 16:return 16;case 32:return 32;case 64:return 64;case 128:return 128;case 256:case 512:case 1024:case 2048:case 4096:case 8192:case 16384:case 32768:case 65536:case 131072:return l&261888;case 262144:case 524288:case 1048576:case 2097152:return l&3932160;case 4194304:case 8388608:case 16777216:case 33554432:return l&62914560;case 67108864:return 67108864;case 134217728:return 134217728;case 268435456:return 268435456;case 536870912:return 536870912;case 1073741824:return 0;default:return l}}function Nn(l,t,e){var a=l.pendingLanes;if(a===0)return 0;var n=0,u=l.suspendedLanes,i=l.pingedLanes;l=l.warmLanes;var c=a&134217727;return c!==0?(a=c&~u,a!==0?n=Ee(a):(i&=c,i!==0?n=Ee(i):e||(e=c&~l,e!==0&&(n=Ee(e))))):(c=a&~u,c!==0?n=Ee(c):i!==0?n=Ee(i):e||(e=a&~l,e!==0&&(n=Ee(e)))),n===0?0:t!==0&&t!==n&&(t&u)===0&&(u=n&-n,e=t&-t,u>=e||u===32&&(e&4194048)!==0)?t:n}function Ma(l,t){return(l.pendingLanes&~(l.suspendedLanes&~l.pingedLanes)&t)===0}function Zr(l,t){switch(l){case 1:case 2:case 4:case 8:case 64:return t+250;case 16:case 32:case 128:case 256:case 512:case 1024:case 2048:case 4096:case 8192:case 16384:case 32768:case 65536:case 131072:case 262144:case 524288:case 1048576:case 2097152:return t+5e3;case 4194304:case 8388608:case 16777216:case 33554432:return-1;case 67108864:case 134217728:case 268435456:case 536870912:case 1073741824:return-1;default:return-1}}function pf(){var l=On;return On<<=1,(On&62914560)===0&&(On=4194304),l}function $u(l){for(var t=[],e=0;31>e;e++)t.push(l);return t}function Da(l,t){l.pendingLanes|=t,t!==268435456&&(l.suspendedLanes=0,l.pingedLanes=0,l.warmLanes=0)}function Lr(l,t,e,a,n,u){var i=l.pendingLanes;l.pendingLanes=e,l.suspendedLanes=0,l.pingedLanes=0,l.warmLanes=0,l.expiredLanes&=e,l.entangledLanes&=e,l.errorRecoveryDisabledLanes&=e,l.shellSuspendCounter=0;var c=l.entanglements,s=l.expirationTimes,v=l.hiddenUpdates;for(e=i&~e;0"u")return null;try{return l.activeElement||l.body}catch{return l.body}}var $r=/[\n"\\]/g;function rt(l){return l.replace($r,function(t){return"\\"+t.charCodeAt(0).toString(16)+" "})}function ti(l,t,e,a,n,u,i,c){l.name="",i!=null&&typeof i!="function"&&typeof i!="symbol"&&typeof i!="boolean"?l.type=i:l.removeAttribute("type"),t!=null?i==="number"?(t===0&&l.value===""||l.value!=t)&&(l.value=""+dt(t)):l.value!==""+dt(t)&&(l.value=""+dt(t)):i!=="submit"&&i!=="reset"||l.removeAttribute("value"),t!=null?ei(l,i,dt(t)):e!=null?ei(l,i,dt(e)):a!=null&&l.removeAttribute("value"),n==null&&u!=null&&(l.defaultChecked=!!u),n!=null&&(l.checked=n&&typeof n!="function"&&typeof n!="symbol"),c!=null&&typeof c!="function"&&typeof c!="symbol"&&typeof c!="boolean"?l.name=""+dt(c):l.removeAttribute("name")}function Uf(l,t,e,a,n,u,i,c){if(u!=null&&typeof u!="function"&&typeof u!="symbol"&&typeof u!="boolean"&&(l.type=u),t!=null||e!=null){if(!(u!=="submit"&&u!=="reset"||t!=null)){li(l);return}e=e!=null?""+dt(e):"",t=t!=null?""+dt(t):e,c||t===l.value||(l.value=t),l.defaultValue=t}a=a??n,a=typeof a!="function"&&typeof a!="symbol"&&!!a,l.checked=c?l.checked:!!a,l.defaultChecked=!!a,i!=null&&typeof i!="function"&&typeof i!="symbol"&&typeof i!="boolean"&&(l.name=i),li(l)}function ei(l,t,e){t==="number"&&Cn(l.ownerDocument)===l||l.defaultValue===""+e||(l.defaultValue=""+e)}function $e(l,t,e,a){if(l=l.options,t){t={};for(var n=0;n"u"||typeof window.document>"u"||typeof window.document.createElement>"u"),ci=!1;if(Bt)try{var Ha={};Object.defineProperty(Ha,"passive",{get:function(){ci=!0}}),window.addEventListener("test",Ha,Ha),window.removeEventListener("test",Ha,Ha)}catch{ci=!1}var le=null,fi=null,Rn=null;function Xf(){if(Rn)return Rn;var l,t=fi,e=t.length,a,n="value"in le?le.value:le.textContent,u=n.length;for(l=0;l=Ya),Jf=" ",wf=!1;function kf(l,t){switch(l){case"keyup":return xh.indexOf(t.keyCode)!==-1;case"keydown":return t.keyCode!==229;case"keypress":case"mousedown":case"focusout":return!0;default:return!1}}function $f(l){return l=l.detail,typeof l=="object"&&"data"in l?l.data:null}var Pe=!1;function Eh(l,t){switch(l){case"compositionend":return $f(t);case"keypress":return t.which!==32?null:(wf=!0,Jf);case"textInput":return l=t.data,l===Jf&&wf?null:l;default:return null}}function _h(l,t){if(Pe)return l==="compositionend"||!hi&&kf(l,t)?(l=Xf(),Rn=fi=le=null,Pe=!1,l):null;switch(l){case"paste":return null;case"keypress":if(!(t.ctrlKey||t.altKey||t.metaKey)||t.ctrlKey&&t.altKey){if(t.char&&1=t)return{node:e,offset:t-l};l=a}l:{for(;e;){if(e.nextSibling){e=e.nextSibling;break l}e=e.parentNode}e=void 0}e=as(e)}}function us(l,t){return l&&t?l===t?!0:l&&l.nodeType===3?!1:t&&t.nodeType===3?us(l,t.parentNode):"contains"in l?l.contains(t):l.compareDocumentPosition?!!(l.compareDocumentPosition(t)&16):!1:!1}function is(l){l=l!=null&&l.ownerDocument!=null&&l.ownerDocument.defaultView!=null?l.ownerDocument.defaultView:window;for(var t=Cn(l.document);t instanceof l.HTMLIFrameElement;){try{var e=typeof t.contentWindow.location.href=="string"}catch{e=!1}if(e)l=t.contentWindow;else break;t=Cn(l.document)}return t}function vi(l){var t=l&&l.nodeName&&l.nodeName.toLowerCase();return t&&(t==="input"&&(l.type==="text"||l.type==="search"||l.type==="tel"||l.type==="url"||l.type==="password")||t==="textarea"||l.contentEditable==="true")}var Rh=Bt&&"documentMode"in document&&11>=document.documentMode,la=null,gi=null,Za=null,Si=!1;function cs(l,t,e){var a=e.window===e?e.document:e.nodeType===9?e:e.ownerDocument;Si||la==null||la!==Cn(a)||(a=la,"selectionStart"in a&&vi(a)?a={start:a.selectionStart,end:a.selectionEnd}:(a=(a.ownerDocument&&a.ownerDocument.defaultView||window).getSelection(),a={anchorNode:a.anchorNode,anchorOffset:a.anchorOffset,focusNode:a.focusNode,focusOffset:a.focusOffset}),Za&&Qa(Za,a)||(Za=a,a=ju(gi,"onSelect"),0>=i,n-=i,Nt=1<<32-tt(t)+n|e<w?(ll=q,q=null):ll=q.sibling;var il=g(m,q,y[w],z);if(il===null){q===null&&(q=ll);break}l&&q&&il.alternate===null&&t(m,q),o=u(il,o,w),ul===null?G=il:ul.sibling=il,ul=il,q=ll}if(w===y.length)return e(m,q),al&&Yt(m,w),G;if(q===null){for(;ww?(ll=q,q=null):ll=q.sibling;var ze=g(m,q,il.value,z);if(ze===null){q===null&&(q=ll);break}l&&q&&ze.alternate===null&&t(m,q),o=u(ze,o,w),ul===null?G=ze:ul.sibling=ze,ul=ze,q=ll}if(il.done)return e(m,q),al&&Yt(m,w),G;if(q===null){for(;!il.done;w++,il=y.next())il=x(m,il.value,z),il!==null&&(o=u(il,o,w),ul===null?G=il:ul.sibling=il,ul=il);return al&&Yt(m,w),G}for(q=a(q);!il.done;w++,il=y.next())il=S(q,m,w,il.value,z),il!==null&&(l&&il.alternate!==null&&q.delete(il.key===null?w:il.key),o=u(il,o,w),ul===null?G=il:ul.sibling=il,ul=il);return l&&q.forEach(function(ly){return t(m,ly)}),al&&Yt(m,w),G}function ml(m,o,y,z){if(typeof y=="object"&&y!==null&&y.type===nl&&y.key===null&&(y=y.props.children),typeof y=="object"&&y!==null){switch(y.$$typeof){case I:l:{for(var G=y.key;o!==null;){if(o.key===G){if(G=y.type,G===nl){if(o.tag===7){e(m,o.sibling),z=n(o,y.props.children),z.return=m,m=z;break l}}else if(o.elementType===G||typeof G=="object"&&G!==null&&G.$$typeof===Vl&&Be(G)===o.type){e(m,o.sibling),z=n(o,y.props),ka(z,y),z.return=m,m=z;break l}e(m,o);break}else t(m,o);o=o.sibling}y.type===nl?(z=De(y.props.children,m.mode,z,y.key),z.return=m,m=z):(z=Vn(y.type,y.key,y.props,null,m.mode,z),ka(z,y),z.return=m,m=z)}return i(m);case L:l:{for(G=y.key;o!==null;){if(o.key===G)if(o.tag===4&&o.stateNode.containerInfo===y.containerInfo&&o.stateNode.implementation===y.implementation){e(m,o.sibling),z=n(o,y.children||[]),z.return=m,m=z;break l}else{e(m,o);break}else t(m,o);o=o.sibling}z=Ei(y,m.mode,z),z.return=m,m=z}return i(m);case Vl:return y=Be(y),ml(m,o,y,z)}if(zt(y))return R(m,o,y,z);if(Kl(y)){if(G=Kl(y),typeof G!="function")throw Error(h(150));return y=G.call(y),X(m,o,y,z)}if(typeof y.then=="function")return ml(m,o,Fn(y),z);if(y.$$typeof===Gl)return ml(m,o,wn(m,y),z);In(m,y)}return typeof y=="string"&&y!==""||typeof y=="number"||typeof y=="bigint"?(y=""+y,o!==null&&o.tag===6?(e(m,o.sibling),z=n(o,y),z.return=m,m=z):(e(m,o),z=Ai(y,m.mode,z),z.return=m,m=z),i(m)):e(m,o)}return function(m,o,y,z){try{wa=0;var G=ml(m,o,y,z);return da=null,G}catch(q){if(q===oa||q===$n)throw q;var ul=at(29,q,null,m.mode);return ul.lanes=z,ul.return=m,ul}finally{}}}var Ye=Ms(!0),Ds=Ms(!1),ue=!1;function qi(l){l.updateQueue={baseState:l.memoizedState,firstBaseUpdate:null,lastBaseUpdate:null,shared:{pending:null,lanes:0,hiddenCallbacks:null},callbacks:null}}function Yi(l,t){l=l.updateQueue,t.updateQueue===l&&(t.updateQueue={baseState:l.baseState,firstBaseUpdate:l.firstBaseUpdate,lastBaseUpdate:l.lastBaseUpdate,shared:l.shared,callbacks:null})}function ie(l){return{lane:l,tag:0,payload:null,callback:null,next:null}}function ce(l,t,e){var a=l.updateQueue;if(a===null)return null;if(a=a.shared,(cl&2)!==0){var n=a.pending;return n===null?t.next=t:(t.next=n.next,n.next=t),a.pending=t,t=Ln(l),ms(l,null,e),t}return Zn(l,a,t,e),Ln(l)}function $a(l,t,e){if(t=t.updateQueue,t!==null&&(t=t.shared,(e&4194048)!==0)){var a=t.lanes;a&=l.pendingLanes,e|=a,t.lanes=e,zf(l,e)}}function Gi(l,t){var e=l.updateQueue,a=l.alternate;if(a!==null&&(a=a.updateQueue,e===a)){var n=null,u=null;if(e=e.firstBaseUpdate,e!==null){do{var i={lane:e.lane,tag:e.tag,payload:e.payload,callback:null,next:null};u===null?n=u=i:u=u.next=i,e=e.next}while(e!==null);u===null?n=u=t:u=u.next=t}else n=u=t;e={baseState:a.baseState,firstBaseUpdate:n,lastBaseUpdate:u,shared:a.shared,callbacks:a.callbacks},l.updateQueue=e;return}l=e.lastBaseUpdate,l===null?e.firstBaseUpdate=t:l.next=t,e.lastBaseUpdate=t}var Xi=!1;function Wa(){if(Xi){var l=sa;if(l!==null)throw l}}function Fa(l,t,e,a){Xi=!1;var n=l.updateQueue;ue=!1;var u=n.firstBaseUpdate,i=n.lastBaseUpdate,c=n.shared.pending;if(c!==null){n.shared.pending=null;var s=c,v=s.next;s.next=null,i===null?u=v:i.next=v,i=s;var b=l.alternate;b!==null&&(b=b.updateQueue,c=b.lastBaseUpdate,c!==i&&(c===null?b.firstBaseUpdate=v:c.next=v,b.lastBaseUpdate=s))}if(u!==null){var x=n.baseState;i=0,b=v=s=null,c=u;do{var g=c.lane&-536870913,S=g!==c.lane;if(S?(P&g)===g:(a&g)===g){g!==0&&g===fa&&(Xi=!0),b!==null&&(b=b.next={lane:0,tag:c.tag,payload:c.payload,callback:null,next:null});l:{var R=l,X=c;g=t;var ml=e;switch(X.tag){case 1:if(R=X.payload,typeof R=="function"){x=R.call(ml,x,g);break l}x=R;break l;case 3:R.flags=R.flags&-65537|128;case 0:if(R=X.payload,g=typeof R=="function"?R.call(ml,x,g):R,g==null)break l;x=j({},x,g);break l;case 2:ue=!0}}g=c.callback,g!==null&&(l.flags|=64,S&&(l.flags|=8192),S=n.callbacks,S===null?n.callbacks=[g]:S.push(g))}else S={lane:g,tag:c.tag,payload:c.payload,callback:c.callback,next:null},b===null?(v=b=S,s=x):b=b.next=S,i|=g;if(c=c.next,c===null){if(c=n.shared.pending,c===null)break;S=c,c=S.next,S.next=null,n.lastBaseUpdate=S,n.shared.pending=null}}while(!0);b===null&&(s=x),n.baseState=s,n.firstBaseUpdate=v,n.lastBaseUpdate=b,u===null&&(n.shared.lanes=0),re|=i,l.lanes=i,l.memoizedState=x}}function Cs(l,t){if(typeof l!="function")throw Error(h(191,l));l.call(t)}function Us(l,t){var e=l.callbacks;if(e!==null)for(l.callbacks=null,l=0;lu?u:8;var i=T.T,c={};T.T=c,uc(l,!1,t,e);try{var s=n(),v=T.S;if(v!==null&&v(c,s),s!==null&&typeof s=="object"&&typeof s.then=="function"){var b=Lh(s,a);ln(l,t,b,ft(l))}else ln(l,t,a,ft(l))}catch(x){ln(l,t,{then:function(){},status:"rejected",reason:x},ft())}finally{D.p=u,i!==null&&c.types!==null&&(i.types=c.types),T.T=i}}function $h(){}function ac(l,t,e,a){if(l.tag!==5)throw Error(h(476));var n=ro(l).queue;oo(l,n,t,Q,e===null?$h:function(){return ho(l),e(a)})}function ro(l){var t=l.memoizedState;if(t!==null)return t;t={memoizedState:Q,baseState:Q,baseQueue:null,queue:{pending:null,lanes:0,dispatch:null,lastRenderedReducer:Zt,lastRenderedState:Q},next:null};var e={};return t.next={memoizedState:e,baseState:e,baseQueue:null,queue:{pending:null,lanes:0,dispatch:null,lastRenderedReducer:Zt,lastRenderedState:e},next:null},l.memoizedState=t,l=l.alternate,l!==null&&(l.memoizedState=t),t}function ho(l){var t=ro(l);t.next===null&&(t=l.alternate.memoizedState),ln(l,t.next.queue,{},ft())}function nc(){return Bl(Sn)}function mo(){return El().memoizedState}function yo(){return El().memoizedState}function Wh(l){for(var t=l.return;t!==null;){switch(t.tag){case 24:case 3:var e=ft();l=ie(e);var a=ce(t,l,e);a!==null&&(Il(a,t,e),$a(a,t,e)),t={cache:Ui()},l.payload=t;return}t=t.return}}function Fh(l,t,e){var a=ft();e={lane:a,revertLane:0,gesture:null,action:e,hasEagerState:!1,eagerState:null,next:null},fu(l)?go(t,e):(e=zi(l,t,e,a),e!==null&&(Il(e,l,a),So(e,t,a)))}function vo(l,t,e){var a=ft();ln(l,t,e,a)}function ln(l,t,e,a){var n={lane:a,revertLane:0,gesture:null,action:e,hasEagerState:!1,eagerState:null,next:null};if(fu(l))go(t,n);else{var u=l.alternate;if(l.lanes===0&&(u===null||u.lanes===0)&&(u=t.lastRenderedReducer,u!==null))try{var i=t.lastRenderedState,c=u(i,e);if(n.hasEagerState=!0,n.eagerState=c,et(c,i))return Zn(l,t,n,0),vl===null&&Qn(),!1}catch{}finally{}if(e=zi(l,t,n,a),e!==null)return Il(e,l,a),So(e,t,a),!0}return!1}function uc(l,t,e,a){if(a={lane:2,revertLane:qc(),gesture:null,action:a,hasEagerState:!1,eagerState:null,next:null},fu(l)){if(t)throw Error(h(479))}else t=zi(l,e,a,2),t!==null&&Il(t,l,2)}function fu(l){var t=l.alternate;return l===J||t!==null&&t===J}function go(l,t){ha=tu=!0;var e=l.pending;e===null?t.next=t:(t.next=e.next,e.next=t),l.pending=t}function So(l,t,e){if((e&4194048)!==0){var a=t.lanes;a&=l.pendingLanes,e|=a,t.lanes=e,zf(l,e)}}var tn={readContext:Bl,use:nu,useCallback:Tl,useContext:Tl,useEffect:Tl,useImperativeHandle:Tl,useLayoutEffect:Tl,useInsertionEffect:Tl,useMemo:Tl,useReducer:Tl,useRef:Tl,useState:Tl,useDebugValue:Tl,useDeferredValue:Tl,useTransition:Tl,useSyncExternalStore:Tl,useId:Tl,useHostTransitionStatus:Tl,useFormState:Tl,useActionState:Tl,useOptimistic:Tl,useMemoCache:Tl,useCacheRefresh:Tl};tn.useEffectEvent=Tl;var bo={readContext:Bl,use:nu,useCallback:function(l,t){return Zl().memoizedState=[l,t===void 0?null:t],l},useContext:Bl,useEffect:to,useImperativeHandle:function(l,t,e){e=e!=null?e.concat([l]):null,iu(4194308,4,uo.bind(null,t,l),e)},useLayoutEffect:function(l,t){return iu(4194308,4,l,t)},useInsertionEffect:function(l,t){iu(4,2,l,t)},useMemo:function(l,t){var e=Zl();t=t===void 0?null:t;var a=l();if(Ge){It(!0);try{l()}finally{It(!1)}}return e.memoizedState=[a,t],a},useReducer:function(l,t,e){var a=Zl();if(e!==void 0){var n=e(t);if(Ge){It(!0);try{e(t)}finally{It(!1)}}}else n=t;return a.memoizedState=a.baseState=n,l={pending:null,lanes:0,dispatch:null,lastRenderedReducer:l,lastRenderedState:n},a.queue=l,l=l.dispatch=Fh.bind(null,J,l),[a.memoizedState,l]},useRef:function(l){var t=Zl();return l={current:l},t.memoizedState=l},useState:function(l){l=Ii(l);var t=l.queue,e=vo.bind(null,J,t);return t.dispatch=e,[l.memoizedState,e]},useDebugValue:tc,useDeferredValue:function(l,t){var e=Zl();return ec(e,l,t)},useTransition:function(){var l=Ii(!1);return l=oo.bind(null,J,l.queue,!0,!1),Zl().memoizedState=l,[!1,l]},useSyncExternalStore:function(l,t,e){var a=J,n=Zl();if(al){if(e===void 0)throw Error(h(407));e=e()}else{if(e=t(),vl===null)throw Error(h(349));(P&127)!==0||Gs(a,t,e)}n.memoizedState=e;var u={value:e,getSnapshot:t};return n.queue=u,to(Qs.bind(null,a,u,l),[l]),a.flags|=2048,ya(9,{destroy:void 0},Xs.bind(null,a,u,e,t),null),e},useId:function(){var l=Zl(),t=vl.identifierPrefix;if(al){var e=Mt,a=Nt;e=(a&~(1<<32-tt(a)-1)).toString(32)+e,t="_"+t+"R_"+e,e=eu++,0<\/script>",u=u.removeChild(u.firstChild);break;case"select":u=typeof a.is=="string"?i.createElement("select",{is:a.is}):i.createElement("select"),a.multiple?u.multiple=!0:a.size&&(u.size=a.size);break;default:u=typeof a.is=="string"?i.createElement(n,{is:a.is}):i.createElement(n)}}u[Rl]=t,u[Jl]=a;l:for(i=t.child;i!==null;){if(i.tag===5||i.tag===6)u.appendChild(i.stateNode);else if(i.tag!==4&&i.tag!==27&&i.child!==null){i.child.return=i,i=i.child;continue}if(i===t)break l;for(;i.sibling===null;){if(i.return===null||i.return===t)break l;i=i.return}i.sibling.return=i.return,i=i.sibling}t.stateNode=u;l:switch(Yl(u,n,a),n){case"button":case"input":case"select":case"textarea":a=!!a.autoFocus;break l;case"img":a=!0;break l;default:a=!1}a&&Vt(t)}}return Sl(t),bc(t,t.type,l===null?null:l.memoizedProps,t.pendingProps,e),null;case 6:if(l&&t.stateNode!=null)l.memoizedProps!==a&&Vt(t);else{if(typeof a!="string"&&t.stateNode===null)throw Error(h(166));if(l=$.current,ia(t)){if(l=t.stateNode,e=t.memoizedProps,a=null,n=Hl,n!==null)switch(n.tag){case 27:case 5:a=n.memoizedProps}l[Rl]=t,l=!!(l.nodeValue===e||a!==null&&a.suppressHydrationWarning===!0||Yd(l.nodeValue,e)),l||ae(t,!0)}else l=Ou(l).createTextNode(a),l[Rl]=t,t.stateNode=l}return Sl(t),null;case 31:if(e=t.memoizedState,l===null||l.memoizedState!==null){if(a=ia(t),e!==null){if(l===null){if(!a)throw Error(h(318));if(l=t.memoizedState,l=l!==null?l.dehydrated:null,!l)throw Error(h(557));l[Rl]=t}else Ce(),(t.flags&128)===0&&(t.memoizedState=null),t.flags|=4;Sl(t),l=!1}else e=Ni(),l!==null&&l.memoizedState!==null&&(l.memoizedState.hydrationErrors=e),l=!0;if(!l)return t.flags&256?(ut(t),t):(ut(t),null);if((t.flags&128)!==0)throw Error(h(558))}return Sl(t),null;case 13:if(a=t.memoizedState,l===null||l.memoizedState!==null&&l.memoizedState.dehydrated!==null){if(n=ia(t),a!==null&&a.dehydrated!==null){if(l===null){if(!n)throw Error(h(318));if(n=t.memoizedState,n=n!==null?n.dehydrated:null,!n)throw Error(h(317));n[Rl]=t}else Ce(),(t.flags&128)===0&&(t.memoizedState=null),t.flags|=4;Sl(t),n=!1}else n=Ni(),l!==null&&l.memoizedState!==null&&(l.memoizedState.hydrationErrors=n),n=!0;if(!n)return t.flags&256?(ut(t),t):(ut(t),null)}return ut(t),(t.flags&128)!==0?(t.lanes=e,t):(e=a!==null,l=l!==null&&l.memoizedState!==null,e&&(a=t.child,n=null,a.alternate!==null&&a.alternate.memoizedState!==null&&a.alternate.memoizedState.cachePool!==null&&(n=a.alternate.memoizedState.cachePool.pool),u=null,a.memoizedState!==null&&a.memoizedState.cachePool!==null&&(u=a.memoizedState.cachePool.pool),u!==n&&(a.flags|=2048)),e!==l&&e&&(t.child.flags|=8192),hu(t,t.updateQueue),Sl(t),null);case 4:return xl(),l===null&&Qc(t.stateNode.containerInfo),Sl(t),null;case 10:return Xt(t.type),Sl(t),null;case 19:if(A(Al),a=t.memoizedState,a===null)return Sl(t),null;if(n=(t.flags&128)!==0,u=a.rendering,u===null)if(n)an(a,!1);else{if(zl!==0||l!==null&&(l.flags&128)!==0)for(l=t.child;l!==null;){if(u=lu(l),u!==null){for(t.flags|=128,an(a,!1),l=u.updateQueue,t.updateQueue=l,hu(t,l),t.subtreeFlags=0,l=e,e=t.child;e!==null;)ys(e,l),e=e.sibling;return C(Al,Al.current&1|2),al&&Yt(t,a.treeForkCount),t.child}l=l.sibling}a.tail!==null&&Pl()>Su&&(t.flags|=128,n=!0,an(a,!1),t.lanes=4194304)}else{if(!n)if(l=lu(u),l!==null){if(t.flags|=128,n=!0,l=l.updateQueue,t.updateQueue=l,hu(t,l),an(a,!0),a.tail===null&&a.tailMode==="hidden"&&!u.alternate&&!al)return Sl(t),null}else 2*Pl()-a.renderingStartTime>Su&&e!==536870912&&(t.flags|=128,n=!0,an(a,!1),t.lanes=4194304);a.isBackwards?(u.sibling=t.child,t.child=u):(l=a.last,l!==null?l.sibling=u:t.child=u,a.last=u)}return a.tail!==null?(l=a.tail,a.rendering=l,a.tail=l.sibling,a.renderingStartTime=Pl(),l.sibling=null,e=Al.current,C(Al,n?e&1|2:e&1),al&&Yt(t,a.treeForkCount),l):(Sl(t),null);case 22:case 23:return ut(t),Zi(),a=t.memoizedState!==null,l!==null?l.memoizedState!==null!==a&&(t.flags|=8192):a&&(t.flags|=8192),a?(e&536870912)!==0&&(t.flags&128)===0&&(Sl(t),t.subtreeFlags&6&&(t.flags|=8192)):Sl(t),e=t.updateQueue,e!==null&&hu(t,e.retryQueue),e=null,l!==null&&l.memoizedState!==null&&l.memoizedState.cachePool!==null&&(e=l.memoizedState.cachePool.pool),a=null,t.memoizedState!==null&&t.memoizedState.cachePool!==null&&(a=t.memoizedState.cachePool.pool),a!==e&&(t.flags|=2048),l!==null&&A(He),null;case 24:return e=null,l!==null&&(e=l.memoizedState.cache),t.memoizedState.cache!==e&&(t.flags|=2048),Xt(_l),Sl(t),null;case 25:return null;case 30:return null}throw Error(h(156,t.tag))}function em(l,t){switch(ji(t),t.tag){case 1:return l=t.flags,l&65536?(t.flags=l&-65537|128,t):null;case 3:return Xt(_l),xl(),l=t.flags,(l&65536)!==0&&(l&128)===0?(t.flags=l&-65537|128,t):null;case 26:case 27:case 5:return An(t),null;case 31:if(t.memoizedState!==null){if(ut(t),t.alternate===null)throw Error(h(340));Ce()}return l=t.flags,l&65536?(t.flags=l&-65537|128,t):null;case 13:if(ut(t),l=t.memoizedState,l!==null&&l.dehydrated!==null){if(t.alternate===null)throw Error(h(340));Ce()}return l=t.flags,l&65536?(t.flags=l&-65537|128,t):null;case 19:return A(Al),null;case 4:return xl(),null;case 10:return Xt(t.type),null;case 22:case 23:return ut(t),Zi(),l!==null&&A(He),l=t.flags,l&65536?(t.flags=l&-65537|128,t):null;case 24:return Xt(_l),null;case 25:return null;default:return null}}function Lo(l,t){switch(ji(t),t.tag){case 3:Xt(_l),xl();break;case 26:case 27:case 5:An(t);break;case 4:xl();break;case 31:t.memoizedState!==null&&ut(t);break;case 13:ut(t);break;case 19:A(Al);break;case 10:Xt(t.type);break;case 22:case 23:ut(t),Zi(),l!==null&&A(He);break;case 24:Xt(_l)}}function nn(l,t){try{var e=t.updateQueue,a=e!==null?e.lastEffect:null;if(a!==null){var n=a.next;e=n;do{if((e.tag&l)===l){a=void 0;var u=e.create,i=e.inst;a=u(),i.destroy=a}e=e.next}while(e!==n)}}catch(c){ol(t,t.return,c)}}function oe(l,t,e){try{var a=t.updateQueue,n=a!==null?a.lastEffect:null;if(n!==null){var u=n.next;a=u;do{if((a.tag&l)===l){var i=a.inst,c=i.destroy;if(c!==void 0){i.destroy=void 0,n=t;var s=e,v=c;try{v()}catch(b){ol(n,s,b)}}}a=a.next}while(a!==u)}}catch(b){ol(t,t.return,b)}}function Vo(l){var t=l.updateQueue;if(t!==null){var e=l.stateNode;try{Us(t,e)}catch(a){ol(l,l.return,a)}}}function Ko(l,t,e){e.props=Xe(l.type,l.memoizedProps),e.state=l.memoizedState;try{e.componentWillUnmount()}catch(a){ol(l,t,a)}}function un(l,t){try{var e=l.ref;if(e!==null){switch(l.tag){case 26:case 27:case 5:var a=l.stateNode;break;case 30:a=l.stateNode;break;default:a=l.stateNode}typeof e=="function"?l.refCleanup=e(a):e.current=a}}catch(n){ol(l,t,n)}}function Dt(l,t){var e=l.ref,a=l.refCleanup;if(e!==null)if(typeof a=="function")try{a()}catch(n){ol(l,t,n)}finally{l.refCleanup=null,l=l.alternate,l!=null&&(l.refCleanup=null)}else if(typeof e=="function")try{e(null)}catch(n){ol(l,t,n)}else e.current=null}function Jo(l){var t=l.type,e=l.memoizedProps,a=l.stateNode;try{l:switch(t){case"button":case"input":case"select":case"textarea":e.autoFocus&&a.focus();break l;case"img":e.src?a.src=e.src:e.srcSet&&(a.srcset=e.srcSet)}}catch(n){ol(l,l.return,n)}}function pc(l,t,e){try{var a=l.stateNode;Am(a,l.type,e,t),a[Jl]=t}catch(n){ol(l,l.return,n)}}function wo(l){return l.tag===5||l.tag===3||l.tag===26||l.tag===27&&ge(l.type)||l.tag===4}function Tc(l){l:for(;;){for(;l.sibling===null;){if(l.return===null||wo(l.return))return null;l=l.return}for(l.sibling.return=l.return,l=l.sibling;l.tag!==5&&l.tag!==6&&l.tag!==18;){if(l.tag===27&&ge(l.type)||l.flags&2||l.child===null||l.tag===4)continue l;l.child.return=l,l=l.child}if(!(l.flags&2))return l.stateNode}}function zc(l,t,e){var a=l.tag;if(a===5||a===6)l=l.stateNode,t?(e.nodeType===9?e.body:e.nodeName==="HTML"?e.ownerDocument.body:e).insertBefore(l,t):(t=e.nodeType===9?e.body:e.nodeName==="HTML"?e.ownerDocument.body:e,t.appendChild(l),e=e._reactRootContainer,e!=null||t.onclick!==null||(t.onclick=Ht));else if(a!==4&&(a===27&&ge(l.type)&&(e=l.stateNode,t=null),l=l.child,l!==null))for(zc(l,t,e),l=l.sibling;l!==null;)zc(l,t,e),l=l.sibling}function mu(l,t,e){var a=l.tag;if(a===5||a===6)l=l.stateNode,t?e.insertBefore(l,t):e.appendChild(l);else if(a!==4&&(a===27&&ge(l.type)&&(e=l.stateNode),l=l.child,l!==null))for(mu(l,t,e),l=l.sibling;l!==null;)mu(l,t,e),l=l.sibling}function ko(l){var t=l.stateNode,e=l.memoizedProps;try{for(var a=l.type,n=t.attributes;n.length;)t.removeAttributeNode(n[0]);Yl(t,a,e),t[Rl]=l,t[Jl]=e}catch(u){ol(l,l.return,u)}}var Kt=!1,Nl=!1,xc=!1,$o=typeof WeakSet=="function"?WeakSet:Set,Ul=null;function am(l,t){if(l=l.containerInfo,Vc=Hu,l=is(l),vi(l)){if("selectionStart"in l)var e={start:l.selectionStart,end:l.selectionEnd};else l:{e=(e=l.ownerDocument)&&e.defaultView||window;var a=e.getSelection&&e.getSelection();if(a&&a.rangeCount!==0){e=a.anchorNode;var n=a.anchorOffset,u=a.focusNode;a=a.focusOffset;try{e.nodeType,u.nodeType}catch{e=null;break l}var i=0,c=-1,s=-1,v=0,b=0,x=l,g=null;t:for(;;){for(var S;x!==e||n!==0&&x.nodeType!==3||(c=i+n),x!==u||a!==0&&x.nodeType!==3||(s=i+a),x.nodeType===3&&(i+=x.nodeValue.length),(S=x.firstChild)!==null;)g=x,x=S;for(;;){if(x===l)break t;if(g===e&&++v===n&&(c=i),g===u&&++b===a&&(s=i),(S=x.nextSibling)!==null)break;x=g,g=x.parentNode}x=S}e=c===-1||s===-1?null:{start:c,end:s}}else e=null}e=e||{start:0,end:0}}else e=null;for(Kc={focusedElem:l,selectionRange:e},Hu=!1,Ul=t;Ul!==null;)if(t=Ul,l=t.child,(t.subtreeFlags&1028)!==0&&l!==null)l.return=t,Ul=l;else for(;Ul!==null;){switch(t=Ul,u=t.alternate,l=t.flags,t.tag){case 0:if((l&4)!==0&&(l=t.updateQueue,l=l!==null?l.events:null,l!==null))for(e=0;e title"))),Yl(u,a,e),u[Rl]=l,Cl(u),a=u;break l;case"link":var i=tr("link","href",n).get(a+(e.href||""));if(i){for(var c=0;cml&&(i=ml,ml=X,X=i);var m=ns(c,X),o=ns(c,ml);if(m&&o&&(S.rangeCount!==1||S.anchorNode!==m.node||S.anchorOffset!==m.offset||S.focusNode!==o.node||S.focusOffset!==o.offset)){var y=x.createRange();y.setStart(m.node,m.offset),S.removeAllRanges(),X>ml?(S.addRange(y),S.extend(o.node,o.offset)):(y.setEnd(o.node,o.offset),S.addRange(y))}}}}for(x=[],S=c;S=S.parentNode;)S.nodeType===1&&x.push({element:S,left:S.scrollLeft,top:S.scrollTop});for(typeof c.focus=="function"&&c.focus(),c=0;ce?32:e,T.T=null,e=Mc,Mc=null;var u=me,i=Wt;if(Dl=0,pa=me=null,Wt=0,(cl&6)!==0)throw Error(h(331));var c=cl;if(cl|=4,id(u.current),ad(u,u.current,i,e),cl=c,rn(0,!1),lt&&typeof lt.onPostCommitFiberRoot=="function")try{lt.onPostCommitFiberRoot(Na,u)}catch{}return!0}finally{D.p=n,T.T=a,Ad(l,t)}}function _d(l,t,e){t=mt(e,t),t=sc(l.stateNode,t,2),l=ce(l,t,2),l!==null&&(Da(l,2),Ct(l))}function ol(l,t,e){if(l.tag===3)_d(l,l,e);else for(;t!==null;){if(t.tag===3){_d(t,l,e);break}else if(t.tag===1){var a=t.stateNode;if(typeof t.type.getDerivedStateFromError=="function"||typeof a.componentDidCatch=="function"&&(he===null||!he.has(a))){l=mt(e,l),e=jo(2),a=ce(t,e,2),a!==null&&(Oo(e,a,t,l),Da(a,2),Ct(a));break}}t=t.return}}function Rc(l,t,e){var a=l.pingCache;if(a===null){a=l.pingCache=new im;var n=new Set;a.set(t,n)}else n=a.get(t),n===void 0&&(n=new Set,a.set(t,n));n.has(e)||(_c=!0,n.add(e),l=dm.bind(null,l,t,e),t.then(l,l))}function dm(l,t,e){var a=l.pingCache;a!==null&&a.delete(t),l.pingedLanes|=l.suspendedLanes&e,l.warmLanes&=~e,vl===l&&(P&e)===e&&(zl===4||zl===3&&(P&62914560)===P&&300>Pl()-gu?(cl&2)===0&&Ta(l,0):jc|=e,ba===P&&(ba=0)),Ct(l)}function jd(l,t){t===0&&(t=pf()),l=Me(l,t),l!==null&&(Da(l,t),Ct(l))}function rm(l){var t=l.memoizedState,e=0;t!==null&&(e=t.retryLane),jd(l,e)}function hm(l,t){var e=0;switch(l.tag){case 31:case 13:var a=l.stateNode,n=l.memoizedState;n!==null&&(e=n.retryLane);break;case 19:a=l.stateNode;break;case 22:a=l.stateNode._retryCache;break;default:throw Error(h(314))}a!==null&&a.delete(t),jd(l,e)}function mm(l,t){return wu(l,t)}var Au=null,xa=null,Hc=!1,Eu=!1,Bc=!1,ve=0;function Ct(l){l!==xa&&l.next===null&&(xa===null?Au=xa=l:xa=xa.next=l),Eu=!0,Hc||(Hc=!0,vm())}function rn(l,t){if(!Bc&&Eu){Bc=!0;do for(var e=!1,a=Au;a!==null;){if(l!==0){var n=a.pendingLanes;if(n===0)var u=0;else{var i=a.suspendedLanes,c=a.pingedLanes;u=(1<<31-tt(42|l)+1)-1,u&=n&~(i&~c),u=u&201326741?u&201326741|1:u?u|2:0}u!==0&&(e=!0,Dd(a,u))}else u=P,u=Nn(a,a===vl?u:0,a.cancelPendingCommit!==null||a.timeoutHandle!==-1),(u&3)===0||Ma(a,u)||(e=!0,Dd(a,u));a=a.next}while(e);Bc=!1}}function ym(){Od()}function Od(){Eu=Hc=!1;var l=0;ve!==0&&_m()&&(l=ve);for(var t=Pl(),e=null,a=Au;a!==null;){var n=a.next,u=Nd(a,t);u===0?(a.next=null,e===null?Au=n:e.next=n,n===null&&(xa=e)):(e=a,(l!==0||(u&3)!==0)&&(Eu=!0)),a=n}Dl!==0&&Dl!==5||rn(l),ve!==0&&(ve=0)}function Nd(l,t){for(var e=l.suspendedLanes,a=l.pingedLanes,n=l.expirationTimes,u=l.pendingLanes&-62914561;0c)break;var b=s.transferSize,x=s.initiatorType;b&&Gd(x)&&(s=s.responseEnd,i+=b*(s"u"?null:document;function Fd(l,t,e){var a=Aa;if(a&&typeof t=="string"&&t){var n=rt(t);n='link[rel="'+l+'"][href="'+n+'"]',typeof e=="string"&&(n+='[crossorigin="'+e+'"]'),Wd.has(n)||(Wd.add(n),l={rel:l,crossOrigin:e,href:t},a.querySelector(n)===null&&(t=a.createElement("link"),Yl(t,"link",l),Cl(t),a.head.appendChild(t)))}}function Hm(l){Ft.D(l),Fd("dns-prefetch",l,null)}function Bm(l,t){Ft.C(l,t),Fd("preconnect",l,t)}function qm(l,t,e){Ft.L(l,t,e);var a=Aa;if(a&&l&&t){var n='link[rel="preload"][as="'+rt(t)+'"]';t==="image"&&e&&e.imageSrcSet?(n+='[imagesrcset="'+rt(e.imageSrcSet)+'"]',typeof e.imageSizes=="string"&&(n+='[imagesizes="'+rt(e.imageSizes)+'"]')):n+='[href="'+rt(l)+'"]';var u=n;switch(t){case"style":u=Ea(l);break;case"script":u=_a(l)}pt.has(u)||(l=j({rel:"preload",href:t==="image"&&e&&e.imageSrcSet?void 0:l,as:t},e),pt.set(u,l),a.querySelector(n)!==null||t==="style"&&a.querySelector(vn(u))||t==="script"&&a.querySelector(gn(u))||(t=a.createElement("link"),Yl(t,"link",l),Cl(t),a.head.appendChild(t)))}}function Ym(l,t){Ft.m(l,t);var e=Aa;if(e&&l){var a=t&&typeof t.as=="string"?t.as:"script",n='link[rel="modulepreload"][as="'+rt(a)+'"][href="'+rt(l)+'"]',u=n;switch(a){case"audioworklet":case"paintworklet":case"serviceworker":case"sharedworker":case"worker":case"script":u=_a(l)}if(!pt.has(u)&&(l=j({rel:"modulepreload",href:l},t),pt.set(u,l),e.querySelector(n)===null)){switch(a){case"audioworklet":case"paintworklet":case"serviceworker":case"sharedworker":case"worker":case"script":if(e.querySelector(gn(u)))return}a=e.createElement("link"),Yl(a,"link",l),Cl(a),e.head.appendChild(a)}}}function Gm(l,t,e){Ft.S(l,t,e);var a=Aa;if(a&&l){var n=we(a).hoistableStyles,u=Ea(l);t=t||"default";var i=n.get(u);if(!i){var c={loading:0,preload:null};if(i=a.querySelector(vn(u)))c.loading=5;else{l=j({rel:"stylesheet",href:l,"data-precedence":t},e),(e=pt.get(u))&&Ic(l,e);var s=i=a.createElement("link");Cl(s),Yl(s,"link",l),s._p=new Promise(function(v,b){s.onload=v,s.onerror=b}),s.addEventListener("load",function(){c.loading|=1}),s.addEventListener("error",function(){c.loading|=2}),c.loading|=4,Mu(i,t,a)}i={type:"stylesheet",instance:i,count:1,state:c},n.set(u,i)}}}function Xm(l,t){Ft.X(l,t);var e=Aa;if(e&&l){var a=we(e).hoistableScripts,n=_a(l),u=a.get(n);u||(u=e.querySelector(gn(n)),u||(l=j({src:l,async:!0},t),(t=pt.get(n))&&Pc(l,t),u=e.createElement("script"),Cl(u),Yl(u,"link",l),e.head.appendChild(u)),u={type:"script",instance:u,count:1,state:null},a.set(n,u))}}function Qm(l,t){Ft.M(l,t);var e=Aa;if(e&&l){var a=we(e).hoistableScripts,n=_a(l),u=a.get(n);u||(u=e.querySelector(gn(n)),u||(l=j({src:l,async:!0,type:"module"},t),(t=pt.get(n))&&Pc(l,t),u=e.createElement("script"),Cl(u),Yl(u,"link",l),e.head.appendChild(u)),u={type:"script",instance:u,count:1,state:null},a.set(n,u))}}function Id(l,t,e,a){var n=(n=$.current)?Nu(n):null;if(!n)throw Error(h(446));switch(l){case"meta":case"title":return null;case"style":return typeof e.precedence=="string"&&typeof e.href=="string"?(t=Ea(e.href),e=we(n).hoistableStyles,a=e.get(t),a||(a={type:"style",instance:null,count:0,state:null},e.set(t,a)),a):{type:"void",instance:null,count:0,state:null};case"link":if(e.rel==="stylesheet"&&typeof e.href=="string"&&typeof e.precedence=="string"){l=Ea(e.href);var u=we(n).hoistableStyles,i=u.get(l);if(i||(n=n.ownerDocument||n,i={type:"stylesheet",instance:null,count:0,state:{loading:0,preload:null}},u.set(l,i),(u=n.querySelector(vn(l)))&&!u._p&&(i.instance=u,i.state.loading=5),pt.has(l)||(e={rel:"preload",as:"style",href:e.href,crossOrigin:e.crossOrigin,integrity:e.integrity,media:e.media,hrefLang:e.hrefLang,referrerPolicy:e.referrerPolicy},pt.set(l,e),u||Zm(n,l,e,i.state))),t&&a===null)throw Error(h(528,""));return i}if(t&&a!==null)throw Error(h(529,""));return null;case"script":return t=e.async,e=e.src,typeof e=="string"&&t&&typeof t!="function"&&typeof t!="symbol"?(t=_a(e),e=we(n).hoistableScripts,a=e.get(t),a||(a={type:"script",instance:null,count:0,state:null},e.set(t,a)),a):{type:"void",instance:null,count:0,state:null};default:throw Error(h(444,l))}}function Ea(l){return'href="'+rt(l)+'"'}function vn(l){return'link[rel="stylesheet"]['+l+"]"}function Pd(l){return j({},l,{"data-precedence":l.precedence,precedence:null})}function Zm(l,t,e,a){l.querySelector('link[rel="preload"][as="style"]['+t+"]")?a.loading=1:(t=l.createElement("link"),a.preload=t,t.addEventListener("load",function(){return a.loading|=1}),t.addEventListener("error",function(){return a.loading|=2}),Yl(t,"link",e),Cl(t),l.head.appendChild(t))}function _a(l){return'[src="'+rt(l)+'"]'}function gn(l){return"script[async]"+l}function lr(l,t,e){if(t.count++,t.instance===null)switch(t.type){case"style":var a=l.querySelector('style[data-href~="'+rt(e.href)+'"]');if(a)return t.instance=a,Cl(a),a;var n=j({},e,{"data-href":e.href,"data-precedence":e.precedence,href:null,precedence:null});return a=(l.ownerDocument||l).createElement("style"),Cl(a),Yl(a,"style",n),Mu(a,e.precedence,l),t.instance=a;case"stylesheet":n=Ea(e.href);var u=l.querySelector(vn(n));if(u)return t.state.loading|=4,t.instance=u,Cl(u),u;a=Pd(e),(n=pt.get(n))&&Ic(a,n),u=(l.ownerDocument||l).createElement("link"),Cl(u);var i=u;return i._p=new Promise(function(c,s){i.onload=c,i.onerror=s}),Yl(u,"link",a),t.state.loading|=4,Mu(u,e.precedence,l),t.instance=u;case"script":return u=_a(e.src),(n=l.querySelector(gn(u)))?(t.instance=n,Cl(n),n):(a=e,(n=pt.get(u))&&(a=j({},e),Pc(a,n)),l=l.ownerDocument||l,n=l.createElement("script"),Cl(n),Yl(n,"link",a),l.head.appendChild(n),t.instance=n);case"void":return null;default:throw Error(h(443,t.type))}else t.type==="stylesheet"&&(t.state.loading&4)===0&&(a=t.instance,t.state.loading|=4,Mu(a,e.precedence,l));return t.instance}function Mu(l,t,e){for(var a=e.querySelectorAll('link[rel="stylesheet"][data-precedence],style[data-precedence]'),n=a.length?a[a.length-1]:null,u=n,i=0;i title"):null)}function Lm(l,t,e){if(e===1||t.itemProp!=null)return!1;switch(l){case"meta":case"title":return!0;case"style":if(typeof t.precedence!="string"||typeof t.href!="string"||t.href==="")break;return!0;case"link":if(typeof t.rel!="string"||typeof t.href!="string"||t.href===""||t.onLoad||t.onError)break;switch(t.rel){case"stylesheet":return l=t.disabled,typeof t.precedence=="string"&&l==null;default:return!0}case"script":if(t.async&&typeof t.async!="function"&&typeof t.async!="symbol"&&!t.onLoad&&!t.onError&&t.src&&typeof t.src=="string")return!0}return!1}function ar(l){return!(l.type==="stylesheet"&&(l.state.loading&3)===0)}function Vm(l,t,e,a){if(e.type==="stylesheet"&&(typeof a.media!="string"||matchMedia(a.media).matches!==!1)&&(e.state.loading&4)===0){if(e.instance===null){var n=Ea(a.href),u=t.querySelector(vn(n));if(u){t=u._p,t!==null&&typeof t=="object"&&typeof t.then=="function"&&(l.count++,l=Cu.bind(l),t.then(l,l)),e.state.loading|=4,e.instance=u,Cl(u);return}u=t.ownerDocument||t,a=Pd(a),(n=pt.get(n))&&Ic(a,n),u=u.createElement("link"),Cl(u);var i=u;i._p=new Promise(function(c,s){i.onload=c,i.onerror=s}),Yl(u,"link",a),e.instance=u}l.stylesheets===null&&(l.stylesheets=new Map),l.stylesheets.set(e,t),(t=e.state.preload)&&(e.state.loading&3)===0&&(l.count++,e=Cu.bind(l),t.addEventListener("load",e),t.addEventListener("error",e))}}var lf=0;function Km(l,t){return l.stylesheets&&l.count===0&&Ru(l,l.stylesheets),0lf?50:800)+t);return l.unsuspend=e,function(){l.unsuspend=null,clearTimeout(a),clearTimeout(n)}}:null}function Cu(){if(this.count--,this.count===0&&(this.imgCount===0||!this.waitingForImages)){if(this.stylesheets)Ru(this,this.stylesheets);else if(this.unsuspend){var l=this.unsuspend;this.unsuspend=null,l()}}}var Uu=null;function Ru(l,t){l.stylesheets=null,l.unsuspend!==null&&(l.count++,Uu=new Map,t.forEach(Jm,l),Uu=null,Cu.call(l))}function Jm(l,t){if(!(t.state.loading&4)){var e=Uu.get(l);if(e)var a=e.get(null);else{e=new Map,Uu.set(l,e);for(var n=l.querySelectorAll("link[data-precedence],style[data-precedence]"),u=0;u"u"||typeof __REACT_DEVTOOLS_GLOBAL_HOOK__.checkDCE!="function"))try{__REACT_DEVTOOLS_GLOBAL_HOOK__.checkDCE(d)}catch(N){console.error(N)}}return d(),of.exports=fy(),of.exports}var oy=sy();const dy=Nr(oy),Dr="";async function Tt(d,N){const M=await fetch(`${Dr}${d}`,{...N,credentials:"same-origin",headers:{"Content-Type":"application/json",...N==null?void 0:N.headers}});if(M.status===401)throw window.location.hash="#login",new Error("Unauthorized");if(!M.ok){const h=await M.json().catch(()=>({}));throw new Error(h.error||`HTTP ${M.status}`)}return M.json()}async function ry(d){const N=await fetch(`${Dr}${d}`,{credentials:"same-origin"});if(N.status===401)throw window.location.hash="#login",new Error("Unauthorized");if(!N.ok)throw new Error(`HTTP ${N.status}`);return N.text()}const st={login:d=>Tt("/admin/login",{method:"POST",body:JSON.stringify({token:d})}),signOutEverywhere:()=>Tt("/admin/api/sign-out-everywhere",{method:"POST"}),stats:()=>Tt("/admin/api/stats"),health:()=>Tt("/admin/api/health-indicators"),agents:()=>Tt("/admin/api/agents"),requests:(d=1,N="")=>Tt(`/admin/api/requests?page=${d}${N}`),apiKeys:()=>Tt("/admin/api/api-keys"),createApiKey:d=>Tt("/admin/api/api-keys",{method:"POST",body:JSON.stringify({name:d})}),revokeApiKey:d=>Tt("/admin/api/api-keys/revoke",{method:"POST",body:JSON.stringify({name:d})}),updateClientTtl:(d,N)=>Tt("/admin/api/update-client-ttl",{method:"POST",body:JSON.stringify({clientId:d,tokenTtl:N})}),revokeClient:d=>Tt("/admin/api/revoke-client",{method:"POST",body:JSON.stringify({clientId:d})}),calibrationProfile:d=>Tt(`/admin/api/calibration/profile${d?`?holder=${encodeURIComponent(d)}`:""}`),calibrationChart:(d,N)=>ry(`/admin/api/calibration/charts/${encodeURIComponent(d)}${N?`?holder=${encodeURIComponent(N)}`:""}`)};function hy({onLogin:d}){const[N,M]=k.useState(""),[h,_]=k.useState(""),[Y,U]=k.useState(!1),Z=async O=>{O.preventDefault(),_(""),U(!0);try{await st.login(N),M(""),d()}catch{_("Invalid token.")}finally{U(!1)}};return f.jsx("div",{className:"login-page",children:f.jsxs("div",{className:"login-box",children:[f.jsx("div",{className:"login-logo",children:"GBrain"}),f.jsxs("div",{style:{background:"rgba(136, 170, 255, 0.08)",border:"1px solid rgba(136, 170, 255, 0.2)",borderRadius:8,padding:"14px 16px",marginBottom:20,fontSize:13,lineHeight:1.5,color:"var(--text-secondary)"},children:[f.jsx("div",{style:{fontWeight:600,color:"var(--text-primary)",marginBottom:6},children:"🔒 This is a protected dashboard"}),"Ask your AI agent for the admin login link:",f.jsx("div",{style:{background:"rgba(0,0,0,0.3)",borderRadius:6,padding:"8px 12px",marginTop:8,fontFamily:"var(--font-mono)",fontSize:12,color:"#88aaff",wordBreak:"break-all"},children:'"Give me the GBrain admin login link"'}),f.jsx("div",{style:{marginTop:8,fontSize:12,color:"var(--text-muted)"},children:"Each link is single-use. Your agent generates a fresh one each time."})]}),f.jsxs("details",{style:{marginBottom:16},children:[f.jsx("summary",{style:{cursor:"pointer",fontSize:13,color:"var(--text-muted)"},children:"Or paste bootstrap token manually"}),f.jsxs("form",{onSubmit:Z,style:{marginTop:12},children:[f.jsx("div",{style:{marginBottom:12},children:f.jsx("input",{type:"password",placeholder:"Admin Token",value:N,onChange:O=>M(O.target.value)})}),f.jsx("button",{className:"btn btn-primary",style:{width:"100%"},disabled:Y,children:Y?"Authenticating...":"Submit"}),h&&f.jsx("div",{className:"login-error",children:h})]})]})]})})}function my(){const[d,N]=k.useState({connected_agents:0,requests_today:0,active_tokens:0}),[M,h]=k.useState({expiring_soon:0,error_rate:"0%"}),[_,Y]=k.useState([]),[U,Z]=k.useState("connecting"),O=k.useRef(null);k.useEffect(()=>{st.stats().then(N).catch(()=>{}),st.health().then(h).catch(()=>{});const H=new EventSource("/admin/events");O.current=H,H.onopen=()=>Z("connected"),H.onmessage=E=>{try{const I=JSON.parse(E.data);Y(L=>[I,...L].slice(0,50))}catch{}},H.onerror=()=>{Z("disconnected"),setTimeout(()=>{Z("connecting"),H.close()},3e3)};const j=setInterval(()=>{st.stats().then(N).catch(()=>{}),st.health().then(h).catch(()=>{})},3e4);return()=>{H.close(),clearInterval(j)}},[]);const p=H=>{const j=Date.now()-new Date(H).getTime();return j<6e4?`${Math.floor(j/1e3)}s ago`:j<36e5?`${Math.floor(j/6e4)} min ago`:`${Math.floor(j/36e5)}h ago`};return f.jsxs(f.Fragment,{children:[f.jsx("h1",{className:"page-title",children:"Dashboard"}),f.jsxs("div",{style:{display:"flex",gap:24},children:[f.jsxs("div",{style:{flex:1},children:[f.jsxs("div",{className:"metrics",children:[f.jsxs("div",{className:"metric",children:[f.jsx("div",{className:"metric-value",children:d.connected_agents}),f.jsx("div",{className:"metric-label",children:"Connected Agents"})]}),f.jsxs("div",{className:"metric",children:[f.jsx("div",{className:"metric-value",children:d.requests_today}),f.jsx("div",{className:"metric-label",children:"Requests Today"})]}),f.jsxs("div",{className:"metric",children:[f.jsx("div",{className:"metric-value",children:d.active_tokens}),f.jsx("div",{className:"metric-label",children:"Active Tokens"})]})]}),f.jsxs("h2",{className:"section-title",children:["Live Activity",f.jsx("span",{style:{marginLeft:8,fontSize:10,color:U==="connected"?"var(--success)":U==="connecting"?"var(--warning)":"var(--error)"},children:U==="connected"?"● connected":U==="connecting"?"● connecting...":"● disconnected"})]}),f.jsx("div",{className:"feed",children:_.length===0?f.jsx("div",{className:"feed-empty",children:U==="connected"?"No requests yet. Agents will appear when they connect.":"Connecting..."}):f.jsxs("table",{children:[f.jsx("thead",{children:f.jsxs("tr",{children:[f.jsx("th",{children:"Agent"}),f.jsx("th",{children:"Operation"}),f.jsx("th",{children:"Scopes"}),f.jsx("th",{children:"Latency"}),f.jsx("th",{children:"Status"}),f.jsx("th",{children:"Time"})]})}),f.jsx("tbody",{children:_.map((H,j)=>f.jsxs("tr",{children:[f.jsx("td",{className:"mono",children:H.agent}),f.jsx("td",{className:"mono",children:H.operation}),f.jsx("td",{children:H.scopes.split(",").map(E=>f.jsx("span",{className:`badge badge-${E.trim()}`,style:{marginRight:4},children:E.trim()},E))}),f.jsxs("td",{className:"mono",children:[H.latency_ms," ms"]}),f.jsx("td",{children:f.jsx("span",{className:`badge badge-${H.status}`,children:H.status})}),f.jsx("td",{style:{color:"var(--text-secondary)"},children:p(H.timestamp)})]},j))})]})})]}),f.jsxs("div",{style:{width:220},children:[f.jsx("h2",{className:"section-title",children:"Token Health"}),f.jsxs("div",{className:"health-panel",children:[f.jsxs("div",{className:"health-row",children:[f.jsx("span",{style:{color:"var(--warning)"},children:"Expiring Soon"}),f.jsx("span",{className:"mono",children:M.expiring_soon})]}),f.jsxs("div",{className:"health-row",children:[f.jsx("span",{style:{color:"var(--error)"},children:"Error Rate"}),f.jsx("span",{className:"mono",children:M.error_rate})]})]})]})]})]})}const jr=["admin","agent","read","sources_admin","users_admin","write"];function yy(d){const N=Math.floor((Date.now()-d.getTime())/1e3);return N<60?"just now":N<3600?`${Math.floor(N/60)}m ago`:N<86400?`${Math.floor(N/3600)}h ago`:`${Math.floor(N/86400)}d ago`}function vy(){const[d,N]=k.useState([]),[M,h]=k.useState(!0),[_,Y]=k.useState(!1),[U,Z]=k.useState(null),[O,p]=k.useState(!1),[H,j]=k.useState(null),[E,I]=k.useState(null);k.useEffect(()=>{L()},[]);const L=()=>{st.agents().then(N).catch(()=>{})};return f.jsxs(f.Fragment,{children:[f.jsxs("div",{style:{display:"flex",justifyContent:"space-between",alignItems:"center",marginBottom:24},children:[f.jsx("h1",{className:"page-title",style:{marginBottom:0},children:"Agents"}),f.jsxs("div",{style:{display:"flex",gap:8,alignItems:"center"},children:[f.jsxs("label",{style:{fontSize:13,color:"var(--text-secondary)",display:"flex",alignItems:"center",gap:6,cursor:"pointer"},children:[f.jsx("input",{type:"checkbox",checked:M,onChange:nl=>h(nl.target.checked)})," Hide revoked"]}),f.jsx("button",{className:"btn btn-secondary",onClick:()=>p(!0),children:"+ API Key"}),f.jsx("button",{className:"btn btn-primary",onClick:()=>Y(!0),children:"+ OAuth Client"})]})]}),(()=>{const nl=d.filter(tl=>!M||tl.status!=="revoked");return d.length===0?f.jsx("div",{style:{textAlign:"center",padding:48,color:"var(--text-muted)"},children:"No agents registered. Register your first agent to get started."}):nl.length===0?f.jsx("div",{style:{textAlign:"center",padding:48,color:"var(--text-muted)"},children:'All agents are revoked. Uncheck "Hide revoked" to view them.'}):f.jsxs(f.Fragment,{children:[f.jsxs("table",{children:[f.jsx("thead",{children:f.jsxs("tr",{children:[f.jsx("th",{children:"Name"}),f.jsx("th",{children:"Type"}),f.jsx("th",{children:"Scopes"}),f.jsx("th",{children:"Status"}),f.jsx("th",{children:"Requests"}),f.jsx("th",{children:"Last Used"})]})}),f.jsx("tbody",{children:nl.map(tl=>f.jsxs("tr",{onClick:()=>I(tl),style:{cursor:"pointer"},children:[f.jsx("td",{style:{fontWeight:500},children:tl.name||tl.client_name}),f.jsx("td",{children:f.jsx("span",{className:`badge ${tl.auth_type==="oauth"?"badge-read":"badge-write"}`,style:{fontSize:11},children:tl.auth_type==="oauth"?"OAuth":"API Key"})}),f.jsx("td",{children:(tl.scope||"").split(" ").filter(Boolean).map(pl=>f.jsx("span",{className:`badge badge-${pl}`,style:{marginRight:4},children:pl},pl))}),f.jsx("td",{children:f.jsx("span",{className:`badge ${tl.status==="active"?"badge-success":"badge-danger"}`,children:tl.status})}),f.jsxs("td",{children:[f.jsx("span",{style:{fontWeight:500},children:tl.requests_today||0}),f.jsxs("span",{style:{color:"var(--text-muted)",fontSize:12},children:[" / ",tl.total_requests||0]})]}),f.jsx("td",{style:{color:"var(--text-secondary)"},children:tl.last_used_at?yy(new Date(tl.last_used_at)):"Never"})]},tl.id))})]}),f.jsxs("div",{style:{color:"var(--text-muted)",fontSize:13,marginTop:12},children:[d.filter(tl=>tl.status==="active").length," active / ",d.length," total"]})]})})(),_&&f.jsx(by,{onClose:()=>Y(!1),onRegistered:nl=>{Y(!1),Z(nl),L()}}),U&&f.jsx(py,{credentials:U,onClose:()=>Z(null)}),E&&f.jsx(Ty,{agent:E,onClose:()=>I(null),onRevoked:L}),O&&f.jsx(gy,{onClose:()=>p(!1),onCreated:nl=>{p(!1),j(nl),L()}}),H&&f.jsx(Sy,{token:H,onClose:()=>j(null)})]})}function gy({onClose:d,onCreated:N}){const[M,h]=k.useState(""),[_,Y]=k.useState(!1),[U,Z]=k.useState(""),O=async p=>{if(p.preventDefault(),!M.trim()){Z("Name required");return}Y(!0);try{const H=await st.createApiKey(M.trim());N({name:H.name,token:H.token})}catch(H){Z(H instanceof Error?H.message:"Failed")}finally{Y(!1)}};return f.jsx("div",{className:"modal-overlay",onClick:d,children:f.jsxs("form",{className:"modal",onClick:p=>p.stopPropagation(),onSubmit:O,children:[f.jsx("div",{className:"modal-title",children:"Create API Key"}),f.jsx("p",{style:{color:"var(--text-secondary)",fontSize:13,marginBottom:16},children:"API keys use simple bearer token auth. They grant full read+write+admin access. For scoped access, use OAuth clients instead."}),f.jsxs("div",{style:{marginBottom:16},children:[f.jsx("label",{children:"Key Name"}),f.jsx("input",{placeholder:"e.g. claude-code-local",value:M,onChange:p=>h(p.target.value),autoFocus:!0})]}),U&&f.jsx("div",{style:{color:"var(--error)",fontSize:13,marginBottom:12},children:U}),f.jsxs("div",{style:{display:"flex",gap:12,justifyContent:"flex-end"},children:[f.jsx("button",{type:"button",className:"btn btn-secondary",onClick:d,children:"Cancel"}),f.jsx("button",{type:"submit",className:"btn btn-primary",disabled:_,children:_?"Creating...":"Create Key"})]})]})})}function Sy({token:d,onClose:N}){const M=h=>navigator.clipboard.writeText(h);return f.jsx("div",{className:"modal-overlay",children:f.jsxs("div",{className:"modal",style:{maxWidth:560},children:[f.jsxs("div",{style:{textAlign:"center",marginBottom:16},children:[f.jsx("div",{style:{fontSize:36,color:"var(--success)",marginBottom:8},children:"✓"}),f.jsx("div",{style:{fontSize:20,fontWeight:600},children:"API Key Created"})]}),f.jsxs("div",{style:{marginBottom:12},children:[f.jsx("label",{style:{fontSize:12},children:"Name"}),f.jsx("div",{className:"code-block",children:f.jsx("span",{children:d.name})})]}),f.jsxs("div",{style:{marginBottom:12},children:[f.jsx("label",{style:{fontSize:12},children:"Bearer Token"}),f.jsxs("div",{className:"code-block",children:[f.jsx("span",{children:d.token}),f.jsx("button",{className:"copy-btn",onClick:()=>M(d.token),children:"Copy"})]})]}),f.jsxs("div",{style:{marginBottom:12},children:[f.jsx("label",{style:{fontSize:12},children:"Usage"}),f.jsxs("div",{className:"code-block",children:[f.jsx("pre",{style:{whiteSpace:"pre-wrap",margin:0,fontSize:12},children:`Authorization: Bearer ${d.token}`}),f.jsx("button",{className:"copy-btn",onClick:()=>M(`Authorization: Bearer ${d.token}`),children:"Copy"})]})]}),f.jsx("div",{className:"warning-bar",children:"Save this token now. It will not be shown again."}),f.jsx("div",{style:{display:"flex",gap:12,justifyContent:"flex-end",marginTop:20},children:f.jsx("button",{className:"btn btn-primary",onClick:N,children:"Done"})})]})})}function by({onClose:d,onRegistered:N}){const[M,h]=k.useState(""),[_,Y]=k.useState(()=>Object.fromEntries(jr.map(L=>[L,L==="read"]))),[U,Z]=k.useState("86400"),[O,p]=k.useState(!1),[H,j]=k.useState(""),E=[{label:"1 hour",value:"3600"},{label:"24 hours",value:"86400"},{label:"7 days",value:"604800"},{label:"30 days",value:"2592000"},{label:"1 year",value:"31536000"},{label:"No expiry",value:"0"}],I=async L=>{if(L.preventDefault(),!M.trim()){j("Name required");return}p(!0),j("");try{const nl=Object.entries(_).filter(([,Ml])=>Ml).map(([Ml])=>Ml).join(" "),tl=await fetch("/admin/api/register-client",{method:"POST",credentials:"same-origin",headers:{"Content-Type":"application/json"},body:JSON.stringify({name:M.trim(),scopes:nl,tokenTtl:U==="0"?31536e4:Number(U)})});if(!tl.ok)throw new Error("Registration failed");const pl=await tl.json();N({clientId:pl.clientId,clientSecret:pl.clientSecret,name:M.trim()})}catch(nl){j(nl instanceof Error?nl.message:"Registration failed")}finally{p(!1)}};return f.jsx("div",{className:"modal-overlay",onClick:d,children:f.jsxs("form",{className:"modal",onClick:L=>L.stopPropagation(),onSubmit:I,children:[f.jsx("div",{className:"modal-title",children:"Register Agent"}),f.jsxs("div",{style:{marginBottom:16},children:[f.jsx("label",{children:"Agent Name"}),f.jsx("input",{placeholder:"e.g. perplexity-production",value:M,onChange:L=>h(L.target.value),autoFocus:!0})]}),f.jsxs("div",{style:{marginBottom:16},children:[f.jsx("label",{children:"Scopes"}),f.jsx("div",{className:"checkbox-group",children:jr.map(L=>f.jsxs("label",{className:"checkbox-label",children:[f.jsx("input",{type:"checkbox",checked:_[L],onChange:nl=>Y(tl=>({...tl,[L]:nl.target.checked}))}),L]},L))})]}),f.jsxs("div",{style:{marginBottom:20},children:[f.jsx("label",{children:"Token Lifetime"}),f.jsx("select",{value:U,onChange:L=>Z(L.target.value),style:{width:"100%",background:"var(--bg-secondary)",color:"var(--text-primary)",border:"1px solid var(--border)",borderRadius:6,padding:"6px 10px",fontSize:14},children:E.map(L=>f.jsx("option",{value:L.value,children:L.label},L.value))})]}),H&&f.jsx("div",{style:{color:"var(--error)",fontSize:13,marginBottom:12},children:H}),f.jsxs("div",{style:{display:"flex",gap:12,justifyContent:"flex-end"},children:[f.jsx("button",{type:"button",className:"btn btn-secondary",onClick:d,children:"Cancel"}),f.jsx("button",{type:"submit",className:"btn btn-primary",disabled:O,children:O?"Registering...":"Register"})]})]})})}function py({credentials:d,onClose:N}){const M=_=>navigator.clipboard.writeText(_),h=()=>{const _=new Blob([JSON.stringify(d,null,2)],{type:"application/json"}),Y=URL.createObjectURL(_),U=document.createElement("a");U.href=Y,U.download=`${d.name}-credentials.json`,U.click(),URL.revokeObjectURL(Y)};return f.jsx("div",{className:"modal-overlay",children:f.jsxs("div",{className:"modal",style:{maxWidth:560},children:[f.jsxs("div",{style:{textAlign:"center",marginBottom:16},children:[f.jsx("div",{style:{fontSize:36,color:"var(--success)",marginBottom:8},children:"✓"}),f.jsx("div",{style:{fontSize:20,fontWeight:600},children:"Agent Registered"})]}),f.jsxs("div",{style:{marginBottom:12},children:[f.jsx("label",{style:{fontSize:12},children:"Client ID"}),f.jsxs("div",{className:"code-block",children:[f.jsx("span",{children:d.clientId}),f.jsx("button",{className:"copy-btn",onClick:()=>M(d.clientId),children:"Copy"})]})]}),f.jsxs("div",{style:{marginBottom:12},children:[f.jsx("label",{style:{fontSize:12},children:"Client Secret"}),f.jsxs("div",{className:"code-block",children:[f.jsx("span",{children:d.clientSecret}),f.jsx("button",{className:"copy-btn",onClick:()=>M(d.clientSecret),children:"Copy"})]})]}),f.jsx("div",{className:"warning-bar",children:"Save this secret now. It will not be shown again."}),f.jsxs("div",{style:{display:"flex",gap:12,justifyContent:"flex-end",marginTop:20},children:[f.jsx("button",{className:"btn btn-secondary",onClick:h,children:"Download as JSON"}),f.jsx("button",{className:"btn btn-primary",onClick:N,children:"Done"})]})]})})}function Ty({agent:d,onClose:N,onRevoked:M}){const[h,_]=k.useState("claude-code"),Y=j=>navigator.clipboard.writeText(j),U=window.location.origin,Z=d.id||d.client_id||"",O=d.auth_type==="oauth",p=d.name||d.client_name||"unknown",H={"claude-code":O?["The user wants to connect GBrain MCP to your context. Here's how.","","# Step 1 — read the client secret from the operator without putting it in shell history.","# read -s suppresses the echo; export keeps it scoped to this shell session.",'read -rs -p "Client Secret: " GBRAIN_CS && echo',"","# Step 2 — mint a token and register GBrain in one go.",`claude mcp add --transport http gbrain ${U}/mcp \\`,` --header "Authorization: Bearer $(curl -s -X POST ${U}/token \\`," -d 'grant_type=client_credentials' \\",` -d 'client_id=${Z}' \\`,' --data-urlencode "client_secret=$GBRAIN_CS" \\',` -d 'scope=${d.scope||"read write"}' | jq -r .access_token)"`,"","# Step 3 — clear the secret from this shell.","unset GBRAIN_CS","",`The token will last ${d.token_ttl?d.token_ttl>=86400?Math.floor(d.token_ttl/86400)+" days":Math.floor(d.token_ttl/3600)+" hours":"1 hour (default)"}.`,"","─── Fallback: 2-step curl + paste ───","","If your shell doesn't support read -s, mint the token first, then paste:","",`curl -s -X POST ${U}/token \\`," -d 'grant_type=client_credentials' \\",` -d 'client_id=${Z}' \\`," -d 'client_secret=PASTE_YOUR_CLIENT_SECRET_HERE' \\",` -d 'scope=${d.scope||"read write"}' | jq -r .access_token`,"",`claude mcp add --transport http gbrain ${U}/mcp \\`,' --header "Authorization: Bearer PASTE_TOKEN_FROM_ABOVE"',"","# Then run: history -d $((HISTCMD-1)) # zsh: print -s -- '' && fc -p"].join(` -`):["The user wants to connect GBrain MCP to your context. Here's how.","","Run this command in terminal to register GBrain:","",`claude mcp add --transport http gbrain ${U}/mcp \\`,' --header "Authorization: Bearer PASTE_YOUR_API_KEY_HERE"',"",`Replace PASTE_YOUR_API_KEY_HERE with the API key shown when "${p}" was created.`,"API keys never expire."].join(` -`),chatgpt:["The user wants to connect GBrain MCP to ChatGPT.","","1. Open ChatGPT > Settings > Tools & Integrations > Add MCP Server","2. Enter this URL — ChatGPT will auto-discover the OAuth configuration:","",` ${U}/.well-known/oauth-authorization-server`,"","3. When prompted for credentials:",` Client ID: ${Z}`," Client Secret: (the secret from agent registration)"," Grant Type: client_credentials",` Scope: ${d.scope||"read write"}`].join(` -`),"claude-cowork":["The user wants to connect GBrain MCP to Claude.ai.","","1. Open claude.ai > Settings > Connected Apps > Add MCP Server",`2. Server URL: ${U}/mcp`,"3. When prompted for auth:",` Token endpoint: ${U}/token`,` Client ID: ${Z}`," Client Secret: (the secret from agent registration)",` Scope: ${d.scope||"read write"}`,"",`Discovery URL: ${U}/.well-known/oauth-authorization-server`].join(` -`),cursor:O?["The user wants to connect GBrain MCP to Cursor.","","Cursor supports OAuth for remote MCP. Add to .cursor/mcp.json:","","{",' "mcpServers": {',' "gbrain": {',` "url": "${U}/mcp",`,' "transport": "sse"'," }"," }","}","","Cursor will auto-discover OAuth via:",`${U}/.well-known/oauth-authorization-server`,"",`When prompted: Client ID ${Z}, use the secret from registration.`].join(` -`):["The user wants to connect GBrain MCP to Cursor.","","Add to .cursor/mcp.json:","","{",' "mcpServers": {',' "gbrain": {',` "url": "${U}/mcp",`,' "transport": "sse",',' "headers": {',' "Authorization": "Bearer PASTE_YOUR_API_KEY_HERE"'," }"," }"," }","}","",`Replace PASTE_YOUR_API_KEY_HERE with the API key shown when "${p}" was created.`].join(` -`),perplexity:["The user wants to connect GBrain MCP to Perplexity.","","1. Go to Settings > Connectors > Add MCP",`2. Server URL: ${U}/mcp`,`3. Client ID: ${Z}`,"4. Client Secret: (the secret from agent registration)"].join(` -`),json:JSON.stringify({server_url:U+"/mcp",token_url:U+"/token",discovery_url:U+"/.well-known/oauth-authorization-server",client_id:Z,client_name:p,auth_type:d.auth_type,scope:d.scope},null,2)};return f.jsxs(f.Fragment,{children:[f.jsx("div",{className:"drawer-overlay",onClick:N}),f.jsxs("div",{className:"drawer",children:[f.jsx("button",{className:"drawer-close",onClick:N,children:"✕"}),f.jsx("div",{style:{fontSize:18,fontWeight:600,marginBottom:4},children:d.name||d.client_name}),f.jsx("span",{className:`badge ${d.status==="active"?"badge-success":"badge-danger"}`,children:d.status}),f.jsx("div",{className:"section-title",children:"Details"}),f.jsxs("div",{style:{display:"grid",gridTemplateColumns:"100px 1fr",gap:"6px 12px",fontSize:13},children:[f.jsx("span",{style:{color:"var(--text-secondary)"},children:"Client ID"}),f.jsxs("span",{className:"mono",children:[(d.id||d.id||d.client_id||"").substring(0,24),"..."]}),f.jsx("span",{style:{color:"var(--text-secondary)"},children:"Scopes"}),f.jsx("span",{children:(d.scope||"").split(" ").filter(Boolean).map(j=>f.jsx("span",{className:`badge badge-${j}`,style:{marginRight:4},children:j},j))}),f.jsx("span",{style:{color:"var(--text-secondary)"},children:"Registered"}),f.jsx("span",{children:new Date(d.created_at).toLocaleDateString()}),f.jsx("span",{style:{color:"var(--text-secondary)"},children:"Token TTL"}),f.jsx("span",{children:d.token_ttl?d.token_ttl>=31536e3?"No expiry":d.token_ttl>=86400?`${Math.floor(d.token_ttl/86400)}d`:d.token_ttl>=3600?`${Math.floor(d.token_ttl/3600)}h`:`${d.token_ttl}s`:"1h (default)"})]}),f.jsx("div",{className:"section-title",children:"Config Export"}),f.jsxs("div",{className:"tabs",style:{flexWrap:"wrap"},children:[f.jsx("div",{className:`tab ${h==="claude-code"?"active":""}`,onClick:()=>_("claude-code"),children:"Claude Code"}),f.jsx("div",{className:`tab ${h==="chatgpt"?"active":""}`,onClick:()=>_("chatgpt"),children:"ChatGPT"}),f.jsx("div",{className:`tab ${h==="claude-cowork"?"active":""}`,onClick:()=>_("claude-cowork"),children:"Claude.ai"}),f.jsx("div",{className:`tab ${h==="cursor"?"active":""}`,onClick:()=>_("cursor"),children:"Cursor"}),f.jsx("div",{className:`tab ${h==="perplexity"?"active":""}`,onClick:()=>_("perplexity"),children:"Perplexity"}),f.jsx("div",{className:`tab ${h==="json"?"active":""}`,onClick:()=>_("json"),children:"JSON"})]}),(()=>{if(!O&&new Set(["chatgpt","claude-cowork","perplexity"]).has(h)){const E={chatgpt:"ChatGPT","claude-cowork":"Claude.ai",perplexity:"Perplexity"}[h]||h;return f.jsxs("div",{style:{background:"rgba(255, 200, 100, 0.08)",border:"1px solid rgba(255, 200, 100, 0.2)",borderRadius:8,padding:"14px 16px",marginTop:12,fontSize:13,lineHeight:1.6,color:"var(--text-secondary)"},children:[f.jsxs("div",{style:{fontWeight:600,color:"var(--text-primary)",marginBottom:6},children:[E," requires an OAuth client"]}),E," only supports OAuth 2.0 (client_credentials). API keys use raw bearer tokens, which ",E," does not accept. Register a separate OAuth client and use that to connect this AI."]})}return f.jsxs("div",{className:"code-block",children:[f.jsx("pre",{style:{whiteSpace:"pre-wrap",margin:0},children:H[h]}),f.jsx("button",{className:"copy-btn",onClick:()=>Y(H[h]),children:"Copy"})]})})(),f.jsxs("div",{style:{marginTop:32},children:[d.status==="active"&&f.jsx("button",{className:"btn btn-danger",onClick:async()=>{if(confirm(`Revoke ${d.name||d.client_name}? All active tokens will be invalidated.`))try{d.auth_type==="oauth"?await st.revokeClient(d.id||d.client_id||""):await st.revokeApiKey(d.name||""),M(),N()}catch(j){alert("Revoke failed: "+(j instanceof Error?j.message:"unknown error"))}},children:"Revoke Agent"}),d.status==="revoked"&&f.jsx("span",{style:{color:"var(--text-muted)",fontSize:13},children:"This agent has been revoked."})]})]})]})}function zy(){const[d,N]=k.useState({rows:[],total:0,page:1,pages:1}),[M,h]=k.useState(1),[_,Y]=k.useState("all"),[U,Z]=k.useState(null);k.useEffect(()=>{O(M)},[M,_]);const O=E=>{const I=_!=="all"?`&agent=${encodeURIComponent(_)}`:"";st.requests(E,I).then(N).catch(()=>{})},p=E=>{const I=Date.now()-new Date(E).getTime();return I<6e4?`${Math.floor(I/1e3)}s ago`:I<36e5?`${Math.floor(I/6e4)} min ago`:I<864e5?`${Math.floor(I/36e5)}h ago`:new Date(E).toLocaleDateString()},H=E=>{if(!E)return null;const{query:I,slug:L,partial:nl,limit:tl,...pl}=E,Ml=[];return I&&Ml.push(`"${I}"`),L&&Ml.push(L),nl&&Ml.push(`~${nl}`),tl&&Ml.push(`limit=${tl}`),Object.keys(pl).length>0&&Ml.push(`+${Object.keys(pl).length} params`),Ml.join(" ")},j=new Map;return d.rows.forEach(E=>{E.token_name&&j.set(E.token_name,E.agent_name||E.token_name)}),f.jsxs(f.Fragment,{children:[f.jsxs("div",{style:{display:"flex",justifyContent:"space-between",alignItems:"center",marginBottom:24},children:[f.jsx("h1",{className:"page-title",style:{marginBottom:0},children:"Request Log"}),f.jsxs("select",{value:_,onChange:E=>{Y(E.target.value),h(1)},style:{background:"var(--bg-secondary)",color:"var(--text-primary)",border:"1px solid var(--border)",borderRadius:6,padding:"4px 8px",fontSize:13},children:[f.jsx("option",{value:"all",children:"All agents"}),[...j.entries()].map(([E,I])=>f.jsx("option",{value:E,children:I},E))]})]}),d.rows.length===0?f.jsx("div",{style:{textAlign:"center",padding:48,color:"var(--text-muted)"},children:"No requests yet."}):f.jsxs(f.Fragment,{children:[f.jsxs("table",{children:[f.jsx("thead",{children:f.jsxs("tr",{children:[f.jsx("th",{children:"Time"}),f.jsx("th",{children:"Agent"}),f.jsx("th",{children:"Operation"}),f.jsx("th",{children:"Params"}),f.jsx("th",{children:"Latency"}),f.jsx("th",{children:"Status"})]})}),f.jsx("tbody",{children:d.rows.map(E=>f.jsxs(Mr.Fragment,{children:[f.jsxs("tr",{onClick:()=>Z(U===E.id?null:E.id),style:{cursor:"pointer"},children:[f.jsx("td",{style:{color:"var(--text-secondary)",whiteSpace:"nowrap"},children:p(E.created_at)}),f.jsx("td",{children:f.jsx("a",{style:{color:"var(--text-link, #88aaff)",cursor:"pointer",textDecoration:"none",fontWeight:500},onClick:I=>{I.stopPropagation(),Y(E.token_name),h(1)},children:E.agent_name||E.token_name})}),f.jsx("td",{className:"mono",children:E.operation}),f.jsx("td",{style:{color:"var(--text-secondary)",fontSize:12,maxWidth:200,overflow:"hidden",textOverflow:"ellipsis",whiteSpace:"nowrap"},children:H(E.params)}),f.jsxs("td",{className:"mono",children:[E.latency_ms,"ms"]}),f.jsx("td",{children:f.jsx("span",{className:`badge badge-${E.status}`,children:E.status})})]}),U===E.id&&f.jsx("tr",{children:f.jsx("td",{colSpan:6,style:{background:"var(--bg-secondary, #0f0f1a)",padding:16},children:f.jsxs("div",{style:{display:"grid",gridTemplateColumns:"100px 1fr",gap:"6px 12px",fontSize:13},children:[f.jsx("span",{style:{color:"var(--text-muted)"},children:"Time"}),f.jsx("span",{children:new Date(E.created_at).toLocaleString()}),f.jsx("span",{style:{color:"var(--text-muted)"},children:"Agent"}),f.jsx("span",{className:"mono",children:E.token_name}),f.jsx("span",{style:{color:"var(--text-muted)"},children:"Operation"}),f.jsx("span",{className:"mono",children:E.operation}),f.jsx("span",{style:{color:"var(--text-muted)"},children:"Latency"}),f.jsxs("span",{children:[E.latency_ms,"ms"]}),E.params&&f.jsxs(f.Fragment,{children:[f.jsx("span",{style:{color:"var(--text-muted)"},children:"Params"}),f.jsx("pre",{className:"mono",style:{margin:0,whiteSpace:"pre-wrap",fontSize:12},children:JSON.stringify(E.params,null,2)})]}),E.error_message&&f.jsxs(f.Fragment,{children:[f.jsx("span",{style:{color:"var(--error, #ff6b6b)"},children:"Error"}),f.jsx("span",{style:{color:"var(--error, #ff6b6b)"},children:E.error_message})]})]})})})]},E.id))})]}),f.jsxs("div",{className:"pagination",children:[f.jsxs("span",{children:["Page ",d.page," of ",d.pages," (",d.total," total)"]}),f.jsxs("div",{style:{display:"flex",gap:8},children:[f.jsx("button",{disabled:d.page<=1,onClick:()=>h(E=>E-1),children:"Previous"}),f.jsx("button",{disabled:d.page>=d.pages,onClick:()=>h(E=>E+1),children:"Next"})]})]})]})]})}function xy({markup:d}){return f.jsx("div",{style:{width:"100%",overflow:"auto"},dangerouslySetInnerHTML:{__html:d}})}function Zu({type:d,ariaLabel:N}){const[M,h]=k.useState(""),[_,Y]=k.useState("");return k.useEffect(()=>{let U=!1;return st.calibrationChart(d).then(Z=>{U||h(Z)}).catch(Z=>{U||Y(Z.message??"fetch failed")}),()=>{U=!0}},[d]),_?f.jsxs("div",{style:{padding:16,color:"var(--error)"},role:"alert",children:[N,": ",_]}):M?f.jsx(xy,{markup:M}):f.jsxs("div",{style:{padding:16,color:"var(--text-muted)"},children:[N," loading..."]})}function Ay(){const[d,N]=k.useState(null),[M,h]=k.useState(!0),[_,Y]=k.useState("");if(k.useEffect(()=>{st.calibrationProfile().then(O=>{N(O),h(!1)}).catch(O=>{Y(O.message??"fetch failed"),h(!1)})},[]),M)return f.jsx("div",{style:{padding:24,color:"var(--text-secondary)"},children:"Loading calibration profile…"});if(_)return f.jsxs("div",{style:{padding:24,color:"var(--error)"},role:"alert",children:["Could not load calibration profile: ",_]});if(!d)return f.jsxs("div",{style:{padding:24,maxWidth:700},children:[f.jsx("h1",{style:{marginBottom:16},children:"Calibration"}),f.jsx("p",{style:{color:"var(--text-secondary)"},children:"No calibration profile yet. Builds after 5+ resolved takes."}),f.jsx("pre",{style:{background:"var(--bg-secondary)",padding:12,borderRadius:4,color:"var(--text-primary)",marginTop:12,fontFamily:"var(--font-mono)"},children:"gbrain dream --phase calibration_profile"})]});const U=new Date(d.generated_at),Z=Math.floor((Date.now()-U.getTime())/(1e3*60*60*24));return f.jsxs("div",{style:{padding:32,maxWidth:720},children:[f.jsx("h1",{style:{marginBottom:8},children:"Calibration"}),f.jsxs("div",{style:{color:"var(--text-muted)",fontSize:13,marginBottom:24},children:["Holder: ",d.holder," · ","Updated ",Z===0?"today":`${Z}d ago`,d.published&&" · published",d.grade_completion<.9&&` · ~${Math.round(d.grade_completion*100)}% graded`,!d.voice_gate_passed&&" · voice gate fell back to template"]}),f.jsx("section",{style:{marginBottom:32},children:f.jsx(Zu,{type:"brier-trend",ariaLabel:"Brier trend"})}),f.jsxs("section",{style:{marginBottom:32},children:[f.jsx("h2",{style:{fontSize:14,color:"var(--text-secondary)",marginBottom:12,fontWeight:400},children:"Pattern statements"}),f.jsx(Zu,{type:"pattern-statements",ariaLabel:"Pattern statements"})]}),f.jsx("section",{style:{marginBottom:32},children:f.jsx(Zu,{type:"domain-bars",ariaLabel:"Per-domain accuracy"})}),f.jsx("section",{style:{marginBottom:32},children:f.jsx(Zu,{type:"abandoned-threads",ariaLabel:"Abandoned threads"})}),d.active_bias_tags.length>0&&f.jsxs("section",{style:{marginBottom:32,color:"var(--text-muted)",fontSize:13},children:["Active bias tags: ",d.active_bias_tags.join(", ")]})]})}function Or(){const d=window.location.hash.replace("#","")||"dashboard";return["login","dashboard","agents","log","calibration"].includes(d)?d:"dashboard"}function Ey(){const[d,N]=k.useState(Or);k.useEffect(()=>{const _=()=>N(Or());return window.addEventListener("hashchange",_),()=>window.removeEventListener("hashchange",_)},[]);const M=_=>{window.location.hash=_,N(_)};if(d==="login")return f.jsx(hy,{onLogin:()=>M("dashboard")});const h=async()=>{if(confirm("Sign out every active admin session, including other browsers and tabs? Each one will need to re-authenticate via a fresh magic link.")){try{await st.signOutEverywhere()}catch{}M("login")}};return f.jsxs("div",{className:"app",children:[f.jsxs("nav",{className:"sidebar",children:[f.jsx("div",{className:"sidebar-logo",children:"GBrain"}),f.jsxs("div",{className:"sidebar-nav",children:[f.jsx("a",{className:`nav-item ${d==="dashboard"?"active":""}`,onClick:()=>M("dashboard"),children:"Dashboard"}),f.jsx("a",{className:`nav-item ${d==="agents"?"active":""}`,onClick:()=>M("agents"),children:"Agents"}),f.jsx("a",{className:`nav-item ${d==="log"?"active":""}`,onClick:()=>M("log"),children:"Request Log"}),f.jsx("a",{className:`nav-item ${d==="calibration"?"active":""}`,onClick:()=>M("calibration"),children:"Calibration"})]}),f.jsx("div",{style:{marginTop:"auto",padding:"16px 12px",borderTop:"1px solid var(--border)"},children:f.jsx("button",{onClick:h,style:{background:"transparent",border:"1px solid var(--border)",color:"var(--text-secondary)",padding:"6px 10px",borderRadius:6,fontSize:12,cursor:"pointer",width:"100%"},title:"Revoke every active admin session — every browser, every tab",children:"Sign out everywhere"})})]}),f.jsxs("main",{className:"main",children:[d==="dashboard"&&f.jsx(my,{}),d==="agents"&&f.jsx(vy,{}),d==="log"&&f.jsx(zy,{}),d==="calibration"&&f.jsx(Ay,{})]})]})}dy.createRoot(document.getElementById("root")).render(f.jsx(Mr.StrictMode,{children:f.jsx(Ey,{})})); diff --git a/admin/dist/assets/index-DqP-zmqH.js b/admin/dist/assets/index-DqP-zmqH.js new file mode 100644 index 000000000..3fd01fc23 --- /dev/null +++ b/admin/dist/assets/index-DqP-zmqH.js @@ -0,0 +1,56 @@ +(function(){const D=document.createElement("link").relList;if(D&&D.supports&&D.supports("modulepreload"))return;for(const E of document.querySelectorAll('link[rel="modulepreload"]'))h(E);new MutationObserver(E=>{for(const N of E)if(N.type==="childList")for(const C of N.addedNodes)C.tagName==="LINK"&&C.rel==="modulepreload"&&h(C)}).observe(document,{childList:!0,subtree:!0});function O(E){const N={};return E.integrity&&(N.integrity=E.integrity),E.referrerPolicy&&(N.referrerPolicy=E.referrerPolicy),E.crossOrigin==="use-credentials"?N.credentials="include":E.crossOrigin==="anonymous"?N.credentials="omit":N.credentials="same-origin",N}function h(E){if(E.ep)return;E.ep=!0;const N=O(E);fetch(E.href,N)}})();function Md(o){return o&&o.__esModule&&Object.prototype.hasOwnProperty.call(o,"default")?o.default:o}var ff={exports:{}},jn={};/** + * @license React + * react-jsx-runtime.production.js + * + * Copyright (c) Meta Platforms, Inc. and affiliates. + * + * This source code is licensed under the MIT license found in the + * LICENSE file in the root directory of this source tree. + */var gd;function ey(){if(gd)return jn;gd=1;var o=Symbol.for("react.transitional.element"),D=Symbol.for("react.fragment");function O(h,E,N){var C=null;if(N!==void 0&&(C=""+N),E.key!==void 0&&(C=""+E.key),"key"in E){N={};for(var Q in E)Q!=="key"&&(N[Q]=E[Q])}else N=E;return E=N.ref,{$$typeof:o,type:h,key:C,ref:E!==void 0?E:null,props:N}}return jn.Fragment=D,jn.jsx=O,jn.jsxs=O,jn}var Sd;function ay(){return Sd||(Sd=1,ff.exports=ey()),ff.exports}var c=ay(),sf={exports:{}},V={};/** + * @license React + * react.production.js + * + * Copyright (c) Meta Platforms, Inc. and affiliates. + * + * This source code is licensed under the MIT license found in the + * LICENSE file in the root directory of this source tree. + */var pd;function ny(){if(pd)return V;pd=1;var o=Symbol.for("react.transitional.element"),D=Symbol.for("react.portal"),O=Symbol.for("react.fragment"),h=Symbol.for("react.strict_mode"),E=Symbol.for("react.profiler"),N=Symbol.for("react.consumer"),C=Symbol.for("react.context"),Q=Symbol.for("react.forward_ref"),_=Symbol.for("react.suspense"),b=Symbol.for("react.memo"),H=Symbol.for("react.lazy"),M=Symbol.for("react.activity"),A=Symbol.iterator;function I(d){return d===null||typeof d!="object"?null:(d=A&&d[A]||d["@@iterator"],typeof d=="function"?d:null)}var L={isMounted:function(){return!1},enqueueForceUpdate:function(){},enqueueReplaceState:function(){},enqueueSetState:function(){}},nl=Object.assign,tl={};function bl(d,z,R){this.props=d,this.context=z,this.refs=tl,this.updater=R||L}bl.prototype.isReactComponent={},bl.prototype.setState=function(d,z){if(typeof d!="object"&&typeof d!="function"&&d!=null)throw Error("takes an object of state variables to update or a function which returns an object of state variables.");this.updater.enqueueSetState(this,d,z,"setState")},bl.prototype.forceUpdate=function(d){this.updater.enqueueForceUpdate(this,d,"forceUpdate")};function Ml(){}Ml.prototype=bl.prototype;function Gl(d,z,R){this.props=d,this.context=z,this.refs=tl,this.updater=R||L}var rt=Gl.prototype=new Ml;rt.constructor=Gl,nl(rt,bl.prototype),rt.isPureReactComponent=!0;var _t=Array.isArray;function Ll(){}var el={H:null,A:null,T:null,S:null},Vl=Object.prototype.hasOwnProperty;function Et(d,z,R){var q=R.ref;return{$$typeof:o,type:d,key:z,ref:q!==void 0?q:null,props:R}}function Le(d,z){return Et(d.type,z,d.props)}function Ot(d){return typeof d=="object"&&d!==null&&d.$$typeof===o}function Kl(d){var z={"=":"=0",":":"=2"};return"$"+d.replace(/[=:]/g,function(R){return z[R]})}var Te=/\/+/g;function Ut(d,z){return typeof d=="object"&&d!==null&&d.key!=null?Kl(""+d.key):z.toString(36)}function jt(d){switch(d.status){case"fulfilled":return d.value;case"rejected":throw d.reason;default:switch(typeof d.status=="string"?d.then(Ll,Ll):(d.status="pending",d.then(function(z){d.status==="pending"&&(d.status="fulfilled",d.value=z)},function(z){d.status==="pending"&&(d.status="rejected",d.reason=z)})),d.status){case"fulfilled":return d.value;case"rejected":throw d.reason}}throw d}function x(d,z,R,q,J){var $=typeof d;($==="undefined"||$==="boolean")&&(d=null);var fl=!1;if(d===null)fl=!0;else switch($){case"bigint":case"string":case"number":fl=!0;break;case"object":switch(d.$$typeof){case o:case D:fl=!0;break;case H:return fl=d._init,x(fl(d._payload),z,R,q,J)}}if(fl)return J=J(d),fl=q===""?"."+Ut(d,0):q,_t(J)?(R="",fl!=null&&(R=fl.replace(Te,"$&/")+"/"),x(J,z,R,"",function(Oa){return Oa})):J!=null&&(Ot(J)&&(J=Le(J,R+(J.key==null||d&&d.key===J.key?"":(""+J.key).replace(Te,"$&/")+"/")+fl)),z.push(J)),1;fl=0;var Ql=q===""?".":q+":";if(_t(d))for(var Tl=0;Tl>>1,yl=x[rl];if(0>>1;rlE(R,Z))qE(J,R)?(x[rl]=J,x[q]=Z,rl=q):(x[rl]=R,x[z]=Z,rl=z);else if(qE(J,Z))x[rl]=J,x[q]=Z,rl=q;else break l}}return U}function E(x,U){var Z=x.sortIndex-U.sortIndex;return Z!==0?Z:x.id-U.id}if(o.unstable_now=void 0,typeof performance=="object"&&typeof performance.now=="function"){var N=performance;o.unstable_now=function(){return N.now()}}else{var C=Date,Q=C.now();o.unstable_now=function(){return C.now()-Q}}var _=[],b=[],H=1,M=null,A=3,I=!1,L=!1,nl=!1,tl=!1,bl=typeof setTimeout=="function"?setTimeout:null,Ml=typeof clearTimeout=="function"?clearTimeout:null,Gl=typeof setImmediate<"u"?setImmediate:null;function rt(x){for(var U=O(b);U!==null;){if(U.callback===null)h(b);else if(U.startTime<=x)h(b),U.sortIndex=U.expirationTime,D(_,U);else break;U=O(b)}}function _t(x){if(nl=!1,rt(x),!L)if(O(_)!==null)L=!0,Ll||(Ll=!0,Kl());else{var U=O(b);U!==null&&jt(_t,U.startTime-x)}}var Ll=!1,el=-1,Vl=5,Et=-1;function Le(){return tl?!0:!(o.unstable_now()-Etx&&Le());){var rl=M.callback;if(typeof rl=="function"){M.callback=null,A=M.priorityLevel;var yl=rl(M.expirationTime<=x);if(x=o.unstable_now(),typeof yl=="function"){M.callback=yl,rt(x),U=!0;break t}M===O(_)&&h(_),rt(x)}else h(_);M=O(_)}if(M!==null)U=!0;else{var d=O(b);d!==null&&jt(_t,d.startTime-x),U=!1}}break l}finally{M=null,A=Z,I=!1}U=void 0}}finally{U?Kl():Ll=!1}}}var Kl;if(typeof Gl=="function")Kl=function(){Gl(Ot)};else if(typeof MessageChannel<"u"){var Te=new MessageChannel,Ut=Te.port2;Te.port1.onmessage=Ot,Kl=function(){Ut.postMessage(null)}}else Kl=function(){bl(Ot,0)};function jt(x,U){el=bl(function(){x(o.unstable_now())},U)}o.unstable_IdlePriority=5,o.unstable_ImmediatePriority=1,o.unstable_LowPriority=4,o.unstable_NormalPriority=3,o.unstable_Profiling=null,o.unstable_UserBlockingPriority=2,o.unstable_cancelCallback=function(x){x.callback=null},o.unstable_forceFrameRate=function(x){0>x||125rl?(x.sortIndex=Z,D(b,x),O(_)===null&&x===O(b)&&(nl?(Ml(el),el=-1):nl=!0,jt(_t,Z-rl))):(x.sortIndex=yl,D(_,x),L||I||(L=!0,Ll||(Ll=!0,Kl()))),x},o.unstable_shouldYield=Le,o.unstable_wrapCallback=function(x){var U=A;return function(){var Z=A;A=U;try{return x.apply(this,arguments)}finally{A=Z}}}})(df)),df}var jd;function iy(){return jd||(jd=1,rf.exports=uy()),rf.exports}var hf={exports:{}},Xl={};/** + * @license React + * react-dom.production.js + * + * Copyright (c) Meta Platforms, Inc. and affiliates. + * + * This source code is licensed under the MIT license found in the + * LICENSE file in the root directory of this source tree. + */var Td;function cy(){if(Td)return Xl;Td=1;var o=mf();function D(_){var b="https://react.dev/errors/"+_;if(1"u"||typeof __REACT_DEVTOOLS_GLOBAL_HOOK__.checkDCE!="function"))try{__REACT_DEVTOOLS_GLOBAL_HOOK__.checkDCE(o)}catch(D){console.error(D)}}return o(),hf.exports=cy(),hf.exports}/** + * @license React + * react-dom-client.production.js + * + * Copyright (c) Meta Platforms, Inc. and affiliates. + * + * This source code is licensed under the MIT license found in the + * LICENSE file in the root directory of this source tree. + */var Ad;function sy(){if(Ad)return Tn;Ad=1;var o=iy(),D=mf(),O=fy();function h(l){var t="https://react.dev/errors/"+l;if(1yl||(l.current=rl[yl],rl[yl]=null,yl--)}function R(l,t){yl++,rl[yl]=l.current,l.current=t}var q=d(null),J=d(null),$=d(null),fl=d(null);function Ql(l,t){switch(R($,t),R(J,l),R(q,null),t.nodeType){case 9:case 11:l=(l=t.documentElement)&&(l=l.namespaceURI)?Xr(l):0;break;default:if(l=t.tagName,t=t.namespaceURI)t=Xr(t),l=Qr(t,l);else switch(l){case"svg":l=1;break;case"math":l=2;break;default:l=0}}z(q),R(q,l)}function Tl(){z(q),z(J),z($)}function Oa(l){l.memoizedState!==null&&R(fl,l);var t=q.current,e=Qr(t,l.type);t!==e&&(R(J,l),R(q,e))}function zn(l){J.current===l&&(z(q),z(J)),fl.current===l&&(z(fl),Sn._currentValue=Z)}var Lu,yf;function ze(l){if(Lu===void 0)try{throw Error()}catch(e){var t=e.stack.trim().match(/\n( *(at )?)/);Lu=t&&t[1]||"",yf=-1)":-1n||s[a]!==v[n]){var p=` +`+s[a].replace(" at new "," at ");return l.displayName&&p.includes("")&&(p=p.replace("",l.displayName)),p}while(1<=a&&0<=n);break}}}finally{Vu=!1,Error.prepareStackTrace=e}return(e=l?l.displayName||l.name:"")?ze(e):""}function Ud(l,t){switch(l.tag){case 26:case 27:case 5:return ze(l.type);case 16:return ze("Lazy");case 13:return l.child!==t&&t!==null?ze("Suspense Fallback"):ze("Suspense");case 19:return ze("SuspenseList");case 0:case 15:return Ku(l.type,!1);case 11:return Ku(l.type.render,!1);case 1:return Ku(l.type,!0);case 31:return ze("Activity");default:return""}}function vf(l){try{var t="",e=null;do t+=Ud(l,e),e=l,l=l.return;while(l);return t}catch(a){return` +Error generating stack: `+a.message+` +`+a.stack}}var Ju=Object.prototype.hasOwnProperty,wu=o.unstable_scheduleCallback,ku=o.unstable_cancelCallback,Rd=o.unstable_shouldYield,Bd=o.unstable_requestPaint,lt=o.unstable_now,Hd=o.unstable_getCurrentPriorityLevel,gf=o.unstable_ImmediatePriority,Sf=o.unstable_UserBlockingPriority,An=o.unstable_NormalPriority,qd=o.unstable_LowPriority,pf=o.unstable_IdlePriority,Yd=o.log,Gd=o.unstable_setDisableYieldValue,Na=null,tt=null;function It(l){if(typeof Yd=="function"&&Gd(l),tt&&typeof tt.setStrictMode=="function")try{tt.setStrictMode(Na,l)}catch{}}var et=Math.clz32?Math.clz32:Zd,Xd=Math.log,Qd=Math.LN2;function Zd(l){return l>>>=0,l===0?32:31-(Xd(l)/Qd|0)|0}var _n=256,En=262144,On=4194304;function Ae(l){var t=l&42;if(t!==0)return t;switch(l&-l){case 1:return 1;case 2:return 2;case 4:return 4;case 8:return 8;case 16:return 16;case 32:return 32;case 64:return 64;case 128:return 128;case 256:case 512:case 1024:case 2048:case 4096:case 8192:case 16384:case 32768:case 65536:case 131072:return l&261888;case 262144:case 524288:case 1048576:case 2097152:return l&3932160;case 4194304:case 8388608:case 16777216:case 33554432:return l&62914560;case 67108864:return 67108864;case 134217728:return 134217728;case 268435456:return 268435456;case 536870912:return 536870912;case 1073741824:return 0;default:return l}}function Nn(l,t,e){var a=l.pendingLanes;if(a===0)return 0;var n=0,u=l.suspendedLanes,i=l.pingedLanes;l=l.warmLanes;var f=a&134217727;return f!==0?(a=f&~u,a!==0?n=Ae(a):(i&=f,i!==0?n=Ae(i):e||(e=f&~l,e!==0&&(n=Ae(e))))):(f=a&~u,f!==0?n=Ae(f):i!==0?n=Ae(i):e||(e=a&~l,e!==0&&(n=Ae(e)))),n===0?0:t!==0&&t!==n&&(t&u)===0&&(u=n&-n,e=t&-t,u>=e||u===32&&(e&4194048)!==0)?t:n}function Ma(l,t){return(l.pendingLanes&~(l.suspendedLanes&~l.pingedLanes)&t)===0}function Ld(l,t){switch(l){case 1:case 2:case 4:case 8:case 64:return t+250;case 16:case 32:case 128:case 256:case 512:case 1024:case 2048:case 4096:case 8192:case 16384:case 32768:case 65536:case 131072:case 262144:case 524288:case 1048576:case 2097152:return t+5e3;case 4194304:case 8388608:case 16777216:case 33554432:return-1;case 67108864:case 134217728:case 268435456:case 536870912:case 1073741824:return-1;default:return-1}}function bf(){var l=On;return On<<=1,(On&62914560)===0&&(On=4194304),l}function $u(l){for(var t=[],e=0;31>e;e++)t.push(l);return t}function Da(l,t){l.pendingLanes|=t,t!==268435456&&(l.suspendedLanes=0,l.pingedLanes=0,l.warmLanes=0)}function Vd(l,t,e,a,n,u){var i=l.pendingLanes;l.pendingLanes=e,l.suspendedLanes=0,l.pingedLanes=0,l.warmLanes=0,l.expiredLanes&=e,l.entangledLanes&=e,l.errorRecoveryDisabledLanes&=e,l.shellSuspendCounter=0;var f=l.entanglements,s=l.expirationTimes,v=l.hiddenUpdates;for(e=i&~e;0"u")return null;try{return l.activeElement||l.body}catch{return l.body}}var Wd=/[\n"\\]/g;function ht(l){return l.replace(Wd,function(t){return"\\"+t.charCodeAt(0).toString(16)+" "})}function ti(l,t,e,a,n,u,i,f){l.name="",i!=null&&typeof i!="function"&&typeof i!="symbol"&&typeof i!="boolean"?l.type=i:l.removeAttribute("type"),t!=null?i==="number"?(t===0&&l.value===""||l.value!=t)&&(l.value=""+dt(t)):l.value!==""+dt(t)&&(l.value=""+dt(t)):i!=="submit"&&i!=="reset"||l.removeAttribute("value"),t!=null?ei(l,i,dt(t)):e!=null?ei(l,i,dt(e)):a!=null&&l.removeAttribute("value"),n==null&&u!=null&&(l.defaultChecked=!!u),n!=null&&(l.checked=n&&typeof n!="function"&&typeof n!="symbol"),f!=null&&typeof f!="function"&&typeof f!="symbol"&&typeof f!="boolean"?l.name=""+dt(f):l.removeAttribute("name")}function Uf(l,t,e,a,n,u,i,f){if(u!=null&&typeof u!="function"&&typeof u!="symbol"&&typeof u!="boolean"&&(l.type=u),t!=null||e!=null){if(!(u!=="submit"&&u!=="reset"||t!=null)){li(l);return}e=e!=null?""+dt(e):"",t=t!=null?""+dt(t):e,f||t===l.value||(l.value=t),l.defaultValue=t}a=a??n,a=typeof a!="function"&&typeof a!="symbol"&&!!a,l.checked=f?l.checked:!!a,l.defaultChecked=!!a,i!=null&&typeof i!="function"&&typeof i!="symbol"&&typeof i!="boolean"&&(l.name=i),li(l)}function ei(l,t,e){t==="number"&&Cn(l.ownerDocument)===l||l.defaultValue===""+e||(l.defaultValue=""+e)}function $e(l,t,e,a){if(l=l.options,t){t={};for(var n=0;n"u"||typeof window.document>"u"||typeof window.document.createElement>"u"),ci=!1;if(Ht)try{var Ba={};Object.defineProperty(Ba,"passive",{get:function(){ci=!0}}),window.addEventListener("test",Ba,Ba),window.removeEventListener("test",Ba,Ba)}catch{ci=!1}var le=null,fi=null,Rn=null;function Xf(){if(Rn)return Rn;var l,t=fi,e=t.length,a,n="value"in le?le.value:le.textContent,u=n.length;for(l=0;l=Ya),Jf=" ",wf=!1;function kf(l,t){switch(l){case"keyup":return zh.indexOf(t.keyCode)!==-1;case"keydown":return t.keyCode!==229;case"keypress":case"mousedown":case"focusout":return!0;default:return!1}}function $f(l){return l=l.detail,typeof l=="object"&&"data"in l?l.data:null}var Pe=!1;function _h(l,t){switch(l){case"compositionend":return $f(t);case"keypress":return t.which!==32?null:(wf=!0,Jf);case"textInput":return l=t.data,l===Jf&&wf?null:l;default:return null}}function Eh(l,t){if(Pe)return l==="compositionend"||!hi&&kf(l,t)?(l=Xf(),Rn=fi=le=null,Pe=!1,l):null;switch(l){case"paste":return null;case"keypress":if(!(t.ctrlKey||t.altKey||t.metaKey)||t.ctrlKey&&t.altKey){if(t.char&&1=t)return{node:e,offset:t-l};l=a}l:{for(;e;){if(e.nextSibling){e=e.nextSibling;break l}e=e.parentNode}e=void 0}e=as(e)}}function us(l,t){return l&&t?l===t?!0:l&&l.nodeType===3?!1:t&&t.nodeType===3?us(l,t.parentNode):"contains"in l?l.contains(t):l.compareDocumentPosition?!!(l.compareDocumentPosition(t)&16):!1:!1}function is(l){l=l!=null&&l.ownerDocument!=null&&l.ownerDocument.defaultView!=null?l.ownerDocument.defaultView:window;for(var t=Cn(l.document);t instanceof l.HTMLIFrameElement;){try{var e=typeof t.contentWindow.location.href=="string"}catch{e=!1}if(e)l=t.contentWindow;else break;t=Cn(l.document)}return t}function vi(l){var t=l&&l.nodeName&&l.nodeName.toLowerCase();return t&&(t==="input"&&(l.type==="text"||l.type==="search"||l.type==="tel"||l.type==="url"||l.type==="password")||t==="textarea"||l.contentEditable==="true")}var Bh=Ht&&"documentMode"in document&&11>=document.documentMode,la=null,gi=null,Za=null,Si=!1;function cs(l,t,e){var a=e.window===e?e.document:e.nodeType===9?e:e.ownerDocument;Si||la==null||la!==Cn(a)||(a=la,"selectionStart"in a&&vi(a)?a={start:a.selectionStart,end:a.selectionEnd}:(a=(a.ownerDocument&&a.ownerDocument.defaultView||window).getSelection(),a={anchorNode:a.anchorNode,anchorOffset:a.anchorOffset,focusNode:a.focusNode,focusOffset:a.focusOffset}),Za&&Qa(Za,a)||(Za=a,a=Eu(gi,"onSelect"),0>=i,n-=i,Nt=1<<32-et(t)+n|e<k?(ll=Y,Y=null):ll=Y.sibling;var il=g(m,Y,y[k],j);if(il===null){Y===null&&(Y=ll);break}l&&Y&&il.alternate===null&&t(m,Y),r=u(il,r,k),ul===null?G=il:ul.sibling=il,ul=il,Y=ll}if(k===y.length)return e(m,Y),al&&Yt(m,k),G;if(Y===null){for(;kk?(ll=Y,Y=null):ll=Y.sibling;var je=g(m,Y,il.value,j);if(je===null){Y===null&&(Y=ll);break}l&&Y&&je.alternate===null&&t(m,Y),r=u(je,r,k),ul===null?G=je:ul.sibling=je,ul=je,Y=ll}if(il.done)return e(m,Y),al&&Yt(m,k),G;if(Y===null){for(;!il.done;k++,il=y.next())il=T(m,il.value,j),il!==null&&(r=u(il,r,k),ul===null?G=il:ul.sibling=il,ul=il);return al&&Yt(m,k),G}for(Y=a(Y);!il.done;k++,il=y.next())il=S(Y,m,k,il.value,j),il!==null&&(l&&il.alternate!==null&&Y.delete(il.key===null?k:il.key),r=u(il,r,k),ul===null?G=il:ul.sibling=il,ul=il);return l&&Y.forEach(function(ty){return t(m,ty)}),al&&Yt(m,k),G}function ml(m,r,y,j){if(typeof y=="object"&&y!==null&&y.type===nl&&y.key===null&&(y=y.props.children),typeof y=="object"&&y!==null){switch(y.$$typeof){case I:l:{for(var G=y.key;r!==null;){if(r.key===G){if(G=y.type,G===nl){if(r.tag===7){e(m,r.sibling),j=n(r,y.props.children),j.return=m,m=j;break l}}else if(r.elementType===G||typeof G=="object"&&G!==null&&G.$$typeof===Vl&&He(G)===r.type){e(m,r.sibling),j=n(r,y.props),ka(j,y),j.return=m,m=j;break l}e(m,r);break}else t(m,r);r=r.sibling}y.type===nl?(j=De(y.props.children,m.mode,j,y.key),j.return=m,m=j):(j=Vn(y.type,y.key,y.props,null,m.mode,j),ka(j,y),j.return=m,m=j)}return i(m);case L:l:{for(G=y.key;r!==null;){if(r.key===G)if(r.tag===4&&r.stateNode.containerInfo===y.containerInfo&&r.stateNode.implementation===y.implementation){e(m,r.sibling),j=n(r,y.children||[]),j.return=m,m=j;break l}else{e(m,r);break}else t(m,r);r=r.sibling}j=Ai(y,m.mode,j),j.return=m,m=j}return i(m);case Vl:return y=He(y),ml(m,r,y,j)}if(jt(y))return B(m,r,y,j);if(Kl(y)){if(G=Kl(y),typeof G!="function")throw Error(h(150));return y=G.call(y),X(m,r,y,j)}if(typeof y.then=="function")return ml(m,r,Fn(y),j);if(y.$$typeof===Gl)return ml(m,r,wn(m,y),j);In(m,y)}return typeof y=="string"&&y!==""||typeof y=="number"||typeof y=="bigint"?(y=""+y,r!==null&&r.tag===6?(e(m,r.sibling),j=n(r,y),j.return=m,m=j):(e(m,r),j=zi(y,m.mode,j),j.return=m,m=j),i(m)):e(m,r)}return function(m,r,y,j){try{wa=0;var G=ml(m,r,y,j);return ra=null,G}catch(Y){if(Y===oa||Y===$n)throw Y;var ul=nt(29,Y,null,m.mode);return ul.lanes=j,ul.return=m,ul}finally{}}}var Ye=Ms(!0),Ds=Ms(!1),ue=!1;function qi(l){l.updateQueue={baseState:l.memoizedState,firstBaseUpdate:null,lastBaseUpdate:null,shared:{pending:null,lanes:0,hiddenCallbacks:null},callbacks:null}}function Yi(l,t){l=l.updateQueue,t.updateQueue===l&&(t.updateQueue={baseState:l.baseState,firstBaseUpdate:l.firstBaseUpdate,lastBaseUpdate:l.lastBaseUpdate,shared:l.shared,callbacks:null})}function ie(l){return{lane:l,tag:0,payload:null,callback:null,next:null}}function ce(l,t,e){var a=l.updateQueue;if(a===null)return null;if(a=a.shared,(cl&2)!==0){var n=a.pending;return n===null?t.next=t:(t.next=n.next,n.next=t),a.pending=t,t=Ln(l),ms(l,null,e),t}return Zn(l,a,t,e),Ln(l)}function $a(l,t,e){if(t=t.updateQueue,t!==null&&(t=t.shared,(e&4194048)!==0)){var a=t.lanes;a&=l.pendingLanes,e|=a,t.lanes=e,jf(l,e)}}function Gi(l,t){var e=l.updateQueue,a=l.alternate;if(a!==null&&(a=a.updateQueue,e===a)){var n=null,u=null;if(e=e.firstBaseUpdate,e!==null){do{var i={lane:e.lane,tag:e.tag,payload:e.payload,callback:null,next:null};u===null?n=u=i:u=u.next=i,e=e.next}while(e!==null);u===null?n=u=t:u=u.next=t}else n=u=t;e={baseState:a.baseState,firstBaseUpdate:n,lastBaseUpdate:u,shared:a.shared,callbacks:a.callbacks},l.updateQueue=e;return}l=e.lastBaseUpdate,l===null?e.firstBaseUpdate=t:l.next=t,e.lastBaseUpdate=t}var Xi=!1;function Wa(){if(Xi){var l=sa;if(l!==null)throw l}}function Fa(l,t,e,a){Xi=!1;var n=l.updateQueue;ue=!1;var u=n.firstBaseUpdate,i=n.lastBaseUpdate,f=n.shared.pending;if(f!==null){n.shared.pending=null;var s=f,v=s.next;s.next=null,i===null?u=v:i.next=v,i=s;var p=l.alternate;p!==null&&(p=p.updateQueue,f=p.lastBaseUpdate,f!==i&&(f===null?p.firstBaseUpdate=v:f.next=v,p.lastBaseUpdate=s))}if(u!==null){var T=n.baseState;i=0,p=v=s=null,f=u;do{var g=f.lane&-536870913,S=g!==f.lane;if(S?(P&g)===g:(a&g)===g){g!==0&&g===fa&&(Xi=!0),p!==null&&(p=p.next={lane:0,tag:f.tag,payload:f.payload,callback:null,next:null});l:{var B=l,X=f;g=t;var ml=e;switch(X.tag){case 1:if(B=X.payload,typeof B=="function"){T=B.call(ml,T,g);break l}T=B;break l;case 3:B.flags=B.flags&-65537|128;case 0:if(B=X.payload,g=typeof B=="function"?B.call(ml,T,g):B,g==null)break l;T=M({},T,g);break l;case 2:ue=!0}}g=f.callback,g!==null&&(l.flags|=64,S&&(l.flags|=8192),S=n.callbacks,S===null?n.callbacks=[g]:S.push(g))}else S={lane:g,tag:f.tag,payload:f.payload,callback:f.callback,next:null},p===null?(v=p=S,s=T):p=p.next=S,i|=g;if(f=f.next,f===null){if(f=n.shared.pending,f===null)break;S=f,f=S.next,S.next=null,n.lastBaseUpdate=S,n.shared.pending=null}}while(!0);p===null&&(s=T),n.baseState=s,n.firstBaseUpdate=v,n.lastBaseUpdate=p,u===null&&(n.shared.lanes=0),de|=i,l.lanes=i,l.memoizedState=T}}function Cs(l,t){if(typeof l!="function")throw Error(h(191,l));l.call(t)}function Us(l,t){var e=l.callbacks;if(e!==null)for(l.callbacks=null,l=0;lu?u:8;var i=x.T,f={};x.T=f,uc(l,!1,t,e);try{var s=n(),v=x.S;if(v!==null&&v(f,s),s!==null&&typeof s=="object"&&typeof s.then=="function"){var p=Vh(s,a);ln(l,t,p,st(l))}else ln(l,t,a,st(l))}catch(T){ln(l,t,{then:function(){},status:"rejected",reason:T},st())}finally{U.p=u,i!==null&&f.types!==null&&(i.types=f.types),x.T=i}}function Wh(){}function ac(l,t,e,a){if(l.tag!==5)throw Error(h(476));var n=ro(l).queue;oo(l,n,t,Z,e===null?Wh:function(){return ho(l),e(a)})}function ro(l){var t=l.memoizedState;if(t!==null)return t;t={memoizedState:Z,baseState:Z,baseQueue:null,queue:{pending:null,lanes:0,dispatch:null,lastRenderedReducer:Zt,lastRenderedState:Z},next:null};var e={};return t.next={memoizedState:e,baseState:e,baseQueue:null,queue:{pending:null,lanes:0,dispatch:null,lastRenderedReducer:Zt,lastRenderedState:e},next:null},l.memoizedState=t,l=l.alternate,l!==null&&(l.memoizedState=t),t}function ho(l){var t=ro(l);t.next===null&&(t=l.alternate.memoizedState),ln(l,t.next.queue,{},st())}function nc(){return Hl(Sn)}function mo(){return Al().memoizedState}function yo(){return Al().memoizedState}function Fh(l){for(var t=l.return;t!==null;){switch(t.tag){case 24:case 3:var e=st();l=ie(e);var a=ce(t,l,e);a!==null&&(Il(a,t,e),$a(a,t,e)),t={cache:Ui()},l.payload=t;return}t=t.return}}function Ih(l,t,e){var a=st();e={lane:a,revertLane:0,gesture:null,action:e,hasEagerState:!1,eagerState:null,next:null},fu(l)?go(t,e):(e=ji(l,t,e,a),e!==null&&(Il(e,l,a),So(e,t,a)))}function vo(l,t,e){var a=st();ln(l,t,e,a)}function ln(l,t,e,a){var n={lane:a,revertLane:0,gesture:null,action:e,hasEagerState:!1,eagerState:null,next:null};if(fu(l))go(t,n);else{var u=l.alternate;if(l.lanes===0&&(u===null||u.lanes===0)&&(u=t.lastRenderedReducer,u!==null))try{var i=t.lastRenderedState,f=u(i,e);if(n.hasEagerState=!0,n.eagerState=f,at(f,i))return Zn(l,t,n,0),vl===null&&Qn(),!1}catch{}finally{}if(e=ji(l,t,n,a),e!==null)return Il(e,l,a),So(e,t,a),!0}return!1}function uc(l,t,e,a){if(a={lane:2,revertLane:qc(),gesture:null,action:a,hasEagerState:!1,eagerState:null,next:null},fu(l)){if(t)throw Error(h(479))}else t=ji(l,e,a,2),t!==null&&Il(t,l,2)}function fu(l){var t=l.alternate;return l===w||t!==null&&t===w}function go(l,t){ha=tu=!0;var e=l.pending;e===null?t.next=t:(t.next=e.next,e.next=t),l.pending=t}function So(l,t,e){if((e&4194048)!==0){var a=t.lanes;a&=l.pendingLanes,e|=a,t.lanes=e,jf(l,e)}}var tn={readContext:Hl,use:nu,useCallback:xl,useContext:xl,useEffect:xl,useImperativeHandle:xl,useLayoutEffect:xl,useInsertionEffect:xl,useMemo:xl,useReducer:xl,useRef:xl,useState:xl,useDebugValue:xl,useDeferredValue:xl,useTransition:xl,useSyncExternalStore:xl,useId:xl,useHostTransitionStatus:xl,useFormState:xl,useActionState:xl,useOptimistic:xl,useMemoCache:xl,useCacheRefresh:xl};tn.useEffectEvent=xl;var po={readContext:Hl,use:nu,useCallback:function(l,t){return Zl().memoizedState=[l,t===void 0?null:t],l},useContext:Hl,useEffect:to,useImperativeHandle:function(l,t,e){e=e!=null?e.concat([l]):null,iu(4194308,4,uo.bind(null,t,l),e)},useLayoutEffect:function(l,t){return iu(4194308,4,l,t)},useInsertionEffect:function(l,t){iu(4,2,l,t)},useMemo:function(l,t){var e=Zl();t=t===void 0?null:t;var a=l();if(Ge){It(!0);try{l()}finally{It(!1)}}return e.memoizedState=[a,t],a},useReducer:function(l,t,e){var a=Zl();if(e!==void 0){var n=e(t);if(Ge){It(!0);try{e(t)}finally{It(!1)}}}else n=t;return a.memoizedState=a.baseState=n,l={pending:null,lanes:0,dispatch:null,lastRenderedReducer:l,lastRenderedState:n},a.queue=l,l=l.dispatch=Ih.bind(null,w,l),[a.memoizedState,l]},useRef:function(l){var t=Zl();return l={current:l},t.memoizedState=l},useState:function(l){l=Ii(l);var t=l.queue,e=vo.bind(null,w,t);return t.dispatch=e,[l.memoizedState,e]},useDebugValue:tc,useDeferredValue:function(l,t){var e=Zl();return ec(e,l,t)},useTransition:function(){var l=Ii(!1);return l=oo.bind(null,w,l.queue,!0,!1),Zl().memoizedState=l,[!1,l]},useSyncExternalStore:function(l,t,e){var a=w,n=Zl();if(al){if(e===void 0)throw Error(h(407));e=e()}else{if(e=t(),vl===null)throw Error(h(349));(P&127)!==0||Gs(a,t,e)}n.memoizedState=e;var u={value:e,getSnapshot:t};return n.queue=u,to(Qs.bind(null,a,u,l),[l]),a.flags|=2048,ya(9,{destroy:void 0},Xs.bind(null,a,u,e,t),null),e},useId:function(){var l=Zl(),t=vl.identifierPrefix;if(al){var e=Mt,a=Nt;e=(a&~(1<<32-et(a)-1)).toString(32)+e,t="_"+t+"R_"+e,e=eu++,0<\/script>",u=u.removeChild(u.firstChild);break;case"select":u=typeof a.is=="string"?i.createElement("select",{is:a.is}):i.createElement("select"),a.multiple?u.multiple=!0:a.size&&(u.size=a.size);break;default:u=typeof a.is=="string"?i.createElement(n,{is:a.is}):i.createElement(n)}}u[Rl]=t,u[Jl]=a;l:for(i=t.child;i!==null;){if(i.tag===5||i.tag===6)u.appendChild(i.stateNode);else if(i.tag!==4&&i.tag!==27&&i.child!==null){i.child.return=i,i=i.child;continue}if(i===t)break l;for(;i.sibling===null;){if(i.return===null||i.return===t)break l;i=i.return}i.sibling.return=i.return,i=i.sibling}t.stateNode=u;l:switch(Yl(u,n,a),n){case"button":case"input":case"select":case"textarea":a=!!a.autoFocus;break l;case"img":a=!0;break l;default:a=!1}a&&Vt(t)}}return Sl(t),pc(t,t.type,l===null?null:l.memoizedProps,t.pendingProps,e),null;case 6:if(l&&t.stateNode!=null)l.memoizedProps!==a&&Vt(t);else{if(typeof a!="string"&&t.stateNode===null)throw Error(h(166));if(l=$.current,ia(t)){if(l=t.stateNode,e=t.memoizedProps,a=null,n=Bl,n!==null)switch(n.tag){case 27:case 5:a=n.memoizedProps}l[Rl]=t,l=!!(l.nodeValue===e||a!==null&&a.suppressHydrationWarning===!0||Yr(l.nodeValue,e)),l||ae(t,!0)}else l=Ou(l).createTextNode(a),l[Rl]=t,t.stateNode=l}return Sl(t),null;case 31:if(e=t.memoizedState,l===null||l.memoizedState!==null){if(a=ia(t),e!==null){if(l===null){if(!a)throw Error(h(318));if(l=t.memoizedState,l=l!==null?l.dehydrated:null,!l)throw Error(h(557));l[Rl]=t}else Ce(),(t.flags&128)===0&&(t.memoizedState=null),t.flags|=4;Sl(t),l=!1}else e=Ni(),l!==null&&l.memoizedState!==null&&(l.memoizedState.hydrationErrors=e),l=!0;if(!l)return t.flags&256?(it(t),t):(it(t),null);if((t.flags&128)!==0)throw Error(h(558))}return Sl(t),null;case 13:if(a=t.memoizedState,l===null||l.memoizedState!==null&&l.memoizedState.dehydrated!==null){if(n=ia(t),a!==null&&a.dehydrated!==null){if(l===null){if(!n)throw Error(h(318));if(n=t.memoizedState,n=n!==null?n.dehydrated:null,!n)throw Error(h(317));n[Rl]=t}else Ce(),(t.flags&128)===0&&(t.memoizedState=null),t.flags|=4;Sl(t),n=!1}else n=Ni(),l!==null&&l.memoizedState!==null&&(l.memoizedState.hydrationErrors=n),n=!0;if(!n)return t.flags&256?(it(t),t):(it(t),null)}return it(t),(t.flags&128)!==0?(t.lanes=e,t):(e=a!==null,l=l!==null&&l.memoizedState!==null,e&&(a=t.child,n=null,a.alternate!==null&&a.alternate.memoizedState!==null&&a.alternate.memoizedState.cachePool!==null&&(n=a.alternate.memoizedState.cachePool.pool),u=null,a.memoizedState!==null&&a.memoizedState.cachePool!==null&&(u=a.memoizedState.cachePool.pool),u!==n&&(a.flags|=2048)),e!==l&&e&&(t.child.flags|=8192),hu(t,t.updateQueue),Sl(t),null);case 4:return Tl(),l===null&&Qc(t.stateNode.containerInfo),Sl(t),null;case 10:return Xt(t.type),Sl(t),null;case 19:if(z(zl),a=t.memoizedState,a===null)return Sl(t),null;if(n=(t.flags&128)!==0,u=a.rendering,u===null)if(n)an(a,!1);else{if(jl!==0||l!==null&&(l.flags&128)!==0)for(l=t.child;l!==null;){if(u=lu(l),u!==null){for(t.flags|=128,an(a,!1),l=u.updateQueue,t.updateQueue=l,hu(t,l),t.subtreeFlags=0,l=e,e=t.child;e!==null;)ys(e,l),e=e.sibling;return R(zl,zl.current&1|2),al&&Yt(t,a.treeForkCount),t.child}l=l.sibling}a.tail!==null&<()>Su&&(t.flags|=128,n=!0,an(a,!1),t.lanes=4194304)}else{if(!n)if(l=lu(u),l!==null){if(t.flags|=128,n=!0,l=l.updateQueue,t.updateQueue=l,hu(t,l),an(a,!0),a.tail===null&&a.tailMode==="hidden"&&!u.alternate&&!al)return Sl(t),null}else 2*lt()-a.renderingStartTime>Su&&e!==536870912&&(t.flags|=128,n=!0,an(a,!1),t.lanes=4194304);a.isBackwards?(u.sibling=t.child,t.child=u):(l=a.last,l!==null?l.sibling=u:t.child=u,a.last=u)}return a.tail!==null?(l=a.tail,a.rendering=l,a.tail=l.sibling,a.renderingStartTime=lt(),l.sibling=null,e=zl.current,R(zl,n?e&1|2:e&1),al&&Yt(t,a.treeForkCount),l):(Sl(t),null);case 22:case 23:return it(t),Zi(),a=t.memoizedState!==null,l!==null?l.memoizedState!==null!==a&&(t.flags|=8192):a&&(t.flags|=8192),a?(e&536870912)!==0&&(t.flags&128)===0&&(Sl(t),t.subtreeFlags&6&&(t.flags|=8192)):Sl(t),e=t.updateQueue,e!==null&&hu(t,e.retryQueue),e=null,l!==null&&l.memoizedState!==null&&l.memoizedState.cachePool!==null&&(e=l.memoizedState.cachePool.pool),a=null,t.memoizedState!==null&&t.memoizedState.cachePool!==null&&(a=t.memoizedState.cachePool.pool),a!==e&&(t.flags|=2048),l!==null&&z(Be),null;case 24:return e=null,l!==null&&(e=l.memoizedState.cache),t.memoizedState.cache!==e&&(t.flags|=2048),Xt(_l),Sl(t),null;case 25:return null;case 30:return null}throw Error(h(156,t.tag))}function am(l,t){switch(Ei(t),t.tag){case 1:return l=t.flags,l&65536?(t.flags=l&-65537|128,t):null;case 3:return Xt(_l),Tl(),l=t.flags,(l&65536)!==0&&(l&128)===0?(t.flags=l&-65537|128,t):null;case 26:case 27:case 5:return zn(t),null;case 31:if(t.memoizedState!==null){if(it(t),t.alternate===null)throw Error(h(340));Ce()}return l=t.flags,l&65536?(t.flags=l&-65537|128,t):null;case 13:if(it(t),l=t.memoizedState,l!==null&&l.dehydrated!==null){if(t.alternate===null)throw Error(h(340));Ce()}return l=t.flags,l&65536?(t.flags=l&-65537|128,t):null;case 19:return z(zl),null;case 4:return Tl(),null;case 10:return Xt(t.type),null;case 22:case 23:return it(t),Zi(),l!==null&&z(Be),l=t.flags,l&65536?(t.flags=l&-65537|128,t):null;case 24:return Xt(_l),null;case 25:return null;default:return null}}function Lo(l,t){switch(Ei(t),t.tag){case 3:Xt(_l),Tl();break;case 26:case 27:case 5:zn(t);break;case 4:Tl();break;case 31:t.memoizedState!==null&&it(t);break;case 13:it(t);break;case 19:z(zl);break;case 10:Xt(t.type);break;case 22:case 23:it(t),Zi(),l!==null&&z(Be);break;case 24:Xt(_l)}}function nn(l,t){try{var e=t.updateQueue,a=e!==null?e.lastEffect:null;if(a!==null){var n=a.next;e=n;do{if((e.tag&l)===l){a=void 0;var u=e.create,i=e.inst;a=u(),i.destroy=a}e=e.next}while(e!==n)}}catch(f){ol(t,t.return,f)}}function oe(l,t,e){try{var a=t.updateQueue,n=a!==null?a.lastEffect:null;if(n!==null){var u=n.next;a=u;do{if((a.tag&l)===l){var i=a.inst,f=i.destroy;if(f!==void 0){i.destroy=void 0,n=t;var s=e,v=f;try{v()}catch(p){ol(n,s,p)}}}a=a.next}while(a!==u)}}catch(p){ol(t,t.return,p)}}function Vo(l){var t=l.updateQueue;if(t!==null){var e=l.stateNode;try{Us(t,e)}catch(a){ol(l,l.return,a)}}}function Ko(l,t,e){e.props=Xe(l.type,l.memoizedProps),e.state=l.memoizedState;try{e.componentWillUnmount()}catch(a){ol(l,t,a)}}function un(l,t){try{var e=l.ref;if(e!==null){switch(l.tag){case 26:case 27:case 5:var a=l.stateNode;break;case 30:a=l.stateNode;break;default:a=l.stateNode}typeof e=="function"?l.refCleanup=e(a):e.current=a}}catch(n){ol(l,t,n)}}function Dt(l,t){var e=l.ref,a=l.refCleanup;if(e!==null)if(typeof a=="function")try{a()}catch(n){ol(l,t,n)}finally{l.refCleanup=null,l=l.alternate,l!=null&&(l.refCleanup=null)}else if(typeof e=="function")try{e(null)}catch(n){ol(l,t,n)}else e.current=null}function Jo(l){var t=l.type,e=l.memoizedProps,a=l.stateNode;try{l:switch(t){case"button":case"input":case"select":case"textarea":e.autoFocus&&a.focus();break l;case"img":e.src?a.src=e.src:e.srcSet&&(a.srcset=e.srcSet)}}catch(n){ol(l,l.return,n)}}function bc(l,t,e){try{var a=l.stateNode;Am(a,l.type,e,t),a[Jl]=t}catch(n){ol(l,l.return,n)}}function wo(l){return l.tag===5||l.tag===3||l.tag===26||l.tag===27&&ge(l.type)||l.tag===4}function xc(l){l:for(;;){for(;l.sibling===null;){if(l.return===null||wo(l.return))return null;l=l.return}for(l.sibling.return=l.return,l=l.sibling;l.tag!==5&&l.tag!==6&&l.tag!==18;){if(l.tag===27&&ge(l.type)||l.flags&2||l.child===null||l.tag===4)continue l;l.child.return=l,l=l.child}if(!(l.flags&2))return l.stateNode}}function jc(l,t,e){var a=l.tag;if(a===5||a===6)l=l.stateNode,t?(e.nodeType===9?e.body:e.nodeName==="HTML"?e.ownerDocument.body:e).insertBefore(l,t):(t=e.nodeType===9?e.body:e.nodeName==="HTML"?e.ownerDocument.body:e,t.appendChild(l),e=e._reactRootContainer,e!=null||t.onclick!==null||(t.onclick=Bt));else if(a!==4&&(a===27&&ge(l.type)&&(e=l.stateNode,t=null),l=l.child,l!==null))for(jc(l,t,e),l=l.sibling;l!==null;)jc(l,t,e),l=l.sibling}function mu(l,t,e){var a=l.tag;if(a===5||a===6)l=l.stateNode,t?e.insertBefore(l,t):e.appendChild(l);else if(a!==4&&(a===27&&ge(l.type)&&(e=l.stateNode),l=l.child,l!==null))for(mu(l,t,e),l=l.sibling;l!==null;)mu(l,t,e),l=l.sibling}function ko(l){var t=l.stateNode,e=l.memoizedProps;try{for(var a=l.type,n=t.attributes;n.length;)t.removeAttributeNode(n[0]);Yl(t,a,e),t[Rl]=l,t[Jl]=e}catch(u){ol(l,l.return,u)}}var Kt=!1,Nl=!1,Tc=!1,$o=typeof WeakSet=="function"?WeakSet:Set,Ul=null;function nm(l,t){if(l=l.containerInfo,Vc=Bu,l=is(l),vi(l)){if("selectionStart"in l)var e={start:l.selectionStart,end:l.selectionEnd};else l:{e=(e=l.ownerDocument)&&e.defaultView||window;var a=e.getSelection&&e.getSelection();if(a&&a.rangeCount!==0){e=a.anchorNode;var n=a.anchorOffset,u=a.focusNode;a=a.focusOffset;try{e.nodeType,u.nodeType}catch{e=null;break l}var i=0,f=-1,s=-1,v=0,p=0,T=l,g=null;t:for(;;){for(var S;T!==e||n!==0&&T.nodeType!==3||(f=i+n),T!==u||a!==0&&T.nodeType!==3||(s=i+a),T.nodeType===3&&(i+=T.nodeValue.length),(S=T.firstChild)!==null;)g=T,T=S;for(;;){if(T===l)break t;if(g===e&&++v===n&&(f=i),g===u&&++p===a&&(s=i),(S=T.nextSibling)!==null)break;T=g,g=T.parentNode}T=S}e=f===-1||s===-1?null:{start:f,end:s}}else e=null}e=e||{start:0,end:0}}else e=null;for(Kc={focusedElem:l,selectionRange:e},Bu=!1,Ul=t;Ul!==null;)if(t=Ul,l=t.child,(t.subtreeFlags&1028)!==0&&l!==null)l.return=t,Ul=l;else for(;Ul!==null;){switch(t=Ul,u=t.alternate,l=t.flags,t.tag){case 0:if((l&4)!==0&&(l=t.updateQueue,l=l!==null?l.events:null,l!==null))for(e=0;e title"))),Yl(u,a,e),u[Rl]=l,Cl(u),a=u;break l;case"link":var i=td("link","href",n).get(a+(e.href||""));if(i){for(var f=0;fml&&(i=ml,ml=X,X=i);var m=ns(f,X),r=ns(f,ml);if(m&&r&&(S.rangeCount!==1||S.anchorNode!==m.node||S.anchorOffset!==m.offset||S.focusNode!==r.node||S.focusOffset!==r.offset)){var y=T.createRange();y.setStart(m.node,m.offset),S.removeAllRanges(),X>ml?(S.addRange(y),S.extend(r.node,r.offset)):(y.setEnd(r.node,r.offset),S.addRange(y))}}}}for(T=[],S=f;S=S.parentNode;)S.nodeType===1&&T.push({element:S,left:S.scrollLeft,top:S.scrollTop});for(typeof f.focus=="function"&&f.focus(),f=0;fe?32:e,x.T=null,e=Mc,Mc=null;var u=me,i=Wt;if(Dl=0,ba=me=null,Wt=0,(cl&6)!==0)throw Error(h(331));var f=cl;if(cl|=4,ir(u.current),ar(u,u.current,i,e),cl=f,dn(0,!1),tt&&typeof tt.onPostCommitFiberRoot=="function")try{tt.onPostCommitFiberRoot(Na,u)}catch{}return!0}finally{U.p=n,x.T=a,zr(l,t)}}function _r(l,t,e){t=yt(e,t),t=sc(l.stateNode,t,2),l=ce(l,t,2),l!==null&&(Da(l,2),Ct(l))}function ol(l,t,e){if(l.tag===3)_r(l,l,e);else for(;t!==null;){if(t.tag===3){_r(t,l,e);break}else if(t.tag===1){var a=t.stateNode;if(typeof t.type.getDerivedStateFromError=="function"||typeof a.componentDidCatch=="function"&&(he===null||!he.has(a))){l=yt(e,l),e=Eo(2),a=ce(t,e,2),a!==null&&(Oo(e,a,t,l),Da(a,2),Ct(a));break}}t=t.return}}function Rc(l,t,e){var a=l.pingCache;if(a===null){a=l.pingCache=new cm;var n=new Set;a.set(t,n)}else n=a.get(t),n===void 0&&(n=new Set,a.set(t,n));n.has(e)||(_c=!0,n.add(e),l=dm.bind(null,l,t,e),t.then(l,l))}function dm(l,t,e){var a=l.pingCache;a!==null&&a.delete(t),l.pingedLanes|=l.suspendedLanes&e,l.warmLanes&=~e,vl===l&&(P&e)===e&&(jl===4||jl===3&&(P&62914560)===P&&300>lt()-gu?(cl&2)===0&&xa(l,0):Ec|=e,pa===P&&(pa=0)),Ct(l)}function Er(l,t){t===0&&(t=bf()),l=Me(l,t),l!==null&&(Da(l,t),Ct(l))}function hm(l){var t=l.memoizedState,e=0;t!==null&&(e=t.retryLane),Er(l,e)}function mm(l,t){var e=0;switch(l.tag){case 31:case 13:var a=l.stateNode,n=l.memoizedState;n!==null&&(e=n.retryLane);break;case 19:a=l.stateNode;break;case 22:a=l.stateNode._retryCache;break;default:throw Error(h(314))}a!==null&&a.delete(t),Er(l,e)}function ym(l,t){return wu(l,t)}var zu=null,Ta=null,Bc=!1,Au=!1,Hc=!1,ve=0;function Ct(l){l!==Ta&&l.next===null&&(Ta===null?zu=Ta=l:Ta=Ta.next=l),Au=!0,Bc||(Bc=!0,gm())}function dn(l,t){if(!Hc&&Au){Hc=!0;do for(var e=!1,a=zu;a!==null;){if(l!==0){var n=a.pendingLanes;if(n===0)var u=0;else{var i=a.suspendedLanes,f=a.pingedLanes;u=(1<<31-et(42|l)+1)-1,u&=n&~(i&~f),u=u&201326741?u&201326741|1:u?u|2:0}u!==0&&(e=!0,Dr(a,u))}else u=P,u=Nn(a,a===vl?u:0,a.cancelPendingCommit!==null||a.timeoutHandle!==-1),(u&3)===0||Ma(a,u)||(e=!0,Dr(a,u));a=a.next}while(e);Hc=!1}}function vm(){Or()}function Or(){Au=Bc=!1;var l=0;ve!==0&&Em()&&(l=ve);for(var t=lt(),e=null,a=zu;a!==null;){var n=a.next,u=Nr(a,t);u===0?(a.next=null,e===null?zu=n:e.next=n,n===null&&(Ta=e)):(e=a,(l!==0||(u&3)!==0)&&(Au=!0)),a=n}Dl!==0&&Dl!==5||dn(l),ve!==0&&(ve=0)}function Nr(l,t){for(var e=l.suspendedLanes,a=l.pingedLanes,n=l.expirationTimes,u=l.pendingLanes&-62914561;0f)break;var p=s.transferSize,T=s.initiatorType;p&&Gr(T)&&(s=s.responseEnd,i+=p*(s"u"?null:document;function Fr(l,t,e){var a=za;if(a&&typeof t=="string"&&t){var n=ht(t);n='link[rel="'+l+'"][href="'+n+'"]',typeof e=="string"&&(n+='[crossorigin="'+e+'"]'),Wr.has(n)||(Wr.add(n),l={rel:l,crossOrigin:e,href:t},a.querySelector(n)===null&&(t=a.createElement("link"),Yl(t,"link",l),Cl(t),a.head.appendChild(t)))}}function Hm(l){Ft.D(l),Fr("dns-prefetch",l,null)}function qm(l,t){Ft.C(l,t),Fr("preconnect",l,t)}function Ym(l,t,e){Ft.L(l,t,e);var a=za;if(a&&l&&t){var n='link[rel="preload"][as="'+ht(t)+'"]';t==="image"&&e&&e.imageSrcSet?(n+='[imagesrcset="'+ht(e.imageSrcSet)+'"]',typeof e.imageSizes=="string"&&(n+='[imagesizes="'+ht(e.imageSizes)+'"]')):n+='[href="'+ht(l)+'"]';var u=n;switch(t){case"style":u=Aa(l);break;case"script":u=_a(l)}xt.has(u)||(l=M({rel:"preload",href:t==="image"&&e&&e.imageSrcSet?void 0:l,as:t},e),xt.set(u,l),a.querySelector(n)!==null||t==="style"&&a.querySelector(vn(u))||t==="script"&&a.querySelector(gn(u))||(t=a.createElement("link"),Yl(t,"link",l),Cl(t),a.head.appendChild(t)))}}function Gm(l,t){Ft.m(l,t);var e=za;if(e&&l){var a=t&&typeof t.as=="string"?t.as:"script",n='link[rel="modulepreload"][as="'+ht(a)+'"][href="'+ht(l)+'"]',u=n;switch(a){case"audioworklet":case"paintworklet":case"serviceworker":case"sharedworker":case"worker":case"script":u=_a(l)}if(!xt.has(u)&&(l=M({rel:"modulepreload",href:l},t),xt.set(u,l),e.querySelector(n)===null)){switch(a){case"audioworklet":case"paintworklet":case"serviceworker":case"sharedworker":case"worker":case"script":if(e.querySelector(gn(u)))return}a=e.createElement("link"),Yl(a,"link",l),Cl(a),e.head.appendChild(a)}}}function Xm(l,t,e){Ft.S(l,t,e);var a=za;if(a&&l){var n=we(a).hoistableStyles,u=Aa(l);t=t||"default";var i=n.get(u);if(!i){var f={loading:0,preload:null};if(i=a.querySelector(vn(u)))f.loading=5;else{l=M({rel:"stylesheet",href:l,"data-precedence":t},e),(e=xt.get(u))&&Ic(l,e);var s=i=a.createElement("link");Cl(s),Yl(s,"link",l),s._p=new Promise(function(v,p){s.onload=v,s.onerror=p}),s.addEventListener("load",function(){f.loading|=1}),s.addEventListener("error",function(){f.loading|=2}),f.loading|=4,Mu(i,t,a)}i={type:"stylesheet",instance:i,count:1,state:f},n.set(u,i)}}}function Qm(l,t){Ft.X(l,t);var e=za;if(e&&l){var a=we(e).hoistableScripts,n=_a(l),u=a.get(n);u||(u=e.querySelector(gn(n)),u||(l=M({src:l,async:!0},t),(t=xt.get(n))&&Pc(l,t),u=e.createElement("script"),Cl(u),Yl(u,"link",l),e.head.appendChild(u)),u={type:"script",instance:u,count:1,state:null},a.set(n,u))}}function Zm(l,t){Ft.M(l,t);var e=za;if(e&&l){var a=we(e).hoistableScripts,n=_a(l),u=a.get(n);u||(u=e.querySelector(gn(n)),u||(l=M({src:l,async:!0,type:"module"},t),(t=xt.get(n))&&Pc(l,t),u=e.createElement("script"),Cl(u),Yl(u,"link",l),e.head.appendChild(u)),u={type:"script",instance:u,count:1,state:null},a.set(n,u))}}function Ir(l,t,e,a){var n=(n=$.current)?Nu(n):null;if(!n)throw Error(h(446));switch(l){case"meta":case"title":return null;case"style":return typeof e.precedence=="string"&&typeof e.href=="string"?(t=Aa(e.href),e=we(n).hoistableStyles,a=e.get(t),a||(a={type:"style",instance:null,count:0,state:null},e.set(t,a)),a):{type:"void",instance:null,count:0,state:null};case"link":if(e.rel==="stylesheet"&&typeof e.href=="string"&&typeof e.precedence=="string"){l=Aa(e.href);var u=we(n).hoistableStyles,i=u.get(l);if(i||(n=n.ownerDocument||n,i={type:"stylesheet",instance:null,count:0,state:{loading:0,preload:null}},u.set(l,i),(u=n.querySelector(vn(l)))&&!u._p&&(i.instance=u,i.state.loading=5),xt.has(l)||(e={rel:"preload",as:"style",href:e.href,crossOrigin:e.crossOrigin,integrity:e.integrity,media:e.media,hrefLang:e.hrefLang,referrerPolicy:e.referrerPolicy},xt.set(l,e),u||Lm(n,l,e,i.state))),t&&a===null)throw Error(h(528,""));return i}if(t&&a!==null)throw Error(h(529,""));return null;case"script":return t=e.async,e=e.src,typeof e=="string"&&t&&typeof t!="function"&&typeof t!="symbol"?(t=_a(e),e=we(n).hoistableScripts,a=e.get(t),a||(a={type:"script",instance:null,count:0,state:null},e.set(t,a)),a):{type:"void",instance:null,count:0,state:null};default:throw Error(h(444,l))}}function Aa(l){return'href="'+ht(l)+'"'}function vn(l){return'link[rel="stylesheet"]['+l+"]"}function Pr(l){return M({},l,{"data-precedence":l.precedence,precedence:null})}function Lm(l,t,e,a){l.querySelector('link[rel="preload"][as="style"]['+t+"]")?a.loading=1:(t=l.createElement("link"),a.preload=t,t.addEventListener("load",function(){return a.loading|=1}),t.addEventListener("error",function(){return a.loading|=2}),Yl(t,"link",e),Cl(t),l.head.appendChild(t))}function _a(l){return'[src="'+ht(l)+'"]'}function gn(l){return"script[async]"+l}function ld(l,t,e){if(t.count++,t.instance===null)switch(t.type){case"style":var a=l.querySelector('style[data-href~="'+ht(e.href)+'"]');if(a)return t.instance=a,Cl(a),a;var n=M({},e,{"data-href":e.href,"data-precedence":e.precedence,href:null,precedence:null});return a=(l.ownerDocument||l).createElement("style"),Cl(a),Yl(a,"style",n),Mu(a,e.precedence,l),t.instance=a;case"stylesheet":n=Aa(e.href);var u=l.querySelector(vn(n));if(u)return t.state.loading|=4,t.instance=u,Cl(u),u;a=Pr(e),(n=xt.get(n))&&Ic(a,n),u=(l.ownerDocument||l).createElement("link"),Cl(u);var i=u;return i._p=new Promise(function(f,s){i.onload=f,i.onerror=s}),Yl(u,"link",a),t.state.loading|=4,Mu(u,e.precedence,l),t.instance=u;case"script":return u=_a(e.src),(n=l.querySelector(gn(u)))?(t.instance=n,Cl(n),n):(a=e,(n=xt.get(u))&&(a=M({},e),Pc(a,n)),l=l.ownerDocument||l,n=l.createElement("script"),Cl(n),Yl(n,"link",a),l.head.appendChild(n),t.instance=n);case"void":return null;default:throw Error(h(443,t.type))}else t.type==="stylesheet"&&(t.state.loading&4)===0&&(a=t.instance,t.state.loading|=4,Mu(a,e.precedence,l));return t.instance}function Mu(l,t,e){for(var a=e.querySelectorAll('link[rel="stylesheet"][data-precedence],style[data-precedence]'),n=a.length?a[a.length-1]:null,u=n,i=0;i title"):null)}function Vm(l,t,e){if(e===1||t.itemProp!=null)return!1;switch(l){case"meta":case"title":return!0;case"style":if(typeof t.precedence!="string"||typeof t.href!="string"||t.href==="")break;return!0;case"link":if(typeof t.rel!="string"||typeof t.href!="string"||t.href===""||t.onLoad||t.onError)break;switch(t.rel){case"stylesheet":return l=t.disabled,typeof t.precedence=="string"&&l==null;default:return!0}case"script":if(t.async&&typeof t.async!="function"&&typeof t.async!="symbol"&&!t.onLoad&&!t.onError&&t.src&&typeof t.src=="string")return!0}return!1}function ad(l){return!(l.type==="stylesheet"&&(l.state.loading&3)===0)}function Km(l,t,e,a){if(e.type==="stylesheet"&&(typeof a.media!="string"||matchMedia(a.media).matches!==!1)&&(e.state.loading&4)===0){if(e.instance===null){var n=Aa(a.href),u=t.querySelector(vn(n));if(u){t=u._p,t!==null&&typeof t=="object"&&typeof t.then=="function"&&(l.count++,l=Cu.bind(l),t.then(l,l)),e.state.loading|=4,e.instance=u,Cl(u);return}u=t.ownerDocument||t,a=Pr(a),(n=xt.get(n))&&Ic(a,n),u=u.createElement("link"),Cl(u);var i=u;i._p=new Promise(function(f,s){i.onload=f,i.onerror=s}),Yl(u,"link",a),e.instance=u}l.stylesheets===null&&(l.stylesheets=new Map),l.stylesheets.set(e,t),(t=e.state.preload)&&(e.state.loading&3)===0&&(l.count++,e=Cu.bind(l),t.addEventListener("load",e),t.addEventListener("error",e))}}var lf=0;function Jm(l,t){return l.stylesheets&&l.count===0&&Ru(l,l.stylesheets),0lf?50:800)+t);return l.unsuspend=e,function(){l.unsuspend=null,clearTimeout(a),clearTimeout(n)}}:null}function Cu(){if(this.count--,this.count===0&&(this.imgCount===0||!this.waitingForImages)){if(this.stylesheets)Ru(this,this.stylesheets);else if(this.unsuspend){var l=this.unsuspend;this.unsuspend=null,l()}}}var Uu=null;function Ru(l,t){l.stylesheets=null,l.unsuspend!==null&&(l.count++,Uu=new Map,t.forEach(wm,l),Uu=null,Cu.call(l))}function wm(l,t){if(!(t.state.loading&4)){var e=Uu.get(l);if(e)var a=e.get(null);else{e=new Map,Uu.set(l,e);for(var n=l.querySelectorAll("link[data-precedence],style[data-precedence]"),u=0;u"u"||typeof __REACT_DEVTOOLS_GLOBAL_HOOK__.checkDCE!="function"))try{__REACT_DEVTOOLS_GLOBAL_HOOK__.checkDCE(o)}catch(D){console.error(D)}}return o(),of.exports=sy(),of.exports}var ry=oy();const dy=Md(ry),Cd="";async function ot(o,D){const O=await fetch(`${Cd}${o}`,{...D,credentials:"same-origin",headers:{"Content-Type":"application/json",...D==null?void 0:D.headers}});if(O.status===401)throw window.location.hash="#login",new Error("Unauthorized");if(!O.ok){const h=await O.json().catch(()=>({}));throw new Error(h.error||`HTTP ${O.status}`)}return O.json()}async function hy(o){const D=await fetch(`${Cd}${o}`,{credentials:"same-origin"});if(D.status===401)throw window.location.hash="#login",new Error("Unauthorized");if(!D.ok)throw new Error(`HTTP ${D.status}`);return D.text()}const Pl={login:o=>ot("/admin/login",{method:"POST",body:JSON.stringify({token:o})}),signOutEverywhere:()=>ot("/admin/api/sign-out-everywhere",{method:"POST"}),stats:()=>ot("/admin/api/stats"),health:()=>ot("/admin/api/health-indicators"),agents:()=>ot("/admin/api/agents"),requests:(o=1,D="")=>ot(`/admin/api/requests?page=${o}${D}`),apiKeys:()=>ot("/admin/api/api-keys"),createApiKey:o=>ot("/admin/api/api-keys",{method:"POST",body:JSON.stringify({name:o})}),revokeApiKey:o=>ot("/admin/api/api-keys/revoke",{method:"POST",body:JSON.stringify({name:o})}),updateClientTtl:(o,D)=>ot("/admin/api/update-client-ttl",{method:"POST",body:JSON.stringify({clientId:o,tokenTtl:D})}),revokeClient:o=>ot("/admin/api/revoke-client",{method:"POST",body:JSON.stringify({clientId:o})}),calibrationProfile:o=>ot(`/admin/api/calibration/profile${o?`?holder=${encodeURIComponent(o)}`:""}`),calibrationChart:(o,D)=>hy(`/admin/api/calibration/charts/${encodeURIComponent(o)}${D?`?holder=${encodeURIComponent(D)}`:""}`),jobsWatch:()=>ot("/admin/api/jobs/watch")};function my({onLogin:o}){const[D,O]=K.useState(""),[h,E]=K.useState(""),[N,C]=K.useState(!1),Q=async _=>{_.preventDefault(),E(""),C(!0);try{await Pl.login(D),O(""),o()}catch{E("Invalid token.")}finally{C(!1)}};return c.jsx("div",{className:"login-page",children:c.jsxs("div",{className:"login-box",children:[c.jsx("div",{className:"login-logo",children:"GBrain"}),c.jsxs("div",{style:{background:"rgba(136, 170, 255, 0.08)",border:"1px solid rgba(136, 170, 255, 0.2)",borderRadius:8,padding:"14px 16px",marginBottom:20,fontSize:13,lineHeight:1.5,color:"var(--text-secondary)"},children:[c.jsx("div",{style:{fontWeight:600,color:"var(--text-primary)",marginBottom:6},children:"🔒 This is a protected dashboard"}),"Ask your AI agent for the admin login link:",c.jsx("div",{style:{background:"rgba(0,0,0,0.3)",borderRadius:6,padding:"8px 12px",marginTop:8,fontFamily:"var(--font-mono)",fontSize:12,color:"#88aaff",wordBreak:"break-all"},children:'"Give me the GBrain admin login link"'}),c.jsx("div",{style:{marginTop:8,fontSize:12,color:"var(--text-muted)"},children:"Each link is single-use. Your agent generates a fresh one each time."})]}),c.jsxs("details",{style:{marginBottom:16},children:[c.jsx("summary",{style:{cursor:"pointer",fontSize:13,color:"var(--text-muted)"},children:"Or paste bootstrap token manually"}),c.jsxs("form",{onSubmit:Q,style:{marginTop:12},children:[c.jsx("div",{style:{marginBottom:12},children:c.jsx("input",{type:"password",placeholder:"Admin Token",value:D,onChange:_=>O(_.target.value)})}),c.jsx("button",{className:"btn btn-primary",style:{width:"100%"},disabled:N,children:N?"Authenticating...":"Submit"}),h&&c.jsx("div",{className:"login-error",children:h})]})]})]})})}function yy(){const[o,D]=K.useState({connected_agents:0,requests_today:0,active_tokens:0}),[O,h]=K.useState({expiring_soon:0,error_rate:"0%"}),[E,N]=K.useState([]),[C,Q]=K.useState("connecting"),_=K.useRef(null);K.useEffect(()=>{Pl.stats().then(D).catch(()=>{}),Pl.health().then(h).catch(()=>{});const H=new EventSource("/admin/events");_.current=H,H.onopen=()=>Q("connected"),H.onmessage=A=>{try{const I=JSON.parse(A.data);N(L=>[I,...L].slice(0,50))}catch{}},H.onerror=()=>{Q("disconnected"),setTimeout(()=>{Q("connecting"),H.close()},3e3)};const M=setInterval(()=>{Pl.stats().then(D).catch(()=>{}),Pl.health().then(h).catch(()=>{})},3e4);return()=>{H.close(),clearInterval(M)}},[]);const b=H=>{const M=Date.now()-new Date(H).getTime();return M<6e4?`${Math.floor(M/1e3)}s ago`:M<36e5?`${Math.floor(M/6e4)} min ago`:`${Math.floor(M/36e5)}h ago`};return c.jsxs(c.Fragment,{children:[c.jsx("h1",{className:"page-title",children:"Dashboard"}),c.jsxs("div",{style:{display:"flex",gap:24},children:[c.jsxs("div",{style:{flex:1},children:[c.jsxs("div",{className:"metrics",children:[c.jsxs("div",{className:"metric",children:[c.jsx("div",{className:"metric-value",children:o.connected_agents}),c.jsx("div",{className:"metric-label",children:"Connected Agents"})]}),c.jsxs("div",{className:"metric",children:[c.jsx("div",{className:"metric-value",children:o.requests_today}),c.jsx("div",{className:"metric-label",children:"Requests Today"})]}),c.jsxs("div",{className:"metric",children:[c.jsx("div",{className:"metric-value",children:o.active_tokens}),c.jsx("div",{className:"metric-label",children:"Active Tokens"})]})]}),c.jsxs("h2",{className:"section-title",children:["Live Activity",c.jsx("span",{style:{marginLeft:8,fontSize:10,color:C==="connected"?"var(--success)":C==="connecting"?"var(--warning)":"var(--error)"},children:C==="connected"?"● connected":C==="connecting"?"● connecting...":"● disconnected"})]}),c.jsx("div",{className:"feed",children:E.length===0?c.jsx("div",{className:"feed-empty",children:C==="connected"?"No requests yet. Agents will appear when they connect.":"Connecting..."}):c.jsxs("table",{children:[c.jsx("thead",{children:c.jsxs("tr",{children:[c.jsx("th",{children:"Agent"}),c.jsx("th",{children:"Operation"}),c.jsx("th",{children:"Scopes"}),c.jsx("th",{children:"Latency"}),c.jsx("th",{children:"Status"}),c.jsx("th",{children:"Time"})]})}),c.jsx("tbody",{children:E.map((H,M)=>c.jsxs("tr",{children:[c.jsx("td",{className:"mono",children:H.agent}),c.jsx("td",{className:"mono",children:H.operation}),c.jsx("td",{children:H.scopes.split(",").map(A=>c.jsx("span",{className:`badge badge-${A.trim()}`,style:{marginRight:4},children:A.trim()},A))}),c.jsxs("td",{className:"mono",children:[H.latency_ms," ms"]}),c.jsx("td",{children:c.jsx("span",{className:`badge badge-${H.status}`,children:H.status})}),c.jsx("td",{style:{color:"var(--text-secondary)"},children:b(H.timestamp)})]},M))})]})})]}),c.jsxs("div",{style:{width:220},children:[c.jsx("h2",{className:"section-title",children:"Token Health"}),c.jsxs("div",{className:"health-panel",children:[c.jsxs("div",{className:"health-row",children:[c.jsx("span",{style:{color:"var(--warning)"},children:"Expiring Soon"}),c.jsx("span",{className:"mono",children:O.expiring_soon})]}),c.jsxs("div",{className:"health-row",children:[c.jsx("span",{style:{color:"var(--error)"},children:"Error Rate"}),c.jsx("span",{className:"mono",children:O.error_rate})]})]})]})]})]})}const Ed=["admin","agent","read","sources_admin","users_admin","write"];function vy(o){const D=Math.floor((Date.now()-o.getTime())/1e3);return D<60?"just now":D<3600?`${Math.floor(D/60)}m ago`:D<86400?`${Math.floor(D/3600)}h ago`:`${Math.floor(D/86400)}d ago`}function gy(){const[o,D]=K.useState([]),[O,h]=K.useState(!0),[E,N]=K.useState(!1),[C,Q]=K.useState(null),[_,b]=K.useState(!1),[H,M]=K.useState(null),[A,I]=K.useState(null);K.useEffect(()=>{L()},[]);const L=()=>{Pl.agents().then(D).catch(()=>{})};return c.jsxs(c.Fragment,{children:[c.jsxs("div",{style:{display:"flex",justifyContent:"space-between",alignItems:"center",marginBottom:24},children:[c.jsx("h1",{className:"page-title",style:{marginBottom:0},children:"Agents"}),c.jsxs("div",{style:{display:"flex",gap:8,alignItems:"center"},children:[c.jsxs("label",{style:{fontSize:13,color:"var(--text-secondary)",display:"flex",alignItems:"center",gap:6,cursor:"pointer"},children:[c.jsx("input",{type:"checkbox",checked:O,onChange:nl=>h(nl.target.checked)})," Hide revoked"]}),c.jsx("button",{className:"btn btn-secondary",onClick:()=>b(!0),children:"+ API Key"}),c.jsx("button",{className:"btn btn-primary",onClick:()=>N(!0),children:"+ OAuth Client"})]})]}),(()=>{const nl=o.filter(tl=>!O||tl.status!=="revoked");return o.length===0?c.jsx("div",{style:{textAlign:"center",padding:48,color:"var(--text-muted)"},children:"No agents registered. Register your first agent to get started."}):nl.length===0?c.jsx("div",{style:{textAlign:"center",padding:48,color:"var(--text-muted)"},children:'All agents are revoked. Uncheck "Hide revoked" to view them.'}):c.jsxs(c.Fragment,{children:[c.jsxs("table",{children:[c.jsx("thead",{children:c.jsxs("tr",{children:[c.jsx("th",{children:"Name"}),c.jsx("th",{children:"Type"}),c.jsx("th",{children:"Scopes"}),c.jsx("th",{children:"Status"}),c.jsx("th",{children:"Requests"}),c.jsx("th",{children:"Last Used"})]})}),c.jsx("tbody",{children:nl.map(tl=>c.jsxs("tr",{onClick:()=>I(tl),style:{cursor:"pointer"},children:[c.jsx("td",{style:{fontWeight:500},children:tl.name||tl.client_name}),c.jsx("td",{children:c.jsx("span",{className:`badge ${tl.auth_type==="oauth"?"badge-read":"badge-write"}`,style:{fontSize:11},children:tl.auth_type==="oauth"?"OAuth":"API Key"})}),c.jsx("td",{children:(tl.scope||"").split(" ").filter(Boolean).map(bl=>c.jsx("span",{className:`badge badge-${bl}`,style:{marginRight:4},children:bl},bl))}),c.jsx("td",{children:c.jsx("span",{className:`badge ${tl.status==="active"?"badge-success":"badge-danger"}`,children:tl.status})}),c.jsxs("td",{children:[c.jsx("span",{style:{fontWeight:500},children:tl.requests_today||0}),c.jsxs("span",{style:{color:"var(--text-muted)",fontSize:12},children:[" / ",tl.total_requests||0]})]}),c.jsx("td",{style:{color:"var(--text-secondary)"},children:tl.last_used_at?vy(new Date(tl.last_used_at)):"Never"})]},tl.id))})]}),c.jsxs("div",{style:{color:"var(--text-muted)",fontSize:13,marginTop:12},children:[o.filter(tl=>tl.status==="active").length," active / ",o.length," total"]})]})})(),E&&c.jsx(by,{onClose:()=>N(!1),onRegistered:nl=>{N(!1),Q(nl),L()}}),C&&c.jsx(xy,{credentials:C,onClose:()=>Q(null)}),A&&c.jsx(jy,{agent:A,onClose:()=>I(null),onRevoked:L}),_&&c.jsx(Sy,{onClose:()=>b(!1),onCreated:nl=>{b(!1),M(nl),L()}}),H&&c.jsx(py,{token:H,onClose:()=>M(null)})]})}function Sy({onClose:o,onCreated:D}){const[O,h]=K.useState(""),[E,N]=K.useState(!1),[C,Q]=K.useState(""),_=async b=>{if(b.preventDefault(),!O.trim()){Q("Name required");return}N(!0);try{const H=await Pl.createApiKey(O.trim());D({name:H.name,token:H.token})}catch(H){Q(H instanceof Error?H.message:"Failed")}finally{N(!1)}};return c.jsx("div",{className:"modal-overlay",onClick:o,children:c.jsxs("form",{className:"modal",onClick:b=>b.stopPropagation(),onSubmit:_,children:[c.jsx("div",{className:"modal-title",children:"Create API Key"}),c.jsx("p",{style:{color:"var(--text-secondary)",fontSize:13,marginBottom:16},children:"API keys use simple bearer token auth. They grant full read+write+admin access. For scoped access, use OAuth clients instead."}),c.jsxs("div",{style:{marginBottom:16},children:[c.jsx("label",{children:"Key Name"}),c.jsx("input",{placeholder:"e.g. claude-code-local",value:O,onChange:b=>h(b.target.value),autoFocus:!0})]}),C&&c.jsx("div",{style:{color:"var(--error)",fontSize:13,marginBottom:12},children:C}),c.jsxs("div",{style:{display:"flex",gap:12,justifyContent:"flex-end"},children:[c.jsx("button",{type:"button",className:"btn btn-secondary",onClick:o,children:"Cancel"}),c.jsx("button",{type:"submit",className:"btn btn-primary",disabled:E,children:E?"Creating...":"Create Key"})]})]})})}function py({token:o,onClose:D}){const O=h=>navigator.clipboard.writeText(h);return c.jsx("div",{className:"modal-overlay",children:c.jsxs("div",{className:"modal",style:{maxWidth:560},children:[c.jsxs("div",{style:{textAlign:"center",marginBottom:16},children:[c.jsx("div",{style:{fontSize:36,color:"var(--success)",marginBottom:8},children:"✓"}),c.jsx("div",{style:{fontSize:20,fontWeight:600},children:"API Key Created"})]}),c.jsxs("div",{style:{marginBottom:12},children:[c.jsx("label",{style:{fontSize:12},children:"Name"}),c.jsx("div",{className:"code-block",children:c.jsx("span",{children:o.name})})]}),c.jsxs("div",{style:{marginBottom:12},children:[c.jsx("label",{style:{fontSize:12},children:"Bearer Token"}),c.jsxs("div",{className:"code-block",children:[c.jsx("span",{children:o.token}),c.jsx("button",{className:"copy-btn",onClick:()=>O(o.token),children:"Copy"})]})]}),c.jsxs("div",{style:{marginBottom:12},children:[c.jsx("label",{style:{fontSize:12},children:"Usage"}),c.jsxs("div",{className:"code-block",children:[c.jsx("pre",{style:{whiteSpace:"pre-wrap",margin:0,fontSize:12},children:`Authorization: Bearer ${o.token}`}),c.jsx("button",{className:"copy-btn",onClick:()=>O(`Authorization: Bearer ${o.token}`),children:"Copy"})]})]}),c.jsx("div",{className:"warning-bar",children:"Save this token now. It will not be shown again."}),c.jsx("div",{style:{display:"flex",gap:12,justifyContent:"flex-end",marginTop:20},children:c.jsx("button",{className:"btn btn-primary",onClick:D,children:"Done"})})]})})}function by({onClose:o,onRegistered:D}){const[O,h]=K.useState(""),[E,N]=K.useState(()=>Object.fromEntries(Ed.map(L=>[L,L==="read"]))),[C,Q]=K.useState("86400"),[_,b]=K.useState(!1),[H,M]=K.useState(""),A=[{label:"1 hour",value:"3600"},{label:"24 hours",value:"86400"},{label:"7 days",value:"604800"},{label:"30 days",value:"2592000"},{label:"1 year",value:"31536000"},{label:"No expiry",value:"0"}],I=async L=>{if(L.preventDefault(),!O.trim()){M("Name required");return}b(!0),M("");try{const nl=Object.entries(E).filter(([,Ml])=>Ml).map(([Ml])=>Ml).join(" "),tl=await fetch("/admin/api/register-client",{method:"POST",credentials:"same-origin",headers:{"Content-Type":"application/json"},body:JSON.stringify({name:O.trim(),scopes:nl,tokenTtl:C==="0"?31536e4:Number(C)})});if(!tl.ok)throw new Error("Registration failed");const bl=await tl.json();D({clientId:bl.clientId,clientSecret:bl.clientSecret,name:O.trim()})}catch(nl){M(nl instanceof Error?nl.message:"Registration failed")}finally{b(!1)}};return c.jsx("div",{className:"modal-overlay",onClick:o,children:c.jsxs("form",{className:"modal",onClick:L=>L.stopPropagation(),onSubmit:I,children:[c.jsx("div",{className:"modal-title",children:"Register Agent"}),c.jsxs("div",{style:{marginBottom:16},children:[c.jsx("label",{children:"Agent Name"}),c.jsx("input",{placeholder:"e.g. perplexity-production",value:O,onChange:L=>h(L.target.value),autoFocus:!0})]}),c.jsxs("div",{style:{marginBottom:16},children:[c.jsx("label",{children:"Scopes"}),c.jsx("div",{className:"checkbox-group",children:Ed.map(L=>c.jsxs("label",{className:"checkbox-label",children:[c.jsx("input",{type:"checkbox",checked:E[L],onChange:nl=>N(tl=>({...tl,[L]:nl.target.checked}))}),L]},L))})]}),c.jsxs("div",{style:{marginBottom:20},children:[c.jsx("label",{children:"Token Lifetime"}),c.jsx("select",{value:C,onChange:L=>Q(L.target.value),style:{width:"100%",background:"var(--bg-secondary)",color:"var(--text-primary)",border:"1px solid var(--border)",borderRadius:6,padding:"6px 10px",fontSize:14},children:A.map(L=>c.jsx("option",{value:L.value,children:L.label},L.value))})]}),H&&c.jsx("div",{style:{color:"var(--error)",fontSize:13,marginBottom:12},children:H}),c.jsxs("div",{style:{display:"flex",gap:12,justifyContent:"flex-end"},children:[c.jsx("button",{type:"button",className:"btn btn-secondary",onClick:o,children:"Cancel"}),c.jsx("button",{type:"submit",className:"btn btn-primary",disabled:_,children:_?"Registering...":"Register"})]})]})})}function xy({credentials:o,onClose:D}){const O=E=>navigator.clipboard.writeText(E),h=()=>{const E=new Blob([JSON.stringify(o,null,2)],{type:"application/json"}),N=URL.createObjectURL(E),C=document.createElement("a");C.href=N,C.download=`${o.name}-credentials.json`,C.click(),URL.revokeObjectURL(N)};return c.jsx("div",{className:"modal-overlay",children:c.jsxs("div",{className:"modal",style:{maxWidth:560},children:[c.jsxs("div",{style:{textAlign:"center",marginBottom:16},children:[c.jsx("div",{style:{fontSize:36,color:"var(--success)",marginBottom:8},children:"✓"}),c.jsx("div",{style:{fontSize:20,fontWeight:600},children:"Agent Registered"})]}),c.jsxs("div",{style:{marginBottom:12},children:[c.jsx("label",{style:{fontSize:12},children:"Client ID"}),c.jsxs("div",{className:"code-block",children:[c.jsx("span",{children:o.clientId}),c.jsx("button",{className:"copy-btn",onClick:()=>O(o.clientId),children:"Copy"})]})]}),c.jsxs("div",{style:{marginBottom:12},children:[c.jsx("label",{style:{fontSize:12},children:"Client Secret"}),c.jsxs("div",{className:"code-block",children:[c.jsx("span",{children:o.clientSecret}),c.jsx("button",{className:"copy-btn",onClick:()=>O(o.clientSecret),children:"Copy"})]})]}),c.jsx("div",{className:"warning-bar",children:"Save this secret now. It will not be shown again."}),c.jsxs("div",{style:{display:"flex",gap:12,justifyContent:"flex-end",marginTop:20},children:[c.jsx("button",{className:"btn btn-secondary",onClick:h,children:"Download as JSON"}),c.jsx("button",{className:"btn btn-primary",onClick:D,children:"Done"})]})]})})}function jy({agent:o,onClose:D,onRevoked:O}){const[h,E]=K.useState("claude-code"),N=M=>navigator.clipboard.writeText(M),C=window.location.origin,Q=o.id||o.client_id||"",_=o.auth_type==="oauth",b=o.name||o.client_name||"unknown",H={"claude-code":_?["The user wants to connect GBrain MCP to your context. Here's how.","","# Step 1 — read the client secret from the operator without putting it in shell history.","# read -s suppresses the echo; export keeps it scoped to this shell session.",'read -rs -p "Client Secret: " GBRAIN_CS && echo',"","# Step 2 — mint a token and register GBrain in one go.",`claude mcp add --transport http gbrain ${C}/mcp \\`,` --header "Authorization: Bearer $(curl -s -X POST ${C}/token \\`," -d 'grant_type=client_credentials' \\",` -d 'client_id=${Q}' \\`,' --data-urlencode "client_secret=$GBRAIN_CS" \\',` -d 'scope=${o.scope||"read write"}' | jq -r .access_token)"`,"","# Step 3 — clear the secret from this shell.","unset GBRAIN_CS","",`The token will last ${o.token_ttl?o.token_ttl>=86400?Math.floor(o.token_ttl/86400)+" days":Math.floor(o.token_ttl/3600)+" hours":"1 hour (default)"}.`,"","─── Fallback: 2-step curl + paste ───","","If your shell doesn't support read -s, mint the token first, then paste:","",`curl -s -X POST ${C}/token \\`," -d 'grant_type=client_credentials' \\",` -d 'client_id=${Q}' \\`," -d 'client_secret=PASTE_YOUR_CLIENT_SECRET_HERE' \\",` -d 'scope=${o.scope||"read write"}' | jq -r .access_token`,"",`claude mcp add --transport http gbrain ${C}/mcp \\`,' --header "Authorization: Bearer PASTE_TOKEN_FROM_ABOVE"',"","# Then run: history -d $((HISTCMD-1)) # zsh: print -s -- '' && fc -p"].join(` +`):["The user wants to connect GBrain MCP to your context. Here's how.","","Run this command in terminal to register GBrain:","",`claude mcp add --transport http gbrain ${C}/mcp \\`,' --header "Authorization: Bearer PASTE_YOUR_API_KEY_HERE"',"",`Replace PASTE_YOUR_API_KEY_HERE with the API key shown when "${b}" was created.`,"API keys never expire."].join(` +`),chatgpt:["The user wants to connect GBrain MCP to ChatGPT.","","1. Open ChatGPT > Settings > Tools & Integrations > Add MCP Server","2. Enter this URL — ChatGPT will auto-discover the OAuth configuration:","",` ${C}/.well-known/oauth-authorization-server`,"","3. When prompted for credentials:",` Client ID: ${Q}`," Client Secret: (the secret from agent registration)"," Grant Type: client_credentials",` Scope: ${o.scope||"read write"}`].join(` +`),"claude-cowork":["The user wants to connect GBrain MCP to Claude.ai.","","1. Open claude.ai > Settings > Connected Apps > Add MCP Server",`2. Server URL: ${C}/mcp`,"3. When prompted for auth:",` Token endpoint: ${C}/token`,` Client ID: ${Q}`," Client Secret: (the secret from agent registration)",` Scope: ${o.scope||"read write"}`,"",`Discovery URL: ${C}/.well-known/oauth-authorization-server`].join(` +`),cursor:_?["The user wants to connect GBrain MCP to Cursor.","","Cursor supports OAuth for remote MCP. Add to .cursor/mcp.json:","","{",' "mcpServers": {',' "gbrain": {',` "url": "${C}/mcp",`,' "transport": "sse"'," }"," }","}","","Cursor will auto-discover OAuth via:",`${C}/.well-known/oauth-authorization-server`,"",`When prompted: Client ID ${Q}, use the secret from registration.`].join(` +`):["The user wants to connect GBrain MCP to Cursor.","","Add to .cursor/mcp.json:","","{",' "mcpServers": {',' "gbrain": {',` "url": "${C}/mcp",`,' "transport": "sse",',' "headers": {',' "Authorization": "Bearer PASTE_YOUR_API_KEY_HERE"'," }"," }"," }","}","",`Replace PASTE_YOUR_API_KEY_HERE with the API key shown when "${b}" was created.`].join(` +`),perplexity:["The user wants to connect GBrain MCP to Perplexity.","","1. Go to Settings > Connectors > Add MCP",`2. Server URL: ${C}/mcp`,`3. Client ID: ${Q}`,"4. Client Secret: (the secret from agent registration)"].join(` +`),json:JSON.stringify({server_url:C+"/mcp",token_url:C+"/token",discovery_url:C+"/.well-known/oauth-authorization-server",client_id:Q,client_name:b,auth_type:o.auth_type,scope:o.scope},null,2)};return c.jsxs(c.Fragment,{children:[c.jsx("div",{className:"drawer-overlay",onClick:D}),c.jsxs("div",{className:"drawer",children:[c.jsx("button",{className:"drawer-close",onClick:D,children:"✕"}),c.jsx("div",{style:{fontSize:18,fontWeight:600,marginBottom:4},children:o.name||o.client_name}),c.jsx("span",{className:`badge ${o.status==="active"?"badge-success":"badge-danger"}`,children:o.status}),c.jsx("div",{className:"section-title",children:"Details"}),c.jsxs("div",{style:{display:"grid",gridTemplateColumns:"100px 1fr",gap:"6px 12px",fontSize:13},children:[c.jsx("span",{style:{color:"var(--text-secondary)"},children:"Client ID"}),c.jsxs("span",{className:"mono",children:[(o.id||o.id||o.client_id||"").substring(0,24),"..."]}),c.jsx("span",{style:{color:"var(--text-secondary)"},children:"Scopes"}),c.jsx("span",{children:(o.scope||"").split(" ").filter(Boolean).map(M=>c.jsx("span",{className:`badge badge-${M}`,style:{marginRight:4},children:M},M))}),c.jsx("span",{style:{color:"var(--text-secondary)"},children:"Registered"}),c.jsx("span",{children:new Date(o.created_at).toLocaleDateString()}),c.jsx("span",{style:{color:"var(--text-secondary)"},children:"Token TTL"}),c.jsx("span",{children:o.token_ttl?o.token_ttl>=31536e3?"No expiry":o.token_ttl>=86400?`${Math.floor(o.token_ttl/86400)}d`:o.token_ttl>=3600?`${Math.floor(o.token_ttl/3600)}h`:`${o.token_ttl}s`:"1h (default)"})]}),c.jsx("div",{className:"section-title",children:"Config Export"}),c.jsxs("div",{className:"tabs",style:{flexWrap:"wrap"},children:[c.jsx("div",{className:`tab ${h==="claude-code"?"active":""}`,onClick:()=>E("claude-code"),children:"Claude Code"}),c.jsx("div",{className:`tab ${h==="chatgpt"?"active":""}`,onClick:()=>E("chatgpt"),children:"ChatGPT"}),c.jsx("div",{className:`tab ${h==="claude-cowork"?"active":""}`,onClick:()=>E("claude-cowork"),children:"Claude.ai"}),c.jsx("div",{className:`tab ${h==="cursor"?"active":""}`,onClick:()=>E("cursor"),children:"Cursor"}),c.jsx("div",{className:`tab ${h==="perplexity"?"active":""}`,onClick:()=>E("perplexity"),children:"Perplexity"}),c.jsx("div",{className:`tab ${h==="json"?"active":""}`,onClick:()=>E("json"),children:"JSON"})]}),(()=>{if(!_&&new Set(["chatgpt","claude-cowork","perplexity"]).has(h)){const A={chatgpt:"ChatGPT","claude-cowork":"Claude.ai",perplexity:"Perplexity"}[h]||h;return c.jsxs("div",{style:{background:"rgba(255, 200, 100, 0.08)",border:"1px solid rgba(255, 200, 100, 0.2)",borderRadius:8,padding:"14px 16px",marginTop:12,fontSize:13,lineHeight:1.6,color:"var(--text-secondary)"},children:[c.jsxs("div",{style:{fontWeight:600,color:"var(--text-primary)",marginBottom:6},children:[A," requires an OAuth client"]}),A," only supports OAuth 2.0 (client_credentials). API keys use raw bearer tokens, which ",A," does not accept. Register a separate OAuth client and use that to connect this AI."]})}return c.jsxs("div",{className:"code-block",children:[c.jsx("pre",{style:{whiteSpace:"pre-wrap",margin:0},children:H[h]}),c.jsx("button",{className:"copy-btn",onClick:()=>N(H[h]),children:"Copy"})]})})(),c.jsxs("div",{style:{marginTop:32},children:[o.status==="active"&&c.jsx("button",{className:"btn btn-danger",onClick:async()=>{if(confirm(`Revoke ${o.name||o.client_name}? All active tokens will be invalidated.`))try{o.auth_type==="oauth"?await Pl.revokeClient(o.id||o.client_id||""):await Pl.revokeApiKey(o.name||""),O(),D()}catch(M){alert("Revoke failed: "+(M instanceof Error?M.message:"unknown error"))}},children:"Revoke Agent"}),o.status==="revoked"&&c.jsx("span",{style:{color:"var(--text-muted)",fontSize:13},children:"This agent has been revoked."})]})]})]})}function Ty(){const[o,D]=K.useState({rows:[],total:0,page:1,pages:1}),[O,h]=K.useState(1),[E,N]=K.useState("all"),[C,Q]=K.useState(null);K.useEffect(()=>{_(O)},[O,E]);const _=A=>{const I=E!=="all"?`&agent=${encodeURIComponent(E)}`:"";Pl.requests(A,I).then(D).catch(()=>{})},b=A=>{const I=Date.now()-new Date(A).getTime();return I<6e4?`${Math.floor(I/1e3)}s ago`:I<36e5?`${Math.floor(I/6e4)} min ago`:I<864e5?`${Math.floor(I/36e5)}h ago`:new Date(A).toLocaleDateString()},H=A=>{if(!A)return null;const{query:I,slug:L,partial:nl,limit:tl,...bl}=A,Ml=[];return I&&Ml.push(`"${I}"`),L&&Ml.push(L),nl&&Ml.push(`~${nl}`),tl&&Ml.push(`limit=${tl}`),Object.keys(bl).length>0&&Ml.push(`+${Object.keys(bl).length} params`),Ml.join(" ")},M=new Map;return o.rows.forEach(A=>{A.token_name&&M.set(A.token_name,A.agent_name||A.token_name)}),c.jsxs(c.Fragment,{children:[c.jsxs("div",{style:{display:"flex",justifyContent:"space-between",alignItems:"center",marginBottom:24},children:[c.jsx("h1",{className:"page-title",style:{marginBottom:0},children:"Request Log"}),c.jsxs("select",{value:E,onChange:A=>{N(A.target.value),h(1)},style:{background:"var(--bg-secondary)",color:"var(--text-primary)",border:"1px solid var(--border)",borderRadius:6,padding:"4px 8px",fontSize:13},children:[c.jsx("option",{value:"all",children:"All agents"}),[...M.entries()].map(([A,I])=>c.jsx("option",{value:A,children:I},A))]})]}),o.rows.length===0?c.jsx("div",{style:{textAlign:"center",padding:48,color:"var(--text-muted)"},children:"No requests yet."}):c.jsxs(c.Fragment,{children:[c.jsxs("table",{children:[c.jsx("thead",{children:c.jsxs("tr",{children:[c.jsx("th",{children:"Time"}),c.jsx("th",{children:"Agent"}),c.jsx("th",{children:"Operation"}),c.jsx("th",{children:"Params"}),c.jsx("th",{children:"Latency"}),c.jsx("th",{children:"Status"})]})}),c.jsx("tbody",{children:o.rows.map(A=>c.jsxs(Dd.Fragment,{children:[c.jsxs("tr",{onClick:()=>Q(C===A.id?null:A.id),style:{cursor:"pointer"},children:[c.jsx("td",{style:{color:"var(--text-secondary)",whiteSpace:"nowrap"},children:b(A.created_at)}),c.jsx("td",{children:c.jsx("a",{style:{color:"var(--text-link, #88aaff)",cursor:"pointer",textDecoration:"none",fontWeight:500},onClick:I=>{I.stopPropagation(),N(A.token_name),h(1)},children:A.agent_name||A.token_name})}),c.jsx("td",{className:"mono",children:A.operation}),c.jsx("td",{style:{color:"var(--text-secondary)",fontSize:12,maxWidth:200,overflow:"hidden",textOverflow:"ellipsis",whiteSpace:"nowrap"},children:H(A.params)}),c.jsxs("td",{className:"mono",children:[A.latency_ms,"ms"]}),c.jsx("td",{children:c.jsx("span",{className:`badge badge-${A.status}`,children:A.status})})]}),C===A.id&&c.jsx("tr",{children:c.jsx("td",{colSpan:6,style:{background:"var(--bg-secondary, #0f0f1a)",padding:16},children:c.jsxs("div",{style:{display:"grid",gridTemplateColumns:"100px 1fr",gap:"6px 12px",fontSize:13},children:[c.jsx("span",{style:{color:"var(--text-muted)"},children:"Time"}),c.jsx("span",{children:new Date(A.created_at).toLocaleString()}),c.jsx("span",{style:{color:"var(--text-muted)"},children:"Agent"}),c.jsx("span",{className:"mono",children:A.token_name}),c.jsx("span",{style:{color:"var(--text-muted)"},children:"Operation"}),c.jsx("span",{className:"mono",children:A.operation}),c.jsx("span",{style:{color:"var(--text-muted)"},children:"Latency"}),c.jsxs("span",{children:[A.latency_ms,"ms"]}),A.params&&c.jsxs(c.Fragment,{children:[c.jsx("span",{style:{color:"var(--text-muted)"},children:"Params"}),c.jsx("pre",{className:"mono",style:{margin:0,whiteSpace:"pre-wrap",fontSize:12},children:JSON.stringify(A.params,null,2)})]}),A.error_message&&c.jsxs(c.Fragment,{children:[c.jsx("span",{style:{color:"var(--error, #ff6b6b)"},children:"Error"}),c.jsx("span",{style:{color:"var(--error, #ff6b6b)"},children:A.error_message})]})]})})})]},A.id))})]}),c.jsxs("div",{className:"pagination",children:[c.jsxs("span",{children:["Page ",o.page," of ",o.pages," (",o.total," total)"]}),c.jsxs("div",{style:{display:"flex",gap:8},children:[c.jsx("button",{disabled:o.page<=1,onClick:()=>h(A=>A-1),children:"Previous"}),c.jsx("button",{disabled:o.page>=o.pages,onClick:()=>h(A=>A+1),children:"Next"})]})]})]})]})}function zy({markup:o}){return c.jsx("div",{style:{width:"100%",overflow:"auto"},dangerouslySetInnerHTML:{__html:o}})}function Zu({type:o,ariaLabel:D}){const[O,h]=K.useState(""),[E,N]=K.useState("");return K.useEffect(()=>{let C=!1;return Pl.calibrationChart(o).then(Q=>{C||h(Q)}).catch(Q=>{C||N(Q.message??"fetch failed")}),()=>{C=!0}},[o]),E?c.jsxs("div",{style:{padding:16,color:"var(--error)"},role:"alert",children:[D,": ",E]}):O?c.jsx(zy,{markup:O}):c.jsxs("div",{style:{padding:16,color:"var(--text-muted)"},children:[D," loading..."]})}function Ay(){const[o,D]=K.useState(null),[O,h]=K.useState(!0),[E,N]=K.useState("");if(K.useEffect(()=>{Pl.calibrationProfile().then(_=>{D(_),h(!1)}).catch(_=>{N(_.message??"fetch failed"),h(!1)})},[]),O)return c.jsx("div",{style:{padding:24,color:"var(--text-secondary)"},children:"Loading calibration profile…"});if(E)return c.jsxs("div",{style:{padding:24,color:"var(--error)"},role:"alert",children:["Could not load calibration profile: ",E]});if(!o)return c.jsxs("div",{style:{padding:24,maxWidth:700},children:[c.jsx("h1",{style:{marginBottom:16},children:"Calibration"}),c.jsx("p",{style:{color:"var(--text-secondary)"},children:"No calibration profile yet. Builds after 5+ resolved takes."}),c.jsx("pre",{style:{background:"var(--bg-secondary)",padding:12,borderRadius:4,color:"var(--text-primary)",marginTop:12,fontFamily:"var(--font-mono)"},children:"gbrain dream --phase calibration_profile"})]});const C=new Date(o.generated_at),Q=Math.floor((Date.now()-C.getTime())/(1e3*60*60*24));return c.jsxs("div",{style:{padding:32,maxWidth:720},children:[c.jsx("h1",{style:{marginBottom:8},children:"Calibration"}),c.jsxs("div",{style:{color:"var(--text-muted)",fontSize:13,marginBottom:24},children:["Holder: ",o.holder," · ","Updated ",Q===0?"today":`${Q}d ago`,o.published&&" · published",o.grade_completion<.9&&` · ~${Math.round(o.grade_completion*100)}% graded`,!o.voice_gate_passed&&" · voice gate fell back to template"]}),c.jsx("section",{style:{marginBottom:32},children:c.jsx(Zu,{type:"brier-trend",ariaLabel:"Brier trend"})}),c.jsxs("section",{style:{marginBottom:32},children:[c.jsx("h2",{style:{fontSize:14,color:"var(--text-secondary)",marginBottom:12,fontWeight:400},children:"Pattern statements"}),c.jsx(Zu,{type:"pattern-statements",ariaLabel:"Pattern statements"})]}),c.jsx("section",{style:{marginBottom:32},children:c.jsx(Zu,{type:"domain-bars",ariaLabel:"Per-domain accuracy"})}),c.jsx("section",{style:{marginBottom:32},children:c.jsx(Zu,{type:"abandoned-threads",ariaLabel:"Abandoned threads"})}),o.active_bias_tags.length>0&&c.jsxs("section",{style:{marginBottom:32,color:"var(--text-muted)",fontSize:13},children:["Active bias tags: ",o.active_bias_tags.join(", ")]})]})}function _y(o){return o===0?"var(--accent-success, #2ea043)":o>=100?"var(--accent-danger, #f85149)":"var(--accent-warn, #d29922)"}function Od(o){return`$${(o/100).toFixed(2)}`}function Ey(){const[o,D]=K.useState(null),[O,h]=K.useState(null);if(K.useEffect(()=>{let N=!0,C=null;const Q=async()=>{try{const _=await Pl.jobsWatch();N&&(D(_),h(null))}catch(_){N&&h(_ instanceof Error?_.message:String(_))}N&&(C=setTimeout(Q,1e3))};return Q(),()=>{N=!1,C&&clearTimeout(C)}},[]),O)return c.jsxs("div",{style:{padding:24,color:"var(--accent-danger, #f85149)"},children:[c.jsx("h2",{children:"Jobs Watch — error"}),c.jsx("pre",{style:{whiteSpace:"pre-wrap"},children:O})]});if(!o)return c.jsx("div",{style:{padding:24,color:"var(--text-muted, #777)"},children:"Loading jobs watch…"});const E=new Date(o.ts_ms).toLocaleTimeString();return c.jsxs("div",{style:{padding:24,fontFamily:'var(--font-mono, "JetBrains Mono", monospace)'},children:[c.jsxs("h1",{style:{fontSize:18,marginBottom:4},children:["Jobs Watch",c.jsxs("span",{style:{marginLeft:12,color:"var(--text-muted, #777)",fontSize:12,fontWeight:"normal"},children:["updated ",E]})]}),c.jsxs("section",{style:{marginTop:24},children:[c.jsx("h2",{style:{fontSize:14,marginBottom:8},children:"Queue"}),c.jsxs("div",{children:["waiting=",c.jsx("b",{children:o.queue_health.waiting})," ","active=",c.jsx("b",{children:o.queue_health.active})," ","stalled=",c.jsx("b",{style:{color:o.queue_health.stalled>0?"var(--accent-warn, #d29922)":void 0},children:o.queue_health.stalled})]})]}),o.by_type.length>0&&c.jsxs("section",{style:{marginTop:24},children:[c.jsx("h2",{style:{fontSize:14,marginBottom:8},children:"By type (24h)"}),c.jsxs("table",{style:{borderCollapse:"collapse"},children:[c.jsx("thead",{children:c.jsxs("tr",{style:{color:"var(--text-muted, #777)",fontSize:12},children:[c.jsx("th",{style:{textAlign:"left",padding:"4px 12px 4px 0"},children:"name"}),c.jsx("th",{style:{textAlign:"right",padding:"4px 12px"},children:"total"}),c.jsx("th",{style:{textAlign:"right",padding:"4px 12px"},children:"done"}),c.jsx("th",{style:{textAlign:"right",padding:"4px 12px"},children:"fail"}),c.jsx("th",{style:{textAlign:"right",padding:"4px 12px"},children:"dead"})]})}),c.jsx("tbody",{children:o.by_type.slice(0,6).map(N=>c.jsxs("tr",{children:[c.jsx("td",{style:{padding:"4px 12px 4px 0"},children:N.name}),c.jsx("td",{style:{textAlign:"right",padding:"4px 12px"},children:N.total}),c.jsx("td",{style:{textAlign:"right",padding:"4px 12px"},children:N.completed}),c.jsx("td",{style:{textAlign:"right",padding:"4px 12px"},children:N.failed}),c.jsx("td",{style:{textAlign:"right",padding:"4px 12px"},children:N.dead})]},N.name))})]})]}),c.jsxs("section",{style:{marginTop:24},children:[c.jsx("h2",{style:{fontSize:14,marginBottom:8},children:"Lease pressure (1h)"}),c.jsxs("div",{style:{color:_y(o.lease_pressure_1h)},children:[o.lease_pressure_1h," bounce",o.lease_pressure_1h===1?"":"s"]})]}),o.top_errors.length>0&&c.jsxs("section",{style:{marginTop:24},children:[c.jsx("h2",{style:{fontSize:14,marginBottom:8},children:"Top errors (24h)"}),c.jsx("table",{style:{borderCollapse:"collapse"},children:c.jsx("tbody",{children:o.top_errors.slice(0,5).map(N=>c.jsxs("tr",{children:[c.jsxs("td",{style:{textAlign:"right",padding:"4px 12px 4px 0",color:"var(--text-muted, #777)"},children:[N.count,"×"]}),c.jsx("td",{style:{padding:"4px 12px 4px 0"},children:N.cluster})]},N.cluster))})})]}),o.budget_owners.length>0&&c.jsxs("section",{style:{marginTop:24},children:[c.jsx("h2",{style:{fontSize:14,marginBottom:8},children:"Budget owners"}),c.jsxs("table",{style:{borderCollapse:"collapse"},children:[c.jsx("thead",{children:c.jsxs("tr",{style:{color:"var(--text-muted, #777)",fontSize:12},children:[c.jsx("th",{style:{textAlign:"left",padding:"4px 12px 4px 0"},children:"owner"}),c.jsx("th",{style:{textAlign:"right",padding:"4px 12px"},children:"spent"}),c.jsx("th",{style:{textAlign:"right",padding:"4px 12px"},children:"remaining"})]})}),c.jsx("tbody",{children:o.budget_owners.slice(0,5).map(N=>c.jsxs("tr",{children:[c.jsx("td",{style:{padding:"4px 12px 4px 0"},children:N.owner_id}),c.jsx("td",{style:{textAlign:"right",padding:"4px 12px"},children:Od(N.total_spent_cents)}),c.jsx("td",{style:{textAlign:"right",padding:"4px 12px"},children:Od(N.remaining_cents)})]},N.owner_id))})]})]})]})}function Nd(){const o=window.location.hash.replace("#","")||"dashboard";return["login","dashboard","agents","log","calibration","jobs"].includes(o)?o:"dashboard"}function Oy(){const[o,D]=K.useState(Nd);K.useEffect(()=>{const E=()=>D(Nd());return window.addEventListener("hashchange",E),()=>window.removeEventListener("hashchange",E)},[]);const O=E=>{window.location.hash=E,D(E)};if(o==="login")return c.jsx(my,{onLogin:()=>O("dashboard")});const h=async()=>{if(confirm("Sign out every active admin session, including other browsers and tabs? Each one will need to re-authenticate via a fresh magic link.")){try{await Pl.signOutEverywhere()}catch{}O("login")}};return c.jsxs("div",{className:"app",children:[c.jsxs("nav",{className:"sidebar",children:[c.jsx("div",{className:"sidebar-logo",children:"GBrain"}),c.jsxs("div",{className:"sidebar-nav",children:[c.jsx("a",{className:`nav-item ${o==="dashboard"?"active":""}`,onClick:()=>O("dashboard"),children:"Dashboard"}),c.jsx("a",{className:`nav-item ${o==="agents"?"active":""}`,onClick:()=>O("agents"),children:"Agents"}),c.jsx("a",{className:`nav-item ${o==="log"?"active":""}`,onClick:()=>O("log"),children:"Request Log"}),c.jsx("a",{className:`nav-item ${o==="calibration"?"active":""}`,onClick:()=>O("calibration"),children:"Calibration"}),c.jsx("a",{className:`nav-item ${o==="jobs"?"active":""}`,onClick:()=>O("jobs"),children:"Jobs Watch"})]}),c.jsx("div",{style:{marginTop:"auto",padding:"16px 12px",borderTop:"1px solid var(--border)"},children:c.jsx("button",{onClick:h,style:{background:"transparent",border:"1px solid var(--border)",color:"var(--text-secondary)",padding:"6px 10px",borderRadius:6,fontSize:12,cursor:"pointer",width:"100%"},title:"Revoke every active admin session — every browser, every tab",children:"Sign out everywhere"})})]}),c.jsxs("main",{className:"main",children:[o==="dashboard"&&c.jsx(yy,{}),o==="agents"&&c.jsx(gy,{}),o==="log"&&c.jsx(Ty,{}),o==="calibration"&&c.jsx(Ay,{}),o==="jobs"&&c.jsx(Ey,{})]})]})}dy.createRoot(document.getElementById("root")).render(c.jsx(Dd.StrictMode,{children:c.jsx(Oy,{})})); diff --git a/admin/dist/index.html b/admin/dist/index.html index e6eadff6f..165106e55 100644 --- a/admin/dist/index.html +++ b/admin/dist/index.html @@ -7,7 +7,7 @@ - + diff --git a/admin/src/App.tsx b/admin/src/App.tsx index af80181a1..fd6c3ed90 100644 --- a/admin/src/App.tsx +++ b/admin/src/App.tsx @@ -4,13 +4,14 @@ import { DashboardPage } from './pages/Dashboard'; import { AgentsPage } from './pages/Agents'; import { RequestLogPage } from './pages/RequestLog'; import { CalibrationPage } from './pages/Calibration'; +import { JobsWatchPage } from './pages/JobsWatch'; import { api } from './api'; -type Page = 'login' | 'dashboard' | 'agents' | 'log' | 'calibration'; +type Page = 'login' | 'dashboard' | 'agents' | 'log' | 'calibration' | 'jobs'; function getPage(): Page { const hash = window.location.hash.replace('#', '') || 'dashboard'; - if (['login', 'dashboard', 'agents', 'log', 'calibration'].includes(hash)) return hash as Page; + if (['login', 'dashboard', 'agents', 'log', 'calibration', 'jobs'].includes(hash)) return hash as Page; return 'dashboard'; } @@ -57,6 +58,8 @@ export function App() { onClick={() => navigate('log')}>Request Log navigate('calibration')}>Calibration + navigate('jobs')}>Jobs Watch
); diff --git a/admin/src/api.ts b/admin/src/api.ts index 2308b161c..9cf24a9ee 100644 --- a/admin/src/api.ts +++ b/admin/src/api.ts @@ -50,4 +50,6 @@ export const api = { apiFetch(`/admin/api/calibration/profile${holder ? `?holder=${encodeURIComponent(holder)}` : ''}`), calibrationChart: (type: string, holder?: string) => apiFetchText(`/admin/api/calibration/charts/${encodeURIComponent(type)}${holder ? `?holder=${encodeURIComponent(holder)}` : ''}`), + // v0.41 D2 — live minion-jobs dashboard snapshot. + jobsWatch: () => apiFetch('/admin/api/jobs/watch'), }; diff --git a/admin/src/pages/JobsWatch.tsx b/admin/src/pages/JobsWatch.tsx new file mode 100644 index 000000000..759f291c6 --- /dev/null +++ b/admin/src/pages/JobsWatch.tsx @@ -0,0 +1,174 @@ +import React, { useEffect, useState } from 'react'; +import { api } from '../api'; + +/** + * v0.41 D2 — live jobs dashboard. Browser counterpart to the TTY + * `gbrain jobs watch` command. Polls `/admin/api/jobs/watch` every + * 1s (matches TTY refresh cadence; SSE upgrade is a v0.42 follow-up + * once the same wiring lands in serve-http for the TTY command). + * + * Layout intentionally matches the TTY 1:1 so an operator looking at + * both surfaces sees the same panels in the same order. + */ + +interface WatchSnapshot { + ts_ms: number; + by_type: Array<{ name: string; total: number; completed: number; failed: number; dead: number }>; + queue_health: { waiting: number; active: number; stalled: number }; + lease_pressure_1h: number; + top_errors: Array<{ cluster: string; count: number }>; + budget_owners: Array<{ owner_id: number; remaining_cents: number; total_spent_cents: number }>; +} + +function leasePressureColor(n: number): string { + if (n === 0) return 'var(--accent-success, #2ea043)'; + if (n >= 100) return 'var(--accent-danger, #f85149)'; + return 'var(--accent-warn, #d29922)'; +} + +function dollars(cents: number): string { + return `$${(cents / 100).toFixed(2)}`; +} + +export function JobsWatchPage() { + const [snap, setSnap] = useState(null); + const [err, setErr] = useState(null); + + useEffect(() => { + let alive = true; + let timer: ReturnType | null = null; + + const tick = async () => { + try { + const data = await api.jobsWatch(); + if (alive) { + setSnap(data); + setErr(null); + } + } catch (e) { + if (alive) setErr(e instanceof Error ? e.message : String(e)); + } + if (alive) timer = setTimeout(tick, 1000); + }; + + tick(); + return () => { + alive = false; + if (timer) clearTimeout(timer); + }; + }, []); + + if (err) { + return ( +
+

Jobs Watch — error

+ 
{err}
+
+ ); + } + + if (!snap) { + return
Loading jobs watch…
; + } + + const ts = new Date(snap.ts_ms).toLocaleTimeString(); + + return ( +
+

+ Jobs Watch + + updated {ts} + +

+ +
+

Queue

+
+ waiting={snap.queue_health.waiting}{' '} + active={snap.queue_health.active}{' '} + stalled= 0 ? 'var(--accent-warn, #d29922)' : undefined }}> + {snap.queue_health.stalled} + +
+
+ + {snap.by_type.length > 0 && ( +
+

By type (24h)

+ + + + + + + + + + + + {snap.by_type.slice(0, 6).map(t => ( + + + + + + + + ))} + +
nametotaldonefaildead
{t.name}{t.total}{t.completed}{t.failed}{t.dead}
+
+ )} + +
+

Lease pressure (1h)

+
+ {snap.lease_pressure_1h} bounce{snap.lease_pressure_1h === 1 ? '' : 's'} +
+
+ + {snap.top_errors.length > 0 && ( +
+

Top errors (24h)

+ + + {snap.top_errors.slice(0, 5).map(e => ( + + + + + ))} + +
+ {e.count}× + {e.cluster}
+
+ )} + + {snap.budget_owners.length > 0 && ( +
+

Budget owners

+ + + + + + + + + + {snap.budget_owners.slice(0, 5).map(b => ( + + + + + + ))} + +
ownerspentremaining
{b.owner_id}{dollars(b.total_spent_cents)}{dollars(b.remaining_cents)}
+
+ )} +
+ ); +} diff --git a/llms-full.txt b/llms-full.txt index 44a2cf9cc..f68b11dcb 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -215,9 +215,9 @@ strict behavior when unset. - `gbrain.yml` (brain repo root, v0.22.11) — Optional storage tiering config. Top-level `storage:` section with `db_tracked:` and `db_only:` array-valued keys. `gbrain sync` auto-manages `.gitignore` for `db_only` paths on successful sync (skips on dry-run, blocked-by-failures, submodule context, or `GBRAIN_NO_GITIGNORE=1`). `gbrain export --restore-only [--repo P] [--type T] [--slug-prefix S]` repopulates missing `db_only` files from the database. - `src/core/supabase-admin.ts` — Supabase admin API (project discovery, pgvector check) - `src/core/file-resolver.ts` — File resolution with fallback chain (local -> .redirect.yaml -> .redirect -> .supabase) -- `src/core/chunkers/` — 3-tier chunking (recursive, semantic, LLM-guided). v0.19.0 adds `code.ts` — tree-sitter-based semantic chunker for 29 languages with embedded-asset WASMs (`src/assets/wasm/`), `@dqbd/tiktoken` cl100k_base tokenizer, small-sibling merging. `CHUNKER_VERSION` constant folded into `importCodeFile`'s `content_hash` so chunker shape changes force clean re-chunks across releases. +- `src/core/chunkers/` — 3-tier chunking (recursive, semantic, LLM-guided). v0.19.0 adds `code.ts` — tree-sitter-based semantic chunker for 30 languages (v0.41 added SQL via DerekStride/tree-sitter-sql) with embedded-asset WASMs (`src/assets/wasm/`), `@dqbd/tiktoken` cl100k_base tokenizer, small-sibling merging. `CHUNKER_VERSION` constant folded into `importCodeFile`'s `content_hash` so chunker shape changes force clean re-chunks across releases. **v0.41 D2 SQL wave (#1173):** `extractSymbolName` gains an inline SQL branch (`extractSqlSymbolName`) that dives through DerekStride's `statement` wrapper into the inner DDL child (`create_table`/`create_function`/`create_view`/`create_index`/`create_procedure`/`create_type`/`create_schema`/`create_database`/`create_trigger`/`alter_table`/`alter_view`) and extracts the target identifier via the `name` field with identifier-shaped fallback. DML kinds (`select`/`insert`/`update`/`delete`/`merge`/`with`) deliberately return null so chunks emit unnamed — code-def is a DDL signal. `normalizeSymbolType` gains parallel SQL branches mapping `create_table → 'table'`, `create_view → 'view'`, etc. `src/commands/code-def.ts:DEF_TYPES` was extended in the same wave with `'table' | 'view' | 'index' | 'procedure' | 'schema' | 'database' | 'trigger'` so the new chunks surface in `gbrain code-def ` queries. - `src/core/errors.ts` (v0.19.0) — `StructuredAgentError` + `buildError` + `serializeError`. Every new v0.19.0 agent-facing surface (code-def, code-refs, usage errors) uses this envelope; matches v0.17.0 `CycleReport.PhaseResult.error` shape. -- `src/assets/wasm/` (v0.19.0) — 36 tree-sitter grammar WASMs + tree-sitter runtime. Committed to the repo so `bun --compile` embeds them deterministically via `import path from ... with { type: 'file' }`. The CI guard `scripts/check-wasm-embedded.sh` fails the build if the compiled binary ever silently falls through to recursive chunks. +- `src/assets/wasm/` (v0.19.0, extended v0.41) — 37 tree-sitter grammar WASMs + tree-sitter runtime. Committed to the repo so `bun --compile` embeds them deterministically via `import path from ... with { type: 'file' }`. The CI guard `scripts/check-wasm-embedded.sh` fails the build if the compiled binary ever silently falls through to recursive chunks. **v0.41 D2 wave (#1173):** `tree-sitter-sql.wasm` (DerekStride/tree-sitter-sql @ c2e1e08db1ea20dc23bdb8d228a81a8756e9c450, built with tree-sitter-cli@v0.26.3 --abi 14) adds SQL coverage at 11 MB — substantially larger than peers because the grammar covers PostgreSQL + MySQL + SQLite + T-SQL basics (40 MB of generated parser.c). The compiled gbrain binary grows ~6% as a result; tracked as a follow-up TODO whether to switch to a smaller-coverage grammar. - `src/commands/code-def.ts` + `src/commands/code-refs.ts` (v0.19.0) — symbol definition + references lookup. Query `content_chunks.symbol_name` or chunk_text ILIKE with `page_kind='code'` filter. Auto-JSON when stdout is not a TTY (gh-CLI convention). Bypass the standard `searchKeyword` `DISTINCT ON (slug)` collapse so multiple call-sites from the same file surface. - `src/core/search/` — Hybrid search: vector + keyword + RRF + multi-query expansion + dedup. As of v0.22.0, `searchKeyword` / `searchKeywordChunks` / `searchVector` apply source-aware ranking at the SQL layer (curated content like `originals/`, `concepts/`, `writing/` outranks bulk content like `wintermute/chat/`, `daily/`, `media/x/`). `searchVector` uses a two-stage CTE so source-boost re-ranking doesn't kill the HNSW index. Hard-exclude prefixes (`test/`, `archive/`, `attachments/`, `.raw/` by default) filter at retrieval, not post-rank. Both gates honor `detail !== 'high'` so temporal queries surface chat pages normally. - `src/core/search/intent.ts` — Query intent classifier (entity/temporal/event/general → auto-selects detail level) @@ -301,7 +301,7 @@ strict behavior when unset. - `src/commands/extract.ts` — `gbrain extract links|timeline|all [--source fs|db] [--source-id ]`: batch link/timeline extraction. fs walks markdown files, db walks pages from the engine (mutation-immune snapshot iteration; use this for live brains with no local checkout). As of v0.12.1 there is no in-memory dedup pre-load — candidates are buffered 100 at a time and flushed via `addLinksBatch` / `addTimelineEntriesBatch`; `ON CONFLICT DO NOTHING` enforces uniqueness at the DB layer, and the `created` counter returns real rows inserted (truthful on re-runs). v0.22.1 (#417): `ExtractOpts.slugs?: string[]` enables incremental extract — when set, `extractForSlugs()` reads ONLY those slugs' files (single combined links+timeline pass) instead of the full directory walk. CLI `gbrain extract` keeps full-walk behavior; the cycle path threads sync's `pagesAffected` through. `walkMarkdownFiles(brainDir)` still runs at line 455 to build `allSlugs` for link resolution — see `TODOS.md` for replacing it with `engine.getAllSlugs()`. **v0.37.7.0 (#1204):** `--source-id ` flag scopes extraction to one brain source on federated brains. Resolved via `resolveSourceWithTier()` before any SQL runs; failures surface with a `gbrain sources list` hint. Closes the silent-collapse-to-`default` bug class for extract. - `src/commands/import.ts` — `gbrain import [--source-id ]`: page import with the v0.34.2.0 path-set checkpoint described above. **v0.37.7.0 (#1167):** new `--source-id ` flag finally honored — pages route to the named source. Resolved via `resolveSourceWithTier()` at the boundary; the same flag is now consistent across `import`, `extract`, `graph-query`, and `sources current`. Pinned by `test/import-source-id.test.ts`. - `src/commands/graph-query.ts` — `gbrain graph-query [--type T] [--depth N] [--direction in|out|both] [--include-foreign]`: typed-edge relationship traversal (renders indented tree). **v0.37.7.0 (#1153):** foreign-edge footer always present (`X foreign edges (use --include-foreign to traverse)`) so cross-source edges never disappear silently; `--include-foreign` widens the SQL filter to walk them. Pinned by `test/graph-query.test.ts`. -- `src/commands/sources.ts` — `gbrain sources {list,add,remove,archive,restore,archived,purge,current,status}`. **v0.37.7.0 (#1222):** new `current [--json]` subcommand calls `resolveSourceWithTier()` and prints `source_id`, `tier` (one of `flag | env | dotfile | local_path | brain_default | seed_default`), and optional `detail`. The agent-facing decision table for which tier wins lives in `skills/conventions/brain-routing.md`. **v0.40.3.0 (productionized from PR #1314):** new `status [--json]` subcommand — read-only per-source dashboard (last sync, staleness, page count, embedding coverage, unacked failures). Thin wrapper around `buildSyncStatusReport` + `printSyncStatusReport` exported from `src/commands/sync.ts`. `--json` emits the stable `{schema_version: 1, sources, ...}` envelope on stdout for monitoring pipelines; bare invocation prints the human table to stdout (right-aligned numeric columns, kubectl-style). Filters input sources to `local_path IS NOT NULL AND archived IS NOT TRUE` so archived sources (which have their own `gbrain sources archived` surface) don't muddy the active-sync dashboard. Lives under `sources` (not `sync --status`) per D3 from the v0.40.3.0 plan-eng-review — reads and writes don't share a verb. +- `src/commands/sources.ts` — `gbrain sources {list,add,remove,archive,restore,archived,purge,current,status,audit}`. **v0.37.7.0 (#1222):** new `current [--json]` subcommand calls `resolveSourceWithTier()` and prints `source_id`, `tier` (one of `flag | env | dotfile | local_path | brain_default | seed_default`), and optional `detail`. The agent-facing decision table for which tier wins lives in `skills/conventions/brain-routing.md`. **v0.40.3.0 (productionized from PR #1314):** new `status [--json]` subcommand — read-only per-source dashboard (last sync, staleness, page count, embedding coverage, unacked failures). Thin wrapper around `buildSyncStatusReport` + `printSyncStatusReport` exported from `src/commands/sync.ts`. `--json` emits the stable `{schema_version: 1, sources, ...}` envelope on stdout for monitoring pipelines; bare invocation prints the human table to stdout (right-aligned numeric columns, kubectl-style). Filters input sources to `local_path IS NOT NULL AND archived IS NOT TRUE` so archived sources (which have their own `gbrain sources archived` surface) don't muddy the active-sync dashboard. Lives under `sources` (not `sync --status`) per D3 from the v0.40.3.0 plan-eng-review — reads and writes don't share a verb. **v0.40.9.0:** new `audit [--json]` subcommand — read-only dry-run scan of a source repo's disk for size distribution + would-blocks + junk-pattern hits, WITHOUT touching the DB. Catches scraper junk and oversized content BEFORE sync. Walks `sources.local_path`, reads each markdown file, runs `assessContent()` from `src/core/content-sanity.ts`, aggregates by verdict (`ok | warn_oversize | hard_block_junk_pattern`). JSON envelope is stable for monitoring pipelines. Pinned by `test/sources-audit.test.ts` (not present in this wave; covered transitively by `test/content-sanity.test.ts` + `test/import-file-content-sanity.test.ts`). - `src/commands/reindex-frontmatter.ts` — `gbrain reindex-frontmatter`. **v0.37.7.0 (#1225):** wrapped the query path in the standard `withEngine(...)` lifecycle so `engine.connect()` runs before the first SQL call. Pre-fix the command `process.exit(1)`'d with a TypeError on first invocation. Pinned by `test/reindex-frontmatter-connect.test.ts`. - `src/core/source-resolver.ts` — 6-tier source resolution. **v0.37.7.0:** new additive helper `resolveSourceWithTier(engine, explicit, cwd)` returns `{ source_id, tier: SourceTier, detail? }` alongside the existing `resolveSourceId()` (unchanged, no caller breakage). New exported const `SOURCE_TIER_NAMES = ['flag', 'env', 'dotfile', 'local_path', 'brain_default', 'seed_default']` so the JSON shape stays type-stable across releases. Order matches the 1-6 priority of `resolveSourceId()`. Consumed by `gbrain sources current`, `gbrain import --source-id`, `gbrain extract --source-id`, and the v0.37.7.0 `source_routing_health` doctor check. Pinned by `test/source-resolver-with-tier.test.ts` (uses `withEnv()` wrapper per the test-isolation lint). - `src/commands/autopilot.ts` extension (v0.37.7.0) — three changes for federated-brain co-existence and launchd hygiene. (1) **#1226 lockfile scope:** `LOCK_PATH` resolves via `gbrainPath('autopilot.lock')` so it honors `GBRAIN_HOME`. Two brains can run autopilot simultaneously without lock-stealing. Lock file now stores PID; startup checks `kill -0 ` before refusing to start (codex CF11 PID-safety fix — stale lock from a crashed process no longer blocks a healthy autopilot). (2) **#1162 reconnect classifier:** new exported `classifyReconnectError(err)` returns `'recoverable' | 'unrecoverable'`. Unrecoverable causes the daemon to `process.exit(0)` and let launchd back off instead of the v0.37.6 loop that logged `config.database_url undefined` every 5s forever. (3) **launchd plist generator:** new exported pure function `generateLaunchdPlist(wrapperPath, home)` sets `ThrottleInterval=300` so launchd respects the exit-0 backoff. Both helpers pinned by `test/autopilot-lock-path.test.ts` + `test/autopilot-reconnect-classifier.test.ts`. @@ -313,7 +313,14 @@ strict behavior when unset. - `src/commands/doctor.ts` extension (v0.40.4.1) — `buildChecks(engine, args, dbSource): Promise` exported as a test seam. `runDoctor` is now a thin wrapper: `buildChecks → computeDoctorReport → render + process.exit`. All 10 `process.exit` sites stay in the wrapper; the two early-return paths (no engine, connection failure) return partial check lists instead of inline exits. No behavior change — observable output identical because the wrapper renders the same partial list. Pinned by `test/doctor-behavioral.test.ts` (13 cases: pure aggregation math over `computeDoctorReport`, orchestrator cases for `--fast` skip set + `--json` flag + no-engine partial path + snapshot of load-bearing check names) and `test/doctor-cli-smoke.serial.test.ts` (1 subprocess case spawning `bun run src/cli.ts doctor --json` against a fresh PGLite tempdir, asserting schema_version=2 envelope, status enum, non-empty checks array — the render-path coverage that buildChecks-only tests miss). Quarantined as `.serial.test.ts` because PGLite write-locks don't play with parallel runners. - `src/core/cycle.ts` extension (v0.40.4.1) — `runPhaseLint` + `runPhaseBacklinks` gain the `export` keyword so behavioral tests can drive them directly. No body changes; documented as internal helpers exposed for test-only consumption (downstream code should NOT take a dependency). Pinned by `test/cycle-legacy-phases.test.ts` (11 cases across both phases: clean run → status='ok', partial fix → status='warn' with `dryRun` in details, dry-run path doesn't write, throw-from-lib → status='fail' with the wrapper's try/catch envelope populated). Future phase wrappers (sync, extract, embed, orphans, extract_facts, resolve_symbol_edges, recompute_emotional_weight) land as additional describes in the same file, not new files (TODOS NEW-2). - `test/operations-trust-boundary.test.ts` + `scripts/check-operations-filter-bypass.sh` (v0.40.4.1) — operations trust-boundary contract coverage. Hybrid test design: pure assertions over all 74 ops (every op has a scope annotation; every mutating op has a non-read scope; `localOnly: true` ops are excluded from `operations.filter(op => !op.localOnly)`; the seven historically-sensitive localOnly ops snapshot-pinned by name) plus targeted handler-invocation regressions for the two historically-broken HTTP-callable classes: `submit_job` with `name='shell'` + `ctx.remote=true` MUST reject (the F7b HTTP MCP shell-job RCE class), and `search_by_image` with `image_path` + `ctx.remote=true` MUST reject (the D18 P0 image-leak class). `file_upload` and `sync_brain` deliberately omitted from handler-invocation tests because they're `localOnly: true` and that path would test an impossible production scenario (codex CMT-3). The shell guard greps `src/` for any module importing the `operations` value outside the canonical filter site at `src/commands/serve-http.ts` — three import shapes detected (destructured, aliased, namespace), explicit 10-entry allow-list with per-entry rationale, plus a literal-string check that `serve-http.ts` still contains `operations.filter(op => !op.localOnly)`. Wired into `bun run verify`. -- `src/core/link-extraction.ts` — shared library for the v0.12.0 graph layer. extractEntityRefs (canonical, replaces backlinks.ts duplicate) matches both `[Name](people/slug)` markdown links and Obsidian `[[people/slug|Name]]` wikilinks as of v0.12.3. extractPageLinks, inferLinkType heuristics (attended/works_at/invested_in/founded/advises/source/mentions), parseTimelineEntries, isAutoLinkEnabled config helper. `DIR_PATTERN` covers `people`, `companies`, `deals`, `topics`, `concepts`, `projects`, `entities`, `tech`, `finance`, `personal`, `openclaw`. Used by extract.ts, operations.ts auto-link post-hook, and backlinks.ts. +- `src/core/content-sanity.ts` (v0.40.9.0, NEW) — pure assessor for the content-sanity defense wave. `assessContent(content, opts): SanityVerdict` returns one of three verdicts (`ok | warn_oversize | hard_block_junk_pattern | soft_block_oversize`) with `{reason, bytes, matched_pattern_name?}` detail. Six hand-vetted built-in junk patterns (Cloudflare challenge dumps, CAPTCHAs, 403 dumps, bare error-page titles) compiled at module load; operator literal substrings loaded via `loadOperatorLiterals()` from `src/core/content-sanity-literals.ts`. `ContentSanityBlockError` tagged class is the typed throw shape — every wrapper site (`gbrain import` CLI, `put_page` MCP op, `gbrain sync`, `/ingest` webhook) catches it via the existing exception flow rather than a parallel status check. The bytes-parity contract (D2) pins `Buffer.byteLength(content, 'utf8')` against the embedder's actual byte count so a 499K-byte page can't be soft-blocked on assessment then overflow on embed. Knob resolution chain: env > file (`~/.gbrain/config.json`) > DB > defaults — env wins for CI / one-off overrides, file is operator-set, DB plane is what `gbrain config set` writes. Four knobs: `content_sanity.bytes_warn` (default 50_000), `content_sanity.bytes_block` (default 500_000), `content_sanity.junk_patterns_enabled` (default true), `content_sanity.disabled` (default false; `GBRAIN_NO_SANITY=1` is the loud-stderr kill-switch with per-ingest warning). Pinned by `test/content-sanity.test.ts` (416 lines, 99 assertions across happy path, every junk pattern, bytes-parity, knob resolution, operator literal fail-soft). +- `src/core/content-sanity-literals.ts` (v0.40.9.0, NEW) — operator literal-substring loader. Reads `~/.gbrain/junk-substrings.txt`, one literal per non-comment non-blank line. Optional `# name=` header pairs an identifier with the following literal so audit JSONL groups by site (`linkedin_auth_wall`, `reddit_blocked`, etc.). Fail-soft on ENOENT (missing file = empty array, no error). Loaded on every ingest. Deliberately literal substrings (NOT regex) to defeat ReDoS — the regex-flavored extension is filed for v0.41+ once a real ReDoS budget exists. Pinned by `test/content-sanity-literals.test.ts` (110 lines). +- `src/core/embed-skip.ts` (v0.40.9.0, NEW) — 5-site shared predicate for the soft-block embed-skip filter. Exports `shouldSkipEmbedding(frontmatter): boolean` (JS predicate consumed by callers that already hold the page in memory), `EMBED_SKIP_SQL_FRAGMENT` (the parameterized SQL clause shared by Postgres + PGLite engines via `executeRaw`), and `buildEmbedSkipMarker(reason: string)` (writes `frontmatter.embed_skip = {at: ISO_TIMESTAMP, reason}` so the JSONB shape stays uniform across the 5 read sites). The 5 sites are: `embed.ts --stale`, `embed.ts --all`, the `embed-stale` Minion helper, plus both engines' `listStaleChunks` + `countStaleChunks`. Single source of truth so the soft-block filter cannot drift across sites (the bug class Codex r1 caught). Pinned by `test/embed-skip.test.ts` (cross-site invariant + JSONB shape). +- `src/core/audit/content-sanity-audit.ts` (v0.40.9.0, NEW) — ISO-week JSONL audit at `~/.gbrain/audit/content-sanity-YYYY-Www.jsonl` built on the v0.40.4.0 `audit-writer.ts` primitive. Records every hard-block, soft-block, and warn-trip event with `{kind, source_id, slug, bytes, matched_pattern_name?, reason, ts}`. Doctor reads the last 7 days, aggregates by `(matched_pattern_name, source_id)`, surfaces "31 ingest blocks this week, 28 from straylight-brain" so operators see which scraper is the actual problem. Honors `GBRAIN_AUDIT_DIR` for shared-filesystem multi-host setups (documented caveat in the doctor message for ops that don't share the dir). Pinned by `test/audit/content-sanity-audit.test.ts` (219 lines, 219 assertions). +- `src/commands/doctor.ts` extension (v0.40.9.0) — three new checks wired into `runDoctor()` and the JSON envelope: `oversized_pages` (warns on pages exceeding `content_sanity.bytes_warn`), `scraper_junk_pages` (warns on pages that match any junk pattern despite being live in the DB — these escaped pre-v0.40.9.0 ingest), and `content_sanity_audit_recent` (reads the last 7 days of audit events, aggregates by pattern+source). Default scans the 1000 most-recent pages; new `--content-audit` flag opts into a full scan for the cleanup wave. All three are warn-only with paste-ready fix hints (junk page → `gbrain sources audit ` + `git rm` source-of-truth, oversize → split or accept). +- `src/commands/lint.ts` extension (v0.40.9.0) — two new lint rules: `huge-page` (flags pages exceeding `content_sanity.bytes_warn` threshold) and `scraper-junk` (flags pages matching any junk pattern). Both reuse `assessContent()` from `src/core/content-sanity.ts` so lint, doctor, and ingest share one assessor — adding a junk pattern automatically covers all three surfaces. `lint.ts` lifts DB config when `~/.gbrain/` is reachable (matches what `gbrain config set` writes); falls back to file/env on CI. Pinned by `test/lint-content-sanity.test.ts` (161 lines). +- `src/commands/embed.ts` extension (v0.40.9.0) — applies the `embed-skip` filter at all 5 stale-chunk sites: `runEmbedCore --stale`, `runEmbedCore --all`, the `embed-stale` Minion helper, plus both engines' `listStaleChunks` + `countStaleChunks` via `EMBED_SKIP_SQL_FRAGMENT`. A soft-blocked page is queryable by title and slug but its chunks never enter the embed sweep. The shared helper from `src/core/embed-skip.ts` is the regression guard — no per-site ad-hoc filter is allowed. Pinned by `test/embed-skip.test.ts`. +- `src/core/import-file.ts` extension (v0.40.9.0) — `importFromContent` is the narrow waist that every ingest path passes through (`gbrain import`, `gbrain sync`, `put_page` MCP, `/ingest` webhook). It now calls `assessContent()` BEFORE chunking; verdict `hard_block_junk_pattern` throws `ContentSanityBlockError` (which every wrapper site already catches via its exception flow); verdict `warn_oversize OR oversize-without-junk` sets `frontmatter.embed_skip` via `buildEmbedSkipMarker()` AND deletes any pre-existing chunks for the page in the same transaction so search can't surface stale chunks against content that's now soft-blocked. `gbrain import` honors `errors > 0` for non-zero exit (was silently exit-0 on failed files). `classifyErrorCode` in `src/core/sync.ts` recognizes the new `PAGE_JUNK_PATTERN` code so sync-failures.jsonl grouping bins these correctly. Pinned by `test/import-file-content-sanity.test.ts` (206 lines). — shared library for the v0.12.0 graph layer. extractEntityRefs (canonical, replaces backlinks.ts duplicate) matches both `[Name](people/slug)` markdown links and Obsidian `[[people/slug|Name]]` wikilinks as of v0.12.3. extractPageLinks, inferLinkType heuristics (attended/works_at/invested_in/founded/advises/source/mentions), parseTimelineEntries, isAutoLinkEnabled config helper. `DIR_PATTERN` covers `people`, `companies`, `deals`, `topics`, `concepts`, `projects`, `entities`, `tech`, `finance`, `personal`, `openclaw`. Used by extract.ts, operations.ts auto-link post-hook, and backlinks.ts. - `src/core/zombie-reap.ts` (v0.28.1) — idempotent `installSigchldHandler()` so JS-spawned children get reaped via Bun's internal `waitpid()`. Bun (like Node) only auto-reaps when a SIGCHLD listener is registered; without it, every child the worker spawns (shell jobs, embed batches, sub-agents) becomes a zombie on exit and holds connection slots. Called once at module load from `src/cli.ts` (with Windows platform guard — SIGCHLD doesn't exist on Windows). Cross-file leak guard via `_uninstallSigchldHandlerForTests()` for tests. Layer 1 of the three-layer zombie defense; Layer 2 is tini-as-PID-1 wrapping the worker subtree (via `src/core/minions/spawn-helpers.ts`); Layer 3 is the container's own tini for hard Bun crashes. - `src/core/minions/` — Minions job queue: BullMQ-inspired, Postgres-native (queue, worker, backoff, types, protected-names, quiet-hours, stagger, handlers/shell). - `src/core/minions/queue.ts` — MinionQueue class (submit, claim, complete, fail, stall detection, parent-child, depth/child-cap, per-job timeouts, cascade-kill, attachments, idempotency keys, child_done inbox, removeOnComplete/Fail). `add()` takes a 4th `trusted` arg (separate from `opts` to prevent spread leakage); protected names in `PROTECTED_JOB_NAMES` require `{allowProtectedSubmit: true}` and the check runs trim-normalized (whitespace-bypass safe). v0.14.1 #219: `add()` plumbs `max_stalled` through with a `[1, 100]` clamp; omitted values let the schema DEFAULT (5) kick in. v0.19.0: `handleWallClockTimeouts(lockDurationMs)` is Layer 3 kill shot for jobs where `FOR UPDATE SKIP LOCKED` stall detection and the timeout sweep both fail to evict (wedged worker holding a row lock via a pending transaction). v0.19.1: `maxWaiting` coalesce path now uses `pg_advisory_xact_lock` keyed on `(name, queue)` to serialize concurrent submits for the same key, and filters on `queue` in addition to `name` so cross-queue same-name jobs don't suppress each other. diff --git a/package.json b/package.json index cb018148b..996292459 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "gbrain", - "version": "0.40.8.1", + "version": "0.41.0.0", "description": "Postgres-native personal knowledge brain with hybrid RAG search", "type": "module", "main": "src/core/index.ts", diff --git a/scripts/check-system-of-record.sh b/scripts/check-system-of-record.sh index fac4bd4dc..76c71b6e7 100755 --- a/scripts/check-system-of-record.sh +++ b/scripts/check-system-of-record.sh @@ -20,7 +20,19 @@ set -euo pipefail -ROOT="$(git rev-parse --show-toplevel 2>/dev/null || pwd)" +# Resolution order for the scan root: +# 1. $GBRAIN_SCAN_ROOT explicit override — tests pass this so they +# don't depend on `git rev-parse` walking up to an unrelated parent +# .git/ on filesystems where `git init` silently fails under +# shard-concurrency load (v0.40.10 flake-hardening fix). +# 2. `git rev-parse --show-toplevel` — production callers from inside +# the gbrain repo. +# 3. $PWD — last-resort fallback for callers without git. +if [ -n "${GBRAIN_SCAN_ROOT:-}" ]; then + ROOT="$GBRAIN_SCAN_ROOT" +else + ROOT="$(git rev-parse --show-toplevel 2>/dev/null || pwd)" +fi cd "$ROOT" # Banned direct-call patterns. Each is a method on BrainEngine that diff --git a/scripts/e5-lease-cap-ab.ts b/scripts/e5-lease-cap-ab.ts new file mode 100644 index 000000000..f12eeb856 --- /dev/null +++ b/scripts/e5-lease-cap-ab.ts @@ -0,0 +1,247 @@ +#!/usr/bin/env bun +/** + * v0.41 E5 A/B harness (D11 + codex pass-2 #7 spec). + * + * Manually-runnable script that proves the auto-adaptive lease-cap + * controller beats fixed-cap on a real upstream. Writes a structured + * receipt to test/fixtures/e5-lease-cap-ab/{timestamp}.json — that file + * is committed as the baseline. Future controller changes ship with + * their own receipt + diff against prior. + * + * **Spec (D11):** + * Workload: 500 subagent jobs, log-normal prompt distribution + * (mean 2k tokens, p99 16k tokens). Synthesized via fixture file. + * Provider: Anthropic (real API) via gateway. + * Cost cap: --budget-usd 8 per arm (D5 enforced). + * Failure injection: synthetic 429 burst at minute 15 (10s window). + * Statistical threshold: controller arm must beat fixed-cap on + * (completed_jobs / wall_clock_time) by ≥5% AND match + * (completed_jobs / dollars_spent) within ±2%. + * + * **Usage:** + * ANTHROPIC_API_KEY=sk-... bun scripts/e5-lease-cap-ab.ts + * ANTHROPIC_API_KEY=sk-... bun scripts/e5-lease-cap-ab.ts --dry-run + * + * **Cost:** ~$16 per full run ($8/arm × 2 arms). Approximate; depends on + * actual prompt lengths sampled from the fixture. + * + * **Not in CI:** This script requires a real API key + ~30min wall-clock + * + real Anthropic budget. Intended to be run BEFORE landing a controller + * change; receipt is the durable artifact. CI gating happens via the + * unit-test suite (`lease-cap-controller.test.ts` covers the pure + * decision function exhaustively). + */ + +import { writeFileSync, mkdirSync, existsSync } from 'fs'; +import { join } from 'path'; +import { PostgresEngine } from '../src/core/postgres-engine.ts'; +import { PGLiteEngine } from '../src/core/pglite-engine.ts'; +import { loadConfig } from '../src/core/config.ts'; +import type { BrainEngine } from '../src/core/engine.ts'; + +interface ArmStats { + arm: 'fixed' | 'adaptive'; + jobs_submitted: number; + jobs_completed: number; + jobs_dead: number; + wall_clock_ms: number; + total_cost_usd: number; + lease_cap_history: number[]; + bounces: number; + upstream_429s: number; +} + +interface ABReceipt { + schema_version: 1; + timestamp: string; + spec: { + job_count: number; + budget_per_arm_usd: number; + injection_at_min: number; + }; + arms: ArmStats[]; + verdict: { + throughput_advantage_pct: number; + cost_efficiency_delta_pct: number; + pr_gate_pass: boolean; + note: string; + }; +} + +function parseArgs(argv: string[]) { + const args = { + dryRun: argv.includes('--dry-run'), + jobs: 500, + budgetUsd: 8, + }; + for (const arg of argv) { + if (arg.startsWith('--jobs=')) args.jobs = parseInt(arg.split('=')[1] ?? '500', 10); + if (arg.startsWith('--budget-usd=')) args.budgetUsd = parseFloat(arg.split('=')[1] ?? '8'); + } + return args; +} + +async function openEngine(): Promise { + const cfg = loadConfig(); + if (cfg?.database_url) { + const engine = new PostgresEngine(); + await engine.connect({ database_url: cfg.database_url }); + return engine; + } + // Fallback: PGLite ephemeral. Real A/B runs should use Postgres so the + // lease-cap controller's elected-mutator pattern is exercised cross-process. + process.stderr.write('[e5-ab] WARN: using PGLite ephemeral (no DATABASE_URL); cross-worker tests will not run\n'); + const engine = new PGLiteEngine(); + await engine.connect({ database_url: '' }); + await engine.initSchema(); + return engine; +} + +/** + * Generate a synthetic prompt with the spec'd token distribution. + * Approximate — uses character counts as a stand-in for tokens (1 token + * ≈ 4 chars on English). + */ +function syntheticPrompt(index: number): string { + // Log-normal mean=2k tokens, σ such that p99 = 16k tokens. + // log(p99/p50) = z_99 * σ → σ ≈ ln(8) / 2.33 ≈ 0.89 + const mu = Math.log(2000); + const sigma = 0.89; + // Box-Muller for deterministic-per-index pseudo-Normal sample. + const u1 = ((index * 9301 + 49297) % 233280) / 233280; + const u2 = ((index * 13849 + 65521) % 233280) / 233280; + const z = Math.sqrt(-2 * Math.log(Math.max(u1, 1e-9))) * Math.cos(2 * Math.PI * u2); + const tokens = Math.exp(mu + sigma * z); + const chars = Math.max(40, Math.min(64000, Math.floor(tokens * 4))); + return `Synthetic A/B test prompt #${index}. Body: ` + 'X'.repeat(chars - 40); +} + +async function runArm( + engine: BrainEngine, + arm: 'fixed' | 'adaptive', + opts: { jobs: number; budgetUsd: number; dryRun: boolean }, +): Promise { + const start = Date.now(); + const lease_cap_history: number[] = []; + let jobs_submitted = 0; + let jobs_completed = 0; + let jobs_dead = 0; + let total_cost_usd = 0; + let bounces = 0; + let upstream_429s = 0; + + process.stderr.write(`[e5-ab] === arm=${arm} starting ===\n`); + + // Configure the cap policy for this arm. + if (arm === 'fixed') { + await engine.setConfig('minions.auto_lease_cap', 'false'); + await engine.setConfig('minions.lease_cap_current', '8'); + } else { + await engine.setConfig('minions.auto_lease_cap', 'true'); + await engine.setConfig('minions.lease_cap_current', '8'); + } + + if (opts.dryRun) { + process.stderr.write(`[e5-ab] --dry-run: skipping real submission. Would submit ${opts.jobs} jobs.\n`); + return { + arm, + jobs_submitted: opts.jobs, + jobs_completed: 0, + jobs_dead: 0, + wall_clock_ms: Date.now() - start, + total_cost_usd: 0, + lease_cap_history: [8], + bounces: 0, + upstream_429s: 0, + }; + } + + // Real-run scaffolding lives here. v0.41 ships the spec; the full + // dispatcher (queue submit + worker spin-up + 15-min 429 injector + + // tick loop) lands in the follow-up wave when the controller has been + // exercised manually first. Receipt fixture is committed as a baseline + // shape for future runs to diff against. + process.stderr.write(`[e5-ab] arm=${arm}: real-run implementation deferred to v0.41.1 follow-up.\n`); + process.stderr.write(`[e5-ab] See CHANGELOG.md "v0.41.0.0 → v0.41.1.0 follow-up" for details.\n`); + + return { + arm, + jobs_submitted, + jobs_completed, + jobs_dead, + wall_clock_ms: Date.now() - start, + total_cost_usd, + lease_cap_history, + bounces, + upstream_429s, + }; +} + +function computeVerdict(fixed: ArmStats, adaptive: ArmStats): ABReceipt['verdict'] { + // Throughput ratio: completed_jobs / wall_clock_ms. Higher is better. + const tputFixed = fixed.wall_clock_ms > 0 ? fixed.jobs_completed / fixed.wall_clock_ms : 0; + const tputAdaptive = adaptive.wall_clock_ms > 0 ? adaptive.jobs_completed / adaptive.wall_clock_ms : 0; + const throughputAdvantage = tputFixed > 0 ? ((tputAdaptive - tputFixed) / tputFixed) * 100 : 0; + + // Cost efficiency ratio: completed_jobs / dollars. Higher is better. + const effFixed = fixed.total_cost_usd > 0 ? fixed.jobs_completed / fixed.total_cost_usd : 0; + const effAdaptive = adaptive.total_cost_usd > 0 ? adaptive.jobs_completed / adaptive.total_cost_usd : 0; + const costEfficiencyDelta = effFixed > 0 ? ((effAdaptive - effFixed) / effFixed) * 100 : 0; + + // PR gate: adaptive must beat fixed by ≥5% on throughput AND match + // within ±2% on cost efficiency. + const pr_gate_pass = throughputAdvantage >= 5 && Math.abs(costEfficiencyDelta) <= 2; + + return { + throughput_advantage_pct: Math.round(throughputAdvantage * 100) / 100, + cost_efficiency_delta_pct: Math.round(costEfficiencyDelta * 100) / 100, + pr_gate_pass, + note: pr_gate_pass + ? 'controller beats fixed-cap; safe to default ON' + : 'controller does NOT meet PR gate; defaults stay OFF', + }; +} + +async function main() { + const argv = process.argv.slice(2); + const opts = parseArgs(argv); + + const engine = await openEngine(); + try { + const fixed = await runArm(engine, 'fixed', opts); + const adaptive = await runArm(engine, 'adaptive', opts); + const verdict = computeVerdict(fixed, adaptive); + + const receipt: ABReceipt = { + schema_version: 1, + timestamp: new Date().toISOString(), + spec: { + job_count: opts.jobs, + budget_per_arm_usd: opts.budgetUsd, + injection_at_min: 15, + }, + arms: [fixed, adaptive], + verdict, + }; + + const fixtureDir = join(process.cwd(), 'test/fixtures/e5-lease-cap-ab'); + if (!existsSync(fixtureDir)) mkdirSync(fixtureDir, { recursive: true }); + const receiptPath = join( + fixtureDir, + `${new Date().toISOString().replace(/[:.]/g, '-')}${opts.dryRun ? '-dry-run' : ''}.json`, + ); + writeFileSync(receiptPath, JSON.stringify(receipt, null, 2)); + process.stderr.write(`[e5-ab] receipt written: ${receiptPath}\n`); + process.stderr.write(`[e5-ab] verdict: ${verdict.note}\n`); + process.exit(verdict.pr_gate_pass || opts.dryRun ? 0 : 1); + } finally { + await engine.disconnect().catch(() => {}); + } +} + +if (import.meta.main) { + main().catch(err => { + process.stderr.write(`[e5-ab] FATAL: ${err instanceof Error ? err.message : String(err)}\n`); + process.exit(2); + }); +} diff --git a/scripts/run-slow-tests.sh b/scripts/run-slow-tests.sh index 70dc776c2..f37ee6ccc 100755 --- a/scripts/run-slow-tests.sh +++ b/scripts/run-slow-tests.sh @@ -17,4 +17,9 @@ if [ "${#slow_files[@]}" -eq 0 ]; then fi echo "[run-slow-tests] running ${#slow_files[@]} slow files (CI runs these as part of bun run test)" -exec bun test --timeout=60000 "${slow_files[@]}" +# v0.40.10 flake-hardening: bump per-test timeout 60s → 120s. Slow tests +# legitimately approach 60s in isolation (longmemeval E2E suite is ~50s); +# when bun runs slow files in parallel, CPU contention pushes them past +# 60s and individual tests timeout even though they'd pass solo. Slow +# tests are explicit by-name — generous per-test budget is correct. +exec bun test --timeout=120000 "${slow_files[@]}" diff --git a/scripts/run-unit-parallel.sh b/scripts/run-unit-parallel.sh index cb2a95d95..e4bcdc245 100755 --- a/scripts/run-unit-parallel.sh +++ b/scripts/run-unit-parallel.sh @@ -58,10 +58,27 @@ N="${SHARDS_OVERRIDE:-${SHARDS:-$(detect_cpus)}}" if ! printf '%s' "$N" | grep -qE '^[0-9]+$' || [ "$N" -lt 1 ]; then echo "ERROR: invalid shard count: $N" >&2; exit 2 fi +# v0.40.10 flake-hardening: clamp default to 4 (was 8) to match CI's +# test-shard.sh fan-out. At 8-shard parallel on Apple Silicon we observed +# shard 5 SIGKILL during source-health.test.ts's PGLite migration replay — +# 8 parallel PGLite WASM inits contend severely on the lockfile, and the +# 92-migration replay × 8 simultaneous can wedge past even 900s. CI uses +# 4 and is stable. Trade ~2x wallclock for reliability + parity with CI's +# fan-out. Override via --shards N or SHARDS=N (still capped at 8). [ "$N" -gt 8 ] && N=8 +if [ -z "${SHARDS_OVERRIDE:-}" ] && [ -z "${SHARDS:-}" ] && [ "$N" -gt 4 ]; then + N=4 +fi INTRA_CONC="${MAX_CONCURRENCY_OVERRIDE:-${GBRAIN_TEST_MAX_CONCURRENCY:-4}}" -SHARD_TIMEOUT="${GBRAIN_TEST_SHARD_TIMEOUT:-600}" +# v0.40.10 flake-hardening: bump per-shard cap 600 → 1500 (was 900). At +# 4-shard default each shard runs 159 files / ~2420 tests with internal +# wallclock 960-1020s. The 900s value (sized for 8-shard's ~80 files / +# 1100 tests at 620-770s) false-killed shard 1 at 900s even though it +# had completed in 968s. 1500s cap gives ~55% headroom over observed +# 4-shard wallclock; real hangs still hit it. Override via +# GBRAIN_TEST_SHARD_TIMEOUT=N. +SHARD_TIMEOUT="${GBRAIN_TEST_SHARD_TIMEOUT:-1500}" # ────────────────────────────────────────────────────────────────────────── # Output directories. Prefer workspace-local .context/, fall back to /tmp. diff --git a/src/admin-embedded.ts b/src/admin-embedded.ts index 71de6121a..c1115a521 100644 --- a/src/admin-embedded.ts +++ b/src/admin-embedded.ts @@ -1,13 +1,13 @@ // AUTO-GENERATED — do not edit by hand. // Run `bun run scripts/build-admin-embedded.ts` to regenerate. -// Source: admin/dist/ at 2026-05-22. +// Source: admin/dist/ at 2026-05-24. // // Bun resolves the file: imports to a path that works at runtime even // inside a compiled binary (`bun build --compile`). The manifest maps // the request path the express handler sees to (resolved-path, mime). // @ts-ignore — type: 'file' is Bun ESM, not in lib.d.ts -import A_0_assets_index_DFgMZhBE_js from '../admin/dist/assets/index-DFgMZhBE.js' with { type: 'file' }; +import A_0_assets_index_DqP_zmqH_js from '../admin/dist/assets/index-DqP-zmqH.js' with { type: 'file' }; // @ts-ignore — type: 'file' is Bun ESM, not in lib.d.ts import A_1_assets_index_GxkWX7v3_css from '../admin/dist/assets/index-GxkWX7v3.css' with { type: 'file' }; // @ts-ignore — type: 'file' is Bun ESM, not in lib.d.ts @@ -19,7 +19,7 @@ export interface AdminAsset { } export const ADMIN_ASSETS: Record = { - "/admin/assets/index-DFgMZhBE.js": { path: A_0_assets_index_DFgMZhBE_js as unknown as string, mime: "application/javascript; charset=utf-8" }, + "/admin/assets/index-DqP-zmqH.js": { path: A_0_assets_index_DqP_zmqH_js as unknown as string, mime: "application/javascript; charset=utf-8" }, "/admin/assets/index-GxkWX7v3.css": { path: A_1_assets_index_GxkWX7v3_css as unknown as string, mime: "text/css; charset=utf-8" }, "/admin/index.html": { path: A_2_index_html as unknown as string, mime: "text/html; charset=utf-8" }, }; diff --git a/src/assets/wasm/grammars/tree-sitter-sql.wasm b/src/assets/wasm/grammars/tree-sitter-sql.wasm new file mode 100755 index 000000000..f7110174c Binary files /dev/null and b/src/assets/wasm/grammars/tree-sitter-sql.wasm differ diff --git a/src/cli.ts b/src/cli.ts index 568c73f8a..8dcd86b77 100755 --- a/src/cli.ts +++ b/src/cli.ts @@ -1108,7 +1108,16 @@ async function handleCliOnly(command: string, args: string[]) { switch (command) { case 'import': { const { runImport } = await import('./commands/import.ts'); - await runImport(engine, args); + // v0.41 (Codex r2 #3 fix): honor errors counter for exit code. + // runImport's per-file catch already records failures, but the + // CLI was discarding the result so the process exited 0 even + // when files failed (e.g. content-sanity hard-block throws, + // size-cap throws, parse errors). Surface non-zero on errors > 0 + // so wrappers (sync, CI scripts, `&& gbrain doctor`) propagate. + const importResult = await runImport(engine, args); + if (importResult.errors > 0) { + process.exitCode = 1; + } break; } case 'export': { diff --git a/src/commands/code-def.ts b/src/commands/code-def.ts index beeaf645c..81a91736d 100644 --- a/src/commands/code-def.ts +++ b/src/commands/code-def.ts @@ -32,7 +32,14 @@ export async function findCodeDef( opts: { limit?: number; language?: string } = {}, ): Promise { const limit = opts.limit ?? 20; - const DEF_TYPES = ['function', 'class', 'interface', 'type', 'enum', 'struct', 'trait', 'module', 'contract']; + // v0.41 D2: SQL DDL targets (table/view/index/procedure/schema/database/ + // trigger) are first-class definitions in the SQL sense. The chunker's + // normalizeSymbolType maps create_table → 'table' etc, so adding the SQL + // kinds here is what makes `gbrain code-def users` work against SQL. + const DEF_TYPES = [ + 'function', 'class', 'interface', 'type', 'enum', 'struct', 'trait', 'module', 'contract', + 'table', 'view', 'index', 'procedure', 'schema', 'database', 'trigger', + ]; const params: unknown[] = [symbol, limit]; let whereLang = ''; if (opts.language) { diff --git a/src/commands/doctor.ts b/src/commands/doctor.ts index fd0e8b889..f28fa60fc 100644 --- a/src/commands/doctor.ts +++ b/src/commands/doctor.ts @@ -563,6 +563,9 @@ export async function doctorReportRemote(engine: BrainEngine): Promise { + try { + const bounceRows = await engine.executeRaw<{ count: string }>( + `SELECT count(*)::text AS count FROM minion_lease_pressure_log + WHERE bounced_at > now() - interval '24 hours'`, + ); + const bounces = parseInt(bounceRows[0]?.count ?? '0', 10); + if (bounces === 0) { + return { + name: 'subagent_health', + status: 'ok', + message: 'No rate-lease pressure in last 24h', + }; + } + if (bounces >= 1000) { + return { + name: 'subagent_health', + status: 'fail', + message: `${bounces} lease-pressure bounces in last 24h — this is blocking real work. Raise the cap: \`export GBRAIN_ANTHROPIC_MAX_INFLIGHT=64\` (or \`unlimited\` for Azure / Bedrock / self-hosted upstreams with no provider-side rate limit). After raising, restart \`gbrain jobs work\`.`, + }; + } + // 1-999 bounces: cross-check forward progress. + const completedRows = await engine.executeRaw<{ count: string }>( + `SELECT count(*)::text AS count FROM minion_jobs + WHERE finished_at > now() - interval '24 hours' + AND status = 'completed' + AND name = 'subagent'`, + ).catch(() => [{ count: '0' }]); + const completed = parseInt(completedRows[0]?.count ?? '0', 10); + if (bounces >= 100 && completed === 0) { + return { + name: 'subagent_health', + status: 'warn', + message: `${bounces} lease-pressure bounces in last 24h with no completed subagent jobs — cap is too tight. Raise via \`export GBRAIN_ANTHROPIC_MAX_INFLIGHT=64\` (or \`unlimited\` for upstreams with no provider-side cap).`, + }; + } + return { + name: 'subagent_health', + status: 'ok', + message: `Lease pressure: ${bounces} bounces in last 24h, ${completed} subagent jobs completed — backpressure is binding but throughput is healthy`, + }; + } catch (err) { + const msg = err instanceof Error ? err.message : String(err); + if (process.env.GBRAIN_DEBUG === '1') { + process.stderr.write(`[doctor] subagent_health skipped: ${msg}\n`); + } + return { + name: 'subagent_health', + status: 'ok', + message: 'Skipped (minion_lease_pressure_log unavailable — pre-v0.41 brain)', + }; + } +} + export async function checkVoiceGateHealth(engine: BrainEngine): Promise { try { const rows = await engine.executeRaw<{ total: number; failures: number }>( @@ -3669,6 +3743,176 @@ export async function buildChecks( mbcHb(); } + // 11b. Content sanity checks (v0.41). + // + // Three sibling checks all backed by the shared assessor in + // src/core/content-sanity.ts so the surface stays aligned with the + // ingest gate at importFromContent and the lint rules at lintContent. + // + // - oversized_pages: indexed-free table scan (~100ms on 100K-page brains) + // counting pages whose body (compiled_truth + timeline, UTF-8 bytes + // via octet_length per Codex r2 #13) exceeds the block threshold. + // Status warn when 1+ rows; never fail (oversize is now a soft state). + // - scraper_junk_pages: capped 1000-most-recent default + --content-audit + // opt-in for full scan (D10 mirrors --index-audit precedent). Applies + // the assessor per-page on title + 2KB head-slice + frontmatter. + // - content_sanity_audit_recent: reads ~/.gbrain/audit/content-sanity-*.jsonl + // over the last 7 days, aggregates by event type + source. Caveat + // (Codex r1 #14): JSONL is local-only — multi-host operators should + // share GBRAIN_AUDIT_DIR. Message names this so the limitation is + // visible at the doctor surface. + const fullContentAudit = args.includes('--content-audit'); + progress.heartbeat('oversized_pages'); + try { + const sql = db.getConnection(); + // Read effective bytes_block from the cached effectiveCfg loaded + // earlier in this doctor run if available; otherwise default. + // (We re-read here per-check to avoid threading config through + // every check — bytes_block is read once per doctor run via + // loadConfig which caches in module-level config layer.) + const { loadConfig: _loadCfg } = await import('../core/config.ts'); + const _cfg = _loadCfg(); + const bytesBlock = _cfg?.content_sanity?.bytes_block ?? 500_000; + const rows = await sql` + SELECT p.slug, p.source_id, + octet_length(p.compiled_truth) + octet_length(COALESCE(p.timeline, '')) AS bytes + FROM pages p + WHERE p.deleted_at IS NULL + AND (octet_length(p.compiled_truth) + octet_length(COALESCE(p.timeline, ''))) > ${bytesBlock} + ORDER BY bytes DESC + LIMIT 100 + `; + if (rows.length === 0) { + checks.push({ + name: 'oversized_pages', + status: 'ok', + message: `No pages exceed ${bytesBlock} bytes`, + }); + } else { + const oversizeRows = rows as unknown as Array<{ slug: string; source_id: string; bytes: number }>; + const top = oversizeRows.slice(0, 3) + .map(r => `${r.slug} (${r.bytes}b, src=${r.source_id})`) + .join('; '); + checks.push({ + name: 'oversized_pages', + status: 'warn', + message: `${rows.length} page(s) exceed ${bytesBlock}-byte block threshold. Top: ${top}. New ingests with the same shape get frontmatter.embed_skip set automatically; existing oversized pages can be split or accepted as non-embeddable.`, + }); + } + } catch (err) { + const msg = err instanceof Error ? err.message : String(err); + checks.push({ + name: 'oversized_pages', + status: 'ok', + message: `Skipped (${msg})`, + }); + } + + progress.heartbeat('scraper_junk_pages'); + try { + const sql = db.getConnection(); + const { assessContentSanity } = await import('../core/content-sanity.ts'); + const { loadOperatorLiterals } = await import('../core/content-sanity-literals.ts'); + const literals = loadOperatorLiterals(); + const scanLimit = fullContentAudit ? null : 1000; + const rows = scanLimit + ? await sql` + SELECT p.slug, p.source_id, p.title, + LEFT(p.compiled_truth, 2048) AS body_head, + LEFT(COALESCE(p.timeline, ''), 1024) AS tl_head, + p.frontmatter + FROM pages p + WHERE p.deleted_at IS NULL + ORDER BY p.updated_at DESC + LIMIT ${scanLimit} + ` + : await sql` + SELECT p.slug, p.source_id, p.title, + LEFT(p.compiled_truth, 2048) AS body_head, + LEFT(COALESCE(p.timeline, ''), 1024) AS tl_head, + p.frontmatter + FROM pages p + WHERE p.deleted_at IS NULL + `; + const hits: Array<{ slug: string; matched: string[] }> = []; + const scanRows = rows as unknown as Array<{ slug: string; source_id: string; title: string; body_head: string; tl_head: string; frontmatter: Record | null }>; + for (const r of scanRows) { + const sanity = assessContentSanity({ + compiled_truth: r.body_head ?? '', + timeline: r.tl_head ?? '', + title: r.title ?? '', + bytes_warn: Number.MAX_SAFE_INTEGER, // we ONLY care about junk-pattern hits here + bytes_block: Number.MAX_SAFE_INTEGER, + extra_literals: literals, + }); + if (sanity.shouldHardBlock) { + hits.push({ + slug: r.slug, + matched: [...sanity.junk_pattern_matches, ...sanity.literal_substring_matches], + }); + } + } + if (hits.length === 0) { + checks.push({ + name: 'scraper_junk_pages', + status: 'ok', + message: scanLimit + ? `No junk-pattern hits in ${rows.length} recent page(s) (use --content-audit for full scan)` + : `No junk-pattern hits in ${rows.length} page(s) (full audit)`, + }); + } else { + const top = hits.slice(0, 3).map(h => `${h.slug} [${h.matched.join(',')}]`).join('; '); + checks.push({ + name: 'scraper_junk_pages', + status: 'warn', + message: `${hits.length} page(s) match junk patterns. Top: ${top}. ${scanLimit ? '(scanned 1000 most-recent; rerun with --content-audit for full scan)' : '(full audit)'} New ingests with these shapes are now hard-blocked; existing inventory should be cleaned at source.`, + }); + } + } catch (err) { + const msg = err instanceof Error ? err.message : String(err); + checks.push({ + name: 'scraper_junk_pages', + status: 'ok', + message: `Skipped (${msg})`, + }); + } + + progress.heartbeat('content_sanity_audit_recent'); + try { + const { readRecentContentSanityEvents, summarizeContentSanityEvents } = + await import('../core/audit/content-sanity-audit.ts'); + const events = readRecentContentSanityEvents(7); + if (events.length === 0) { + checks.push({ + name: 'content_sanity_audit_recent', + status: 'ok', + message: 'No content-sanity events in last 7 days (audit JSONL is local to this host; share GBRAIN_AUDIT_DIR for multi-host visibility)', + }); + } else { + const summary = summarizeContentSanityEvents(events); + const topPatterns = summary.top_patterns.slice(0, 3).map(p => `${p.name}=${p.count}`).join(', '); + const topSources = Object.entries(summary.by_source) + .sort((a, b) => b[1] - a[1]) + .slice(0, 3) + .map(([s, n]) => `${s}=${n}`) + .join(', '); + const status: 'ok' | 'warn' | 'fail' = + events.length >= 100 ? 'fail' : events.length >= 10 ? 'warn' : 'ok'; + checks.push({ + name: 'content_sanity_audit_recent', + status, + message: `${events.length} events (hard=${summary.by_type.hard_block} soft=${summary.by_type.soft_block} warn=${summary.by_type.warn})${topPatterns ? ', patterns: ' + topPatterns : ''}${topSources ? ', sources: ' + topSources : ''}. (Local audit only — multi-host operators set GBRAIN_AUDIT_DIR.)`, + }); + } + } catch (err) { + const msg = err instanceof Error ? err.message : String(err); + checks.push({ + name: 'content_sanity_audit_recent', + status: 'ok', + message: `Skipped (${msg})`, + }); + } + // 11a. Frontmatter integrity (v0.22.4, hardened in v0.38.2.0). // scanBrainSources walks every registered source's local_path on disk // (not from the DB), invoking parseMarkdown(..., {validate:true}) per diff --git a/src/commands/embed.ts b/src/commands/embed.ts index 071e930e9..b93e368b3 100644 --- a/src/commands/embed.ts +++ b/src/commands/embed.ts @@ -7,6 +7,7 @@ import { getCliOptions, cliOptsToProgressOptions } from '../core/cli-options.ts' import { assertEmbeddingEnabled } from '../core/embedding-dim-check.ts'; import { loadConfig } from '../core/config.ts'; import { slog, serr } from '../core/console-prefix.ts'; +import { filterOutEmbedSkipped } from '../core/embed-skip.ts'; export interface EmbedOpts { /** Embed ALL pages (every chunk). */ @@ -353,7 +354,18 @@ async function embedAll( } // v0.31.12: when sourceId is set, scope listPages to that source. - const pages = await engine.listPages({ limit: 100000, ...(sourceId && { sourceId }) }); + // v0.41 (D8 + Codex r2 #11): apply embed-skip filter via the shared + // helper so the `--all` path honors `frontmatter.embed_skip` the same + // way the `--stale` path does. Without this filter, `gbrain embed --all` + // (common after model swaps) re-embeds every soft-blocked page, + // defeating the soft-block. Filtering JS-side here mirrors the SQL-side + // filter that listStaleChunks/countStaleChunks apply on --stale. + const allPages = await engine.listPages({ limit: 100000, ...(sourceId && { sourceId }) }); + const pages = filterOutEmbedSkipped(allPages); + const skippedByEmbedSkip = allPages.length - pages.length; + if (skippedByEmbedSkip > 0) { + serr(`[embed] skipped ${skippedByEmbedSkip} page(s) with frontmatter.embed_skip set`); + } let processed = 0; // Concurrency limit for parallel page embedding. diff --git a/src/commands/jobs-watch.ts b/src/commands/jobs-watch.ts new file mode 100644 index 000000000..ec0dd9381 --- /dev/null +++ b/src/commands/jobs-watch.ts @@ -0,0 +1,241 @@ +/** + * v0.41 D2 — `gbrain jobs watch` live TTY dashboard. + * + * Submit and supervise instead of submit and pray. Shows throughput, + * current rate-lease utilization, top errors clustered by error-classify, + * and budget remaining (when a `--budget-usd` parent is in flight). + * + * Refresh tick: 1s (intentionally conservative — operators don't need + * 60fps; 1s keeps the SQL load nominal even when multiple watch sessions + * point at the same brain). + * + * Rendering: manual ANSI cursor management (no TUI dep). Clears the + * screen on first render, then redraws from the top each tick using + * cursor-home + erase-down. On non-TTY (cron / wrapped redirect), + * falls through to one snapshot line per tick in `--progress-json` + * shape so wrappers can parse. + * + * Quit: Ctrl-C (SIGINT), 'q', or stdin close — the watcher restores the + * cursor + clears its own region on shutdown so the terminal isn't left + * with a half-rendered dashboard. + * + * No SSE consumer in v0.41 — local polling against the brain engine is + * the foundation. SSE wiring through `serve-http.ts` is filed as a + * v0.42 follow-up so the watch command can also stream from a remote + * brain server (admin SPA tab in T13 does the same direct-polling fallback + * via the engine). + */ + +import type { BrainEngine } from '../core/engine.ts'; +import { MinionQueue } from '../core/minions/queue.ts'; +import { clusterErrors, type ErrorCluster } from '../core/minions/error-classify.ts'; +import { countRecentLeasePressure } from '../core/minions/lease-pressure-audit.ts'; + +const ANSI = { + clear: '\x1b[2J', + cursorHome: '\x1b[H', + cursorHide: '\x1b[?25l', + cursorShow: '\x1b[?25h', + eraseDown: '\x1b[0J', + bold: '\x1b[1m', + reset: '\x1b[0m', + dim: '\x1b[2m', + green: '\x1b[32m', + yellow: '\x1b[33m', + red: '\x1b[31m', +}; + +export interface WatchSnapshot { + /** Wall-clock at render time (ms since epoch). */ + ts_ms: number; + /** Per-job-name totals over the last 24h window. */ + by_type: Array<{ name: string; total: number; completed: number; failed: number; dead: number }>; + /** waiting / active / stalled counts (point-in-time). */ + queue_health: { waiting: number; active: number; stalled: number }; + /** Lease pressure bounces in last 1h. */ + lease_pressure_1h: number; + /** Top-N error clusters seen in last 24h. */ + top_errors: Array<{ cluster: ErrorCluster; count: number }>; + /** Budget owners in flight: per-owner remaining cents. */ + budget_owners: Array<{ owner_id: number; remaining_cents: number; total_spent_cents: number }>; +} + +/** Pure renderer — takes a snapshot, returns the text to write. Exported for tests. */ +export function renderSnapshot(s: WatchSnapshot, opts: { useAnsi?: boolean } = {}): string { + const a = opts.useAnsi !== false; + const c = (color: string) => (a ? color : ''); + const lines: string[] = []; + lines.push(`${c(ANSI.bold)}gbrain jobs watch${c(ANSI.reset)} ${c(ANSI.dim)}q to quit | ${new Date(s.ts_ms).toLocaleTimeString()}${c(ANSI.reset)}`); + lines.push(''); + + // Queue health panel. + lines.push(`${c(ANSI.bold)}Queue${c(ANSI.reset)} waiting=${s.queue_health.waiting} active=${s.queue_health.active} stalled=${s.queue_health.stalled}`); + lines.push(''); + + // Per-type breakdown. + if (s.by_type.length > 0) { + lines.push(`${c(ANSI.bold)}By type (24h)${c(ANSI.reset)}`); + lines.push(` ${'name'.padEnd(20)} ${'total'.padStart(6)} ${'done'.padStart(6)} ${'fail'.padStart(6)} ${'dead'.padStart(6)}`); + for (const t of s.by_type.slice(0, 6)) { + lines.push( + ` ${t.name.padEnd(20)} ${String(t.total).padStart(6)} ${String(t.completed).padStart(6)} ${String(t.failed).padStart(6)} ${String(t.dead).padStart(6)}`, + ); + } + lines.push(''); + } + + // Lease pressure panel — color-coded by severity. + const lpColor = s.lease_pressure_1h === 0 + ? c(ANSI.green) + : s.lease_pressure_1h >= 100 ? c(ANSI.red) : c(ANSI.yellow); + lines.push(`${c(ANSI.bold)}Lease pressure (1h)${c(ANSI.reset)} ${lpColor}${s.lease_pressure_1h} bounce${s.lease_pressure_1h === 1 ? '' : 's'}${c(ANSI.reset)}`); + lines.push(''); + + // Top errors clustered. + if (s.top_errors.length > 0) { + lines.push(`${c(ANSI.bold)}Top errors (24h)${c(ANSI.reset)}`); + for (const e of s.top_errors.slice(0, 5)) { + lines.push(` ${String(e.count).padStart(4)} × ${e.cluster}`); + } + lines.push(''); + } + + // Budget panel. + if (s.budget_owners.length > 0) { + lines.push(`${c(ANSI.bold)}Budget owners${c(ANSI.reset)}`); + for (const b of s.budget_owners.slice(0, 5)) { + const remaining = `$${(b.remaining_cents / 100).toFixed(2)}`; + const spent = `$${(b.total_spent_cents / 100).toFixed(2)}`; + lines.push(` owner=${b.owner_id} spent=${spent} remaining=${remaining}`); + } + lines.push(''); + } + + return lines.join('\n'); +} + +/** Read a single snapshot of dashboard state from the engine. */ +export async function readSnapshot(engine: BrainEngine): Promise { + const queue = new MinionQueue(engine); + const stats = await queue.getStats(); + + // Lease pressure (best-effort; pre-v93 brains return 0). + let lease_pressure_1h = 0; + try { + lease_pressure_1h = await countRecentLeasePressure(engine, 3600_000); + } catch { + /* pre-v93 brain */ + } + + // Top errors clustered. Best-effort. + let top_errors: Array<{ cluster: ErrorCluster; count: number }> = []; + try { + const errRows = await engine.executeRaw<{ id: number; last_error: string | null }>( + `SELECT id, error_text AS last_error FROM minion_jobs + WHERE status IN ('dead', 'failed') + AND updated_at > now() - interval '24 hours'`, + ); + top_errors = clusterErrors(errRows).slice(0, 5).map(c => ({ cluster: c.cluster, count: c.count })); + } catch { + /* DB unavailable */ + } + + // Budget owners with non-zero cents. Best-effort. + let budget_owners: Array<{ owner_id: number; remaining_cents: number; total_spent_cents: number }> = []; + try { + const ownerRows = await engine.executeRaw<{ + owner_id: number; + remaining_cents: number; + total_spent_cents: number; + }>( + `SELECT + mj.id AS owner_id, + mj.budget_remaining_cents AS remaining_cents, + COALESCE((SELECT SUM(ABS(cents_delta)) FROM minion_budget_log + WHERE owner_id = mj.id AND event_type = 'reserved'), 0) AS total_spent_cents + FROM minion_jobs mj + WHERE mj.budget_remaining_cents IS NOT NULL + AND mj.budget_owner_job_id = mj.id + AND mj.status NOT IN ('completed', 'failed', 'dead', 'cancelled') + ORDER BY mj.id DESC + LIMIT 5`, + ); + budget_owners = ownerRows.map(r => ({ + owner_id: r.owner_id, + remaining_cents: r.remaining_cents ?? 0, + total_spent_cents: r.total_spent_cents ?? 0, + })); + } catch { + /* pre-v93 brain */ + } + + return { + ts_ms: Date.now(), + by_type: stats.by_type.map(t => ({ + name: t.name, + total: t.total, + completed: t.completed, + failed: t.failed, + dead: t.dead, + })), + queue_health: stats.queue_health, + lease_pressure_1h, + top_errors, + budget_owners, + }; +} + +export interface WatchOptions { + /** Refresh interval. Default 1000ms. */ + refreshMs?: number; + /** Stream JSON snapshots to stdout (non-TTY mode). */ + json?: boolean; +} + +/** + * Main entrypoint for `gbrain jobs watch`. Runs until SIGINT or 'q' + * keypress (on TTY). Non-TTY mode loops with --progress-json output. + */ +export async function runWatch(engine: BrainEngine, opts: WatchOptions = {}): Promise { + const refreshMs = opts.refreshMs ?? 1000; + const isTTY = process.stdout.isTTY === true; + const json = opts.json || !isTTY; + + let stopped = false; + const stop = () => { + stopped = true; + }; + + if (isTTY && !json) { + process.stdout.write(ANSI.cursorHide + ANSI.clear + ANSI.cursorHome); + process.on('SIGINT', () => { + process.stdout.write(ANSI.cursorShow + ANSI.clear + ANSI.cursorHome); + stop(); + process.exit(0); + }); + // Read stdin for 'q' keypress. + if (process.stdin.isTTY && process.stdin.setRawMode) { + process.stdin.setRawMode(true); + process.stdin.resume(); + process.stdin.on('data', (data: Buffer) => { + if (data.toString() === 'q' || data[0] === 3) { + process.stdout.write(ANSI.cursorShow + ANSI.clear + ANSI.cursorHome); + stop(); + process.exit(0); + } + }); + } + } + + while (!stopped) { + const snap = await readSnapshot(engine); + if (json) { + process.stdout.write(JSON.stringify({ event: 'jobs.watch.snapshot', ...snap }) + '\n'); + } else { + // TTY: clear + cursor-home + render. + process.stdout.write(ANSI.cursorHome + ANSI.eraseDown); + process.stdout.write(renderSnapshot(snap, { useAnsi: true })); + } + await new Promise(r => setTimeout(r, refreshMs)); + } +} diff --git a/src/commands/jobs.ts b/src/commands/jobs.ts index 880ed0fb5..737b0077a 100644 --- a/src/commands/jobs.ts +++ b/src/commands/jobs.ts @@ -545,6 +545,72 @@ HANDLER TYPES (built in) console.log(' No jobs in the last 24 hours.'); } console.log(`\n Queue health: ${stats.queue_health.waiting} waiting, ${stats.queue_health.active} active, ${stats.queue_health.stalled} stalled`); + + // v0.41 Bug 2 / Eng D8 — surface lease pressure to the operator. + // Reads minion_lease_pressure_log windowed at 1h. Best-effort: pre-v93 + // brains (no table) silently skip; the queue_health line above is the + // operator's primary signal in that case. + try { + const lpRows = await engine.executeRaw<{ count: string }>( + `SELECT count(*)::text AS count FROM minion_lease_pressure_log + WHERE bounced_at > now() - interval '1 hour'`, + ); + const lpCount = parseInt(lpRows[0]?.count ?? '0', 10); + if (lpCount > 0) { + // Also surface whether any of those bounces stalled forward progress. + // Bounces with rising completed counts = healthy backpressure; bounces + // with zero completes = real blocker (matches doctor's subagent_health). + const completedRows = await engine.executeRaw<{ count: string }>( + `SELECT count(*)::text AS count FROM minion_jobs + WHERE finished_at > now() - interval '1 hour' + AND status = 'completed' AND name = 'subagent'`, + ).catch(() => [{ count: '0' }]); + const completed = parseInt(completedRows[0]?.count ?? '0', 10); + const tag = completed > 0 + ? `(${completed} subagent job${completed === 1 ? '' : 's'} completed, throughput healthy)` + : `(no subagent jobs completed — cap may be too tight; \`export GBRAIN_ANTHROPIC_MAX_INFLIGHT=64\`)`; + console.log(` Lease pressure (1h): ${lpCount} bounce${lpCount === 1 ? '' : 's'} ${tag}`); + } else { + console.log(` Lease pressure (1h): 0 bounces`); + } + } catch { + // Pre-v93 brain — no table. Silent skip. + } + + // v0.41 D3 — error clustering. Optional via --cluster-errors flag so + // operators only see the breakdown when triaging a fail-heavy batch + // (default stats output stays scannable). Pulls last 24h of dead + + // failed jobs, classifies by error-classify.ts buckets, sorts by + // count, surfaces top 5 with paste-ready retry hints. + if (hasFlag(args, '--cluster-errors')) { + try { + const { clusterErrors } = await import('../core/minions/error-classify.ts'); + const errRows = await engine.executeRaw<{ id: number; last_error: string | null }>( + `SELECT id, error_text AS last_error FROM minion_jobs + WHERE status IN ('dead', 'failed') + AND updated_at > now() - interval '24 hours'`, + ); + if (errRows.length === 0) { + console.log(`\n Error clusters (24h): no dead/failed jobs`); + } else { + const clusters = clusterErrors(errRows); + console.log(`\n Error clusters (24h):`); + for (const c of clusters.slice(0, 5)) { + const sample = c.sample_ids.length > 0 + ? ` (e.g. \`gbrain jobs get ${c.sample_ids[0]}\`)` : ''; + console.log(` ${String(c.count).padStart(4)} × ${c.cluster.padEnd(22)}${sample}`); + } + if (clusters.length > 5) { + console.log(` + ${clusters.length - 5} more cluster${clusters.length - 5 === 1 ? '' : 's'}`); + } + } + } catch (e) { + // error-classify import or SQL fail. Don't block stats output. + if (process.env.GBRAIN_DEBUG === '1') { + console.error(`[jobs stats] cluster-errors skipped: ${e instanceof Error ? e.message : String(e)}`); + } + } + } break; } @@ -1003,6 +1069,18 @@ HANDLER TYPES (built in) break; } + case 'watch': { + // v0.41 D2 — live TTY dashboard (or JSON snapshots on non-TTY). + try { await queue.ensureSchema(); } + catch (e) { console.error(e instanceof Error ? e.message : String(e)); process.exit(1); } + const { runWatch } = await import('./jobs-watch.ts'); + const refreshArg = args.find(a => a.startsWith('--refresh-ms=')); + const refreshMs = refreshArg ? parseInt(refreshArg.split('=')[1] ?? '1000', 10) : 1000; + const json = hasFlag(args, '--json'); + await runWatch(engine, { refreshMs, json }); + break; + } + default: console.error(`Unknown subcommand: ${sub}. Run 'gbrain jobs --help' for usage.`); process.exit(1); diff --git a/src/commands/lint.ts b/src/commands/lint.ts index ce0e5df4b..521103c39 100644 --- a/src/commands/lint.ts +++ b/src/commands/lint.ts @@ -19,6 +19,13 @@ import { readFileSync, writeFileSync, readdirSync, statSync, lstatSync, existsSync } from 'fs'; import { join, relative } from 'path'; import { parseMarkdown, type ParseValidationCode } from '../core/markdown.ts'; +import { + assessContentSanity, + type OperatorLiteral, + DEFAULT_BYTES_WARN, +} from '../core/content-sanity.ts'; +import { loadOperatorLiterals } from '../core/content-sanity-literals.ts'; +import { loadConfig, loadConfigWithEngine, gbrainPath } from '../core/config.ts'; export interface LintIssue { file: string; @@ -60,7 +67,26 @@ const LLM_PREAMBLES = [ // ── Rules ────────────────────────────────────────────────────────── -export function lintContent(content: string, filePath: string): LintIssue[] { +/** + * Per-call options for `lintContent`. Tests pass content-sanity opts + * directly so the linter can be exercised without an engine. + * Production callers (`runLintCore`) resolve effective config first + * via the file/env/DB precedence chain and pass through. + */ +export interface LintContentOpts { + /** v0.41 content-sanity thresholds + operator literals. When omitted, + * the assessor uses its built-in defaults (50K warn, 500K block, + * built-in junk patterns only). */ + contentSanity?: { + bytes_warn?: number; + bytes_block?: number; + junk_patterns_enabled?: boolean; + disabled?: boolean; + operator_literals?: ReadonlyArray; + }; +} + +export function lintContent(content: string, filePath: string, opts: LintContentOpts = {}): LintIssue[] { const issues: LintIssue[] = []; const lines = content.split('\n'); @@ -182,6 +208,57 @@ export function lintContent(content: string, filePath: string): LintIssue[] { } } + // v0.41 content-sanity rules. Two new lint rules (huge-page + + // scraper-junk) backed by the shared assessor in + // src/core/content-sanity.ts so the threshold + pattern set stays + // in sync with the ingest gate at importFromContent. Kill-switch + // (contentSanity.disabled) suppresses both. + // + // Bytes are measured against the parsed body (compiled_truth + + // timeline) for parity with doctor's `oversized_pages` check (D2). + // The earlier file-byte design disagreed with doctor on pages with + // large frontmatter; pulling from parsed keeps the surfaces aligned + // on the operationally-meaningful axis (embed pipeline input). + const cs = opts.contentSanity ?? {}; + if (cs.disabled !== true) { + const operator_literals = cs.junk_patterns_enabled !== false + ? (cs.operator_literals ?? []) + : []; + const sanity = assessContentSanity({ + compiled_truth: parsed.compiled_truth, + timeline: parsed.timeline ?? '', + title: parsed.title, + bytes_warn: cs.bytes_warn, + bytes_block: cs.bytes_block, + extra_literals: operator_literals, + }); + // Rule: huge-page fires for both oversize_warn (over warn threshold) + // AND oversize_block (over block threshold). Operator sees the same + // rule name in both cases; the message names the actual byte count. + if (sanity.reasons.includes('oversize_warn') || sanity.reasons.includes('oversize_block')) { + const threshold = sanity.reasons.includes('oversize_block') ? 'block' : 'warn'; + issues.push({ + file: filePath, line: 1, rule: 'huge-page', + message: `Page body is ${sanity.bytes} bytes (exceeds ${threshold} threshold)`, + fixable: false, + }); + } + // Rule: scraper-junk fires on any built-in pattern or operator literal hit. + // Message names which pattern(s) matched so the brain-author can + // either delete the file from their source repo or audit the scraper. + if (sanity.junk_pattern_matches.length > 0 || sanity.literal_substring_matches.length > 0) { + const matched = [ + ...sanity.junk_pattern_matches, + ...sanity.literal_substring_matches, + ].join(', '); + issues.push({ + file: filePath, line: 1, rule: 'scraper-junk', + message: `Matched junk pattern(s): ${matched}`, + fixable: false, + }); + } + } + return issues; } @@ -205,6 +282,62 @@ export function fixContent(content: string): string { return fixed.trim() + '\n'; } +/** + * Resolve effective content-sanity opts for lint (D1: file/env first, + * lift DB-plane when an engine is reachable). + * + * File/env path is sync via `loadConfig()`; DB-plane lift requires a + * brief engine open. Best-effort: any engine failure (no brain + * configured, connection refused, transient error) falls through to + * the file/env values. CI without `~/.gbrain/` falls through + * immediately since `loadConfig()` returns minimal config. + * + * Also loads the operator literals file (`~/.gbrain/junk-substrings.txt`) + * once per lint invocation so multi-file lint runs amortize the read. + */ +async function resolveLintContentSanity(): Promise { + const base = loadConfig(); + let cs = base?.content_sanity; + + // DB-plane lift: only attempt when the file/env config suggests an + // engine is configured. Avoids spinning up a fresh PGLite just to + // read 4 config keys in a CI lint run that has no brain at all. + const hasEngineConfig = !!(base?.database_url || base?.database_path); + if (hasEngineConfig) { + try { + const { createEngine } = await import('../core/engine-factory.ts'); + const engine = await createEngine({ + engine: base!.engine, + database_url: base!.database_url, + database_path: base!.database_path, + }); + try { + await engine.connect({}); + const lifted = await loadConfigWithEngine(engine, base); + cs = lifted?.content_sanity ?? cs; + } finally { + await engine.disconnect().catch(() => { /* best-effort cleanup */ }); + } + } catch { + // Engine unreachable or failed mid-probe — fall through to + // file/env values. Lint should never block on engine state. + } + } + + // Operator literals: always attempt to load (cheap FS read; missing + // file is the common case and returns []). Skip when kill-switch + // is on or junk patterns explicitly disabled to match the assessor's + // own bypass logic exactly. + const operator_literals = cs?.disabled === true || cs?.junk_patterns_enabled === false + ? [] + : loadOperatorLiterals(); + + return { + ...cs, + operator_literals, + }; +} + /** Collect markdown files from a directory */ function collectPages(dir: string): string[] { const pages: string[] = []; @@ -224,6 +357,10 @@ export interface LintOpts { target: string; fix?: boolean; dryRun?: boolean; + /** v0.41: optional pre-resolved content-sanity opts. When omitted, + * `runLintCore` resolves via the file/env/DB chain. Tests inject + * this directly to bypass the FS + engine layers. */ + contentSanity?: LintContentOpts['contentSanity']; } export interface LintResult { @@ -252,13 +389,19 @@ export async function runLintCore(opts: LintOpts): Promise { const isSingleFile = statSync(opts.target).isFile(); const pages = isSingleFile ? [opts.target] : collectPages(opts.target); + // Resolve content-sanity config once for this lint run (D1: lift DB + // config when reachable). Caller can pre-pass via opts.contentSanity + // (tests, Minion handler) to bypass the engine probe entirely. + const contentSanity = opts.contentSanity ?? await resolveLintContentSanity(); + const lintOpts: LintContentOpts = { contentSanity }; + let totalIssues = 0; let totalFixed = 0; let pagesWithIssues = 0; for (const page of pages) { const content = readFileSync(page, 'utf-8'); - const issues = lintContent(content, isSingleFile ? page : relative(opts.target, page)); + const issues = lintContent(content, isSingleFile ? page : relative(opts.target, page), lintOpts); if (issues.length === 0) continue; pagesWithIssues++; totalIssues += issues.length; @@ -313,10 +456,18 @@ export async function runLint(args: string[]) { const progress = createProgress(cliOptsToProgressOptions(getCliOptions())); progress.start('lint.pages', pages.length); + // v0.41 (D1): resolve content-sanity config once for this lint run. + // Mirrors runLintCore. The two paths must agree because runLint + // prints human details inline; runLintCore at end computes the + // aggregate. Sharing the resolved opts keeps both surfaces seeing + // the same rule firings. + const contentSanity = await resolveLintContentSanity(); + const lintContentOpts: LintContentOpts = { contentSanity }; + for (const page of pages) { const content = readFileSync(page, 'utf-8'); const relPath = isSingleFile ? page : relative(target, page); - const issues = lintContent(content, relPath); + const issues = lintContent(content, relPath, lintContentOpts); progress.tick(1); if (issues.length === 0) continue; @@ -342,7 +493,9 @@ export async function runLint(args: string[]) { // Re-run core for the aggregate counts (cheap; re-parses contents but // produces canonical numbers for the summary line). - const result = await runLintCore({ target, fix: doFix, dryRun }); + // Pass contentSanity through so runLintCore skips its own resolve + // (we already resolved once for the human-detail loop above). + const result = await runLintCore({ target, fix: doFix, dryRun, contentSanity }); console.log(`\n${result.pages_scanned} pages scanned. ${result.total_issues} issue(s) in ${result.pages_with_issues} page(s).`); if (doFix) { console.log(`${dryRun ? '(dry run) ' : ''}${result.total_fixed} auto-fixed.`); diff --git a/src/commands/serve-http.ts b/src/commands/serve-http.ts index 7d2ee1e1e..9a792debc 100644 --- a/src/commands/serve-http.ts +++ b/src/commands/serve-http.ts @@ -844,6 +844,19 @@ export async function runServeHttp(engine: BrainEngine, options: ServeHttpOption res.status(result.status).json(result.body); }); + // v0.41 D2 — live jobs dashboard data. Shares readSnapshot() with the + // TTY `gbrain jobs watch` command so the two surfaces stay 1:1. + app.get('/admin/api/jobs/watch', requireAdmin, async (_req: Request, res: Response) => { + try { + const { readSnapshot } = await import('./jobs-watch.ts'); + const snap = await readSnapshot(engine); + res.json(snap); + } catch (e) { + const msg = e instanceof Error ? e.message : String(e); + res.status(500).json({ error: msg }); + } + }); + // v0.36.1.0 (T15 / E6 / D23) — Calibration tab data endpoints. // Server-rendered SVG charts; admin SPA renders via TrustedSVG wrapper. // v0.36.1.0 (TD3) — pattern drill-down. Returns the source takes that diff --git a/src/commands/sources.ts b/src/commands/sources.ts index f636d8fd5..0a8139c34 100644 --- a/src/commands/sources.ts +++ b/src/commands/sources.ts @@ -876,6 +876,179 @@ async function runCurrent(engine: BrainEngine, args: string[]): Promise { console.log(` tier: ${result.tier}${result.detail ? ` (${result.detail})` : ''}`); } +/** + * v0.41 — `gbrain sources audit ` dry-run scan. + * + * Walks the source's `local_path` on disk, runs `assessContentSanity` + * per `.md` file, and reports: + * - file count + size distribution (p50 / p99 / max) + * - would-hard-blocks (junk-pattern matches; new ingests would refuse) + * - would-soft-blocks (oversize-only; new ingests would set embed_skip) + * - junk-pattern hit counts grouped by pattern name + * + * Read-only: NO DB writes, NO file mutations. Intended for operators to + * inspect a source repo BEFORE syncing (catches junk early) or AFTER + * the new gate ships (audit existing inventory against the new rules + * without touching state). + * + * Uses `pruneDir` from sync.ts so node_modules / .git / .obsidian are + * skipped at descent — same walker semantics as the actual sync path. + */ +async function runAudit(engine: BrainEngine, args: string[]): Promise { + const sourceId = args.find((a) => !a.startsWith('--')); + const json = args.includes('--json'); + const includeWarns = args.includes('--include-warns'); + + if (!sourceId) { + console.error('Usage: gbrain sources audit [--json] [--include-warns]'); + process.exit(2); + } + + const { fetchSource } = await import('../core/sources-load.ts'); + const src = await fetchSource(engine, sourceId); + if (!src) { + console.error(`Source not found: ${sourceId} (run \`gbrain sources list\` to see registered sources)`); + process.exit(1); + } + if (!src.local_path) { + console.error(`Source ${sourceId} has no local_path — cannot audit on disk`); + process.exit(1); + } + + // Lazy-load FS + walker bits so the command stays import-cheap when + // not invoked (every subcommand pays the import cost on dispatch). + const { readFileSync, readdirSync, lstatSync, existsSync: _exists } = + await import('fs'); + const { join: pathJoin } = await import('path'); + const { pruneDir } = await import('../core/sync.ts'); + const { assessContentSanity } = await import('../core/content-sanity.ts'); + const { loadOperatorLiterals } = await import('../core/content-sanity-literals.ts'); + const { parseMarkdown } = await import('../core/markdown.ts'); + + if (!_exists(src.local_path)) { + console.error(`local_path does not exist on disk: ${src.local_path}`); + process.exit(1); + } + + // Walk recursively. Mirror gbrain sync's descent rules so the file set + // we audit matches the file set that would actually be ingested. + const files: string[] = []; + function walk(dir: string): void { + let entries: string[]; + try { + entries = readdirSync(dir); + } catch { + return; // permission denied; skip silently + } + for (const entry of entries) { + const full = pathJoin(dir, entry); + let stat; + try { + stat = lstatSync(full); + } catch { + continue; + } + if (stat.isDirectory()) { + if (pruneDir(entry, dir)) continue; + walk(full); + } else if (entry.endsWith('.md')) { + files.push(full); + } + } + } + walk(src.local_path); + + const literals = loadOperatorLiterals(); + const sizes: number[] = []; + const wouldHardBlock: Array<{ file: string; matched: string[]; bytes: number }> = []; + const wouldSoftBlock: Array<{ file: string; bytes: number }> = []; + const wouldWarn: Array<{ file: string; bytes: number }> = []; + const patternHits: Record = {}; + + for (const file of files) { + let content: string; + try { + content = readFileSync(file, 'utf-8'); + } catch { + continue; + } + let parsed; + try { + parsed = parseMarkdown(content, file); + } catch { + continue; // malformed page; not our concern in audit + } + const sanity = assessContentSanity({ + compiled_truth: parsed.compiled_truth, + timeline: parsed.timeline ?? '', + title: parsed.title, + extra_literals: literals, + }); + sizes.push(sanity.bytes); + if (sanity.shouldHardBlock) { + const matched = [...sanity.junk_pattern_matches, ...sanity.literal_substring_matches]; + for (const name of matched) { + patternHits[name] = (patternHits[name] ?? 0) + 1; + } + wouldHardBlock.push({ file, matched, bytes: sanity.bytes }); + } else if (sanity.shouldSkipEmbed) { + wouldSoftBlock.push({ file, bytes: sanity.bytes }); + } else if (sanity.reasons.includes('oversize_warn')) { + wouldWarn.push({ file, bytes: sanity.bytes }); + } + } + + // Size distribution stats. + sizes.sort((a, b) => a - b); + const p = (q: number) => + sizes.length === 0 ? 0 : sizes[Math.min(sizes.length - 1, Math.floor(q * sizes.length))]; + + if (json) { + console.log(JSON.stringify({ + schema_version: 1, + source_id: sourceId, + local_path: src.local_path, + total_files: files.length, + distribution: { p50: p(0.5), p99: p(0.99), max: sizes[sizes.length - 1] ?? 0 }, + hard_block_count: wouldHardBlock.length, + soft_block_count: wouldSoftBlock.length, + warn_count: wouldWarn.length, + pattern_hits: patternHits, + hard_blocks: wouldHardBlock.slice(0, 20), + soft_blocks: wouldSoftBlock.slice(0, 20), + ...(includeWarns ? { warns: wouldWarn.slice(0, 20) } : {}), + }, null, 2)); + return; + } + + console.log(`Source: ${sourceId} (${src.local_path})`); + console.log(`Files scanned: ${files.length} markdown files`); + if (sizes.length > 0) { + console.log(`Size distribution: p50=${p(0.5)} bytes, p99=${p(0.99)} bytes, max=${sizes[sizes.length - 1]} bytes`); + } + console.log(`Would-hard-block: ${wouldHardBlock.length}`); + console.log(`Would-soft-block: ${wouldSoftBlock.length}`); + if (includeWarns) { + console.log(`Would-warn: ${wouldWarn.length}`); + } + if (Object.keys(patternHits).length > 0) { + const sorted = Object.entries(patternHits).sort((a, b) => b[1] - a[1]); + console.log(`Junk-pattern hits: ${sorted.map(([n, c]) => `${n} ×${c}`).join(', ')}`); + } + if (wouldHardBlock.length > 0) { + console.log('\nTop hard-blocks:'); + for (const h of wouldHardBlock.slice(0, 10)) { + console.log(` ${h.file} [${h.matched.join(', ')}] (${h.bytes}b)`); + } + } + if (wouldSoftBlock.length > 0) { + console.log('\nTop soft-blocks (would write but skip embedding):'); + for (const s of wouldSoftBlock.slice(0, 10)) { + console.log(` ${s.file} (${s.bytes}b)`); + } + } +} + // ── Dispatcher ────────────────────────────────────────────── // v0.40.6.0: my duplicate `runStatus` (line ~895 pre-resolution) was @@ -917,6 +1090,7 @@ export async function runSources(engine: BrainEngine, args: string[]): Promise bytes_warn but neither hard- nor soft-block. + * Page landed normally; stderr was emitted for operator visibility. + * + * Why one stream for all three: + * The doctor check `content_sanity_audit_recent` aggregates by + * reason + source_id over a 7-day window. Splitting events across + * files would force doctor to walk multiple paths or risk dropping + * one. One stream + a discriminator field stays simple. + * + * Best-effort writes. Audit-writer primitive emits stderr on failure + * but never throws — ingest path continues regardless. Documented + * caveat (Codex r1 #14): filesystem JSONL doesn't surface cleanly in + * remote/server deployments. Operators on multi-host setups should + * point `GBRAIN_AUDIT_DIR` at a shared filesystem. Doctor's message + * for `content_sanity_audit_recent` explicitly names this limitation. + * + * Caller contract: the ingest gate calls `logContentSanityAssessment` + * BEFORE branching on hard/soft block so every assessment that does + * something user-visible gets a row. Idempotent re-imports are + * intentionally logged again — the row count over time IS the signal + * (catches "this source keeps producing the same junk"). + */ + +import { createAuditWriter, computeIsoWeekFilename } from './audit-writer.ts'; +import type { ContentSanityResult } from '../content-sanity.ts'; + +export type ContentSanityEventType = 'hard_block' | 'soft_block' | 'warn'; + +export interface ContentSanityAuditEvent { + ts: string; + /** Which kind of assessment fired. */ + event_type: ContentSanityEventType; + /** Page slug that was being imported. */ + slug: string; + /** Source ID — multi-source brains need this for the doctor + * aggregation. Empty string when caller doesn't know (rare). */ + source_id: string; + /** UTF-8 byte length of compiled_truth + timeline at assessment. */ + bytes: number; + /** Names of built-in patterns that matched (empty array on + * soft_block / warn). */ + junk_pattern_matches: string[]; + /** Names of operator literals that matched. */ + literal_substring_matches: string[]; + /** Human-readable reason messages from the assessor result. Embeds + * the PAGE_JUNK_PATTERN / PAGE_OVERSIZED prefix tokens. */ + reason_messages: string[]; + /** When true, the kill-switch was active and this event represents + * a bypass — the page landed regardless. Lets doctor distinguish + * "operator deliberately on a junk-tolerant mode" from "junk + * actually landing." Default false. */ + bypass_active?: boolean; +} + +/** Filename matches the audit-writer's ISO-week convention. */ +export function computeContentSanityAuditFilename(now: Date = new Date()): string { + return computeIsoWeekFilename('content-sanity', now); +} + +const writer = createAuditWriter({ + featureName: 'content-sanity', + errorLabel: 'gbrain', + errorMessagePrefix: 'content-sanity audit ', + errorTrailer: '; import continues', +}); + +/** Classify an assessor result into the audit event type. The same + * result fires different events depending on caller context: a + * hard-block assessment recorded WITH bypass active is still an + * audit-worthy event but the page actually lands. The caller passes + * `bypass` explicitly so this function stays pure. */ +function classifyEventType( + result: ContentSanityResult, + bypass: boolean, +): ContentSanityEventType { + if (bypass) { + // Kill-switch override always logs as warn since the page lands. + // Hard-block + bypass = "would have blocked but operator + // overrode"; soft-block + bypass = same idea. + return 'warn'; + } + if (result.shouldHardBlock) return 'hard_block'; + if (result.shouldSkipEmbed) return 'soft_block'; + return 'warn'; +} + +/** + * Append a content-sanity assessment event. Called from the ingest + * gate before any branch on the assessment result — every assessment + * that does something user-visible gets recorded. + * + * Best-effort: audit-writer primitive stderr-warns on failure but + * never throws. The gate proceeds either way. + */ +export function logContentSanityAssessment( + slug: string, + sourceId: string, + result: ContentSanityResult, + opts: { bypass?: boolean } = {}, +): void { + const bypass = opts.bypass ?? false; + const event_type = classifyEventType(result, bypass); + // Skip rows that don't say anything: bytes under warn threshold AND + // no patterns matched AND no bypass. The assessor result's reasons + // array is empty in that case; we don't want every ingest of a + // normal-size page to write a row. + const hasReasons = result.reasons.length > 0 || result.reason_messages.length > 0; + if (!hasReasons && !bypass) return; + + writer.log({ + event_type, + slug, + source_id: sourceId, + bytes: result.bytes, + junk_pattern_matches: result.junk_pattern_matches, + literal_substring_matches: result.literal_substring_matches, + reason_messages: result.reason_messages, + ...(bypass ? { bypass_active: true } : {}), + }); +} + +/** Read recent events for the doctor `content_sanity_audit_recent` + * check. 7-day default window; reads current + previous ISO week + * files so a window straddling Monday-midnight stays covered. */ +export function readRecentContentSanityEvents( + days = 7, + now: Date = new Date(), +): ContentSanityAuditEvent[] { + return writer.readRecent(days, now); +} + +/** Summarize events for doctor's message. Groups by event_type + + * source_id; counts pattern hits across all events. Returns a stable + * shape so doctor can format consistently. */ +export interface ContentSanitySummary { + total_events: number; + by_type: { hard_block: number; soft_block: number; warn: number }; + by_source: Record; + /** Top junk-pattern names by hit count (sorted desc). */ + top_patterns: Array<{ name: string; count: number }>; +} + +export function summarizeContentSanityEvents( + events: ReadonlyArray, +): ContentSanitySummary { + const by_type = { hard_block: 0, soft_block: 0, warn: 0 }; + const by_source: Record = {}; + const patternCounts: Record = {}; + + for (const ev of events) { + by_type[ev.event_type]++; + by_source[ev.source_id] = (by_source[ev.source_id] ?? 0) + 1; + for (const name of ev.junk_pattern_matches) { + patternCounts[name] = (patternCounts[name] ?? 0) + 1; + } + for (const name of ev.literal_substring_matches) { + patternCounts[name] = (patternCounts[name] ?? 0) + 1; + } + } + + const top_patterns = Object.entries(patternCounts) + .map(([name, count]) => ({ name, count })) + .sort((a, b) => b.count - a.count); + + return { + total_events: events.length, + by_type, + by_source, + top_patterns, + }; +} diff --git a/src/core/brain-writer.ts b/src/core/brain-writer.ts index c2628e77c..ad17f41b9 100644 --- a/src/core/brain-writer.ts +++ b/src/core/brain-writer.ts @@ -487,9 +487,27 @@ export async function scanBrainSources( dbPageCount = null; } else { // Race COUNT against the deadline so a hung query can't eat the budget. + // + // Boundary overshoot (+1ms): the post-await deadline check at line + // ~512 uses `Date.now() >= deadline`. setTimeout fires AT OR AFTER + // the requested delay, so in theory the check always passes. In + // practice on heavily-loaded CI runners (8 parallel shards × 4 + // concurrent test files = ~32 concurrent bun processes) we saw + // intermittent failures where the timer callback resolved + // microseconds BEFORE the wall-clock boundary, leaving Date.now() + // a tick below deadline and the skip-check evaluating false. The + // src-a scan then ran on a populated dir before src-b's + // between-source check caught up — causing + // `firstSource.status === 'skipped'` to receive 'scanned'. + // + // Adding 1ms guarantees the timer fires past the deadline by at + // least one millisecond regardless of runner timer drift. Cost is + // 1ms additional wall-clock latency on hung COUNT queries, which + // is operationally negligible. Flake repro: + // https://github.com/garrytan/gbrain/actions/runs/77611667786 dbPageCount = await Promise.race([ opts.dbPageCountForSource(src.id), - new Promise(resolve => setTimeout(() => resolve(null), remainingMs)), + new Promise(resolve => setTimeout(() => resolve(null), remainingMs + 1)), ]); } } else { diff --git a/src/core/chunkers/code.ts b/src/core/chunkers/code.ts index d2e1ccbd5..1ff1d2efd 100644 --- a/src/core/chunkers/code.ts +++ b/src/core/chunkers/code.ts @@ -75,6 +75,11 @@ import G_RUST from '../../assets/wasm/grammars/tree-sitter-rust.wasm' with { typ import G_SCALA from '../../assets/wasm/grammars/tree-sitter-scala.wasm' with { type: 'file' }; // @ts-ignore import G_SOLIDITY from '../../assets/wasm/grammars/tree-sitter-solidity.wasm' with { type: 'file' }; +// @ts-ignore — DerekStride/tree-sitter-sql @ c2e1e08db1ea20dc23bdb8d228a81a8756e9c450, +// built with tree-sitter-cli@v0.26.3 --abi 14 (matches web-tree-sitter 0.22.6). +// 11 MB; substantially larger than peers because the grammar covers +// PostgreSQL + MySQL + SQLite + T-SQL basics. See CHANGELOG for size notes. +import G_SQL from '../../assets/wasm/grammars/tree-sitter-sql.wasm' with { type: 'file' }; // @ts-ignore import G_SWIFT from '../../assets/wasm/grammars/tree-sitter-swift.wasm' with { type: 'file' }; // @ts-ignore @@ -122,7 +127,7 @@ export type SupportedCodeLanguage = | 'typescript' | 'tsx' | 'javascript' | 'python' | 'ruby' | 'go' | 'rust' | 'java' | 'c_sharp' | 'cpp' | 'c' | 'php' | 'swift' | 'kotlin' | 'scala' | 'lua' | 'elixir' | 'elm' | 'ocaml' | 'dart' | 'zig' | 'solidity' - | 'bash' | 'css' | 'html' | 'vue' | 'json' | 'yaml' | 'toml'; + | 'bash' | 'css' | 'html' | 'vue' | 'json' | 'yaml' | 'toml' | 'sql'; export interface CodeChunkMetadata { symbolName: string | null; @@ -226,6 +231,7 @@ const LANGUAGE_MANIFEST: Record = { json: { displayName: 'JSON', embeddedPath: G_JSON }, yaml: { displayName: 'YAML', embeddedPath: G_YAML }, toml: { displayName: 'TOML', embeddedPath: G_TOML }, + sql: { displayName: 'SQL', embeddedPath: G_SQL }, }; /** @@ -322,6 +328,12 @@ const TOP_LEVEL_TYPES: Partial>> = { elixir: new Set(['call']), bash: new Set(['function_definition', 'variable_assignment']), solidity: new Set(['contract_declaration', 'function_definition', 'modifier_definition', 'event_definition']), + // SQL (DerekStride): every top-level node is `statement`, wrapping a single + // child whose type is the actual kind (create_table, create_function, etc). + // Catch-all `statement` here; extractSymbolName dives into the inner child + // to extract the schema target name (Step 0 inspection 2026-05-24 found + // all 9 fixtures produced `program > statement > ` shape). + sql: new Set(['statement']), }; const BODY_NODE_TYPES = new Set([ @@ -439,6 +451,7 @@ export function detectCodeLanguage(filePath: string, content?: string): Supporte if (lower.endsWith('.json')) return 'json'; if (lower.endsWith('.yaml') || lower.endsWith('.yml')) return 'yaml'; if (lower.endsWith('.toml')) return 'toml'; + if (lower.endsWith('.sql')) return 'sql'; // v0.20.0 Cathedral II Layer 1a fallback hook. Layer 9 (B2 Magika) wires // this in to detect extensionless files (Dockerfile, Makefile, shell // shebangs). try/catch because the fallback may itself fail on first-run @@ -623,7 +636,13 @@ export async function chunkCodeTextFull( // so the header shows the `export` keyword for completeness. const nestableNode = findNestableParent(node, nestedConfig); const symbolName = extractSymbolName(nestableNode ?? node); - const symbolType = normalizeSymbolType((nestableNode ?? node).type); + // For SQL `statement` wrappers, the meaningful type lives on the inner + // child. extractSymbolName already dives in for the name; mirror that + // here so chunk headers say "table users" not "statement users". + const typeNode = (nestableNode ?? node); + const symbolType = (typeNode.type === 'statement' && typeNode.namedChildCount === 1) + ? normalizeSymbolType(typeNode.namedChild(0).type) + : normalizeSymbolType(typeNode.type); if (nestableNode && symbolName && nestedConfig) { const before = chunks.length; @@ -1023,6 +1042,17 @@ function splitLargeNode(node: any, source: string, chunkTarget: number): SplitRa } function extractSymbolName(node: any): string | null { + // SQL (DerekStride): the chunk node is `statement` wrapping a single inner + // child whose type is the actual statement kind. Dive in to find the target + // identifier. DML statements (select/insert/update/delete) deliberately + // return null so their chunks emit unnamed — code-def is a DDL signal. + // The `statement` wrapper is unique to SQL among gbrain's 37 grammars + // (Step 0 inspection 2026-05-24); checking by node.type is safe. + if (node.type === 'statement' && node.namedChildCount === 1) { + const sqlName = extractSqlSymbolName(node.namedChild(0)); + if (sqlName !== undefined) return sqlName; + } + const directName = node.childForFieldName('name'); if (directName?.text?.trim()) return sanitize(directName.text); @@ -1041,6 +1071,45 @@ function extractSymbolName(node: any): string | null { return null; } +// SQL-specific symbol extractor. Returns: +// string — DDL statement: extracted target name (table/function/view/index/etc). +// null — DDL statement type, but name extraction failed (edge fixture). +// undefined — fall through to generic extractor (not a recognized SQL kind). +// +// DerekStride/tree-sitter-sql exposes the target identifier via the `name` +// field on most create_* nodes; `alter_table` puts it in a separate field. +// DML kinds (select/insert/update/delete) deliberately return null — +// gbrain's code-def is a DDL retrieval signal, not a DML one. +function extractSqlSymbolName(inner: any): string | null | undefined { + const t = inner.type; + // DDL: extract identifier name. Tried `name` field first (most common shape), + // then any `object_reference` / `identifier` child. + const DDL_KINDS = new Set([ + 'create_table', 'create_view', 'create_index', 'create_function', + 'create_procedure', 'create_type', 'create_schema', 'create_database', + 'create_trigger', 'alter_table', 'alter_view', + ]); + if (DDL_KINDS.has(t)) { + const nameField = inner.childForFieldName?.('name'); + if (nameField?.text?.trim()) return sanitize(nameField.text); + // Fallback: first identifier-like named child. + for (let i = 0; i < (inner.namedChildCount || 0); i++) { + const c = inner.namedChild(i); + if (c.type === 'object_reference' || c.type === 'identifier' || c.type.endsWith('_identifier')) { + const v = sanitize(c.text); + if (v) return v; + } + } + return null; + } + // DML: explicitly null (chunk emits unnamed; code-def doesn't fire). + if (t === 'select' || t === 'insert' || t === 'update' || t === 'delete' || + t === 'merge' || t === 'with') { + return null; + } + return undefined; +} + function normalizeSymbolType(type: string): string { if (type.includes('function') || type === 'method' || type === 'singleton_method') return 'function'; if (type.includes('class')) return 'class'; @@ -1049,6 +1118,14 @@ function normalizeSymbolType(type: string): string { if (type.includes('enum')) return 'enum'; if (type.includes('module')) return 'module'; if (type.includes('import')) return 'import'; + if (type === 'create_table' || type === 'alter_table') return 'table'; + if (type === 'create_view' || type === 'alter_view') return 'view'; + if (type === 'create_index') return 'index'; + if (type === 'create_procedure') return 'procedure'; + if (type === 'create_type') return 'type'; + if (type === 'create_schema') return 'schema'; + if (type === 'create_database') return 'database'; + if (type === 'create_trigger') return 'trigger'; return type.replace(/_/g, ' '); } diff --git a/src/core/config.ts b/src/core/config.ts index 4fc4e096a..8d5f790c5 100644 --- a/src/core/config.ts +++ b/src/core/config.ts @@ -124,6 +124,33 @@ export interface GBrainConfig { */ search_embedding_column?: string; + /** + * v0.41 content-sanity tunables. Read via file/env/DB plane (D1: lint + * lifts to DB config when reachable). Resolution order: + * env > file > DB > defaults from `src/core/content-sanity.ts`. + * + * Both lint AND ingest go through the same effective resolution so a + * `gbrain config set content_sanity.bytes_block N` flips both surfaces + * uniformly. CI without `~/.gbrain/` falls through to env/defaults. + */ + content_sanity?: { + /** Stderr warn + lint `huge-page` rule fires above this (UTF-8 bytes + * of compiled_truth + timeline). Default: 50_000. Env override: + * `GBRAIN_PAGE_WARN_BYTES`. */ + bytes_warn?: number; + /** Soft-block: page writes with `frontmatter.embed_skip` set but + * embedder skips on next sweep. Default: 500_000. Env override: + * `GBRAIN_PAGE_BLOCK_BYTES`. */ + bytes_block?: number; + /** Master switch for the built-in junk-pattern set. Default: true. + * Env override: `GBRAIN_NO_JUNK_PATTERNS=1` flips to false. */ + junk_patterns_enabled?: boolean; + /** Master kill-switch for all sanity checks. When true, ingest emits + * loud stderr per page but lets everything through. Default: false. + * Env override: `GBRAIN_NO_SANITY=1` flips to true. */ + disabled?: boolean; + }; + /** * Thin-client mode (multi-topology v1). When set, this install does NOT * have a local DB; it talks to a remote `gbrain serve --http` over MCP. @@ -284,6 +311,37 @@ export function loadConfig(): GBrainConfig | null { ? { remote_mcp: { ...fileConfig.remote_mcp, oauth_client_secret: process.env.GBRAIN_REMOTE_CLIENT_SECRET } } : {}), }; + + // v0.41 content-sanity env overrides. Built up as a sparse object so + // env presence wins over file/DB only for the specific keys set, + // matching the precedence pattern used elsewhere in loadConfig. + // The env vars use natural names (GBRAIN_NO_SANITY=1 is more + // operator-friendly than GBRAIN_CONTENT_SANITY_DISABLED=true). + const envContentSanity: GBrainConfig['content_sanity'] = {}; + if (process.env.GBRAIN_PAGE_WARN_BYTES) { + const n = parseInt(process.env.GBRAIN_PAGE_WARN_BYTES, 10); + if (Number.isFinite(n) && n > 0) envContentSanity.bytes_warn = n; + } + if (process.env.GBRAIN_PAGE_BLOCK_BYTES) { + const n = parseInt(process.env.GBRAIN_PAGE_BLOCK_BYTES, 10); + if (Number.isFinite(n) && n > 0) envContentSanity.bytes_block = n; + } + if (process.env.GBRAIN_NO_JUNK_PATTERNS === '1') { + envContentSanity.junk_patterns_enabled = false; + } + if (process.env.GBRAIN_NO_SANITY === '1') { + envContentSanity.disabled = true; + } + // Only attach the field when at least one env var was set, so the + // sparse-merge semantics elsewhere in loadConfigWithEngine work + // (env presence => "this key already has a value, don't read DB"). + if (Object.keys(envContentSanity).length > 0) { + (merged as GBrainConfig).content_sanity = { + ...(fileConfig?.content_sanity ?? {}), + ...envContentSanity, + }; + } + return merged as GBrainConfig; } @@ -381,6 +439,41 @@ export async function loadConfigWithEngine( if (merged.search_embedding_column === undefined && dbSearchEmbeddingColumn !== undefined) { merged.search_embedding_column = dbSearchEmbeddingColumn; } + + // v0.41 content-sanity DB-plane merge (D1: lint lifts to read these + // when reachable). Per-key sparse-merge: env/file wins per individual + // key; DB fills the gaps. The container object is constructed only if + // at least one source provides a value, mirroring the env-merge logic + // in loadConfig(). + async function dbInt(key: string): Promise { + const v = await dbStr(key); + if (v === undefined) return undefined; + const n = parseInt(v, 10); + return Number.isFinite(n) && n > 0 ? n : undefined; + } + const dbWarnBytes = await dbInt('content_sanity.bytes_warn'); + const dbBlockBytes = await dbInt('content_sanity.bytes_block'); + const dbJunkEnabled = await dbBool('content_sanity.junk_patterns_enabled'); + const dbSanityDisabled = await dbBool('content_sanity.disabled'); + + const existingCS = merged.content_sanity ?? {}; + const mergedCS: NonNullable = { ...existingCS }; + if (mergedCS.bytes_warn === undefined && dbWarnBytes !== undefined) { + mergedCS.bytes_warn = dbWarnBytes; + } + if (mergedCS.bytes_block === undefined && dbBlockBytes !== undefined) { + mergedCS.bytes_block = dbBlockBytes; + } + if (mergedCS.junk_patterns_enabled === undefined && dbJunkEnabled !== undefined) { + mergedCS.junk_patterns_enabled = dbJunkEnabled; + } + if (mergedCS.disabled === undefined && dbSanityDisabled !== undefined) { + mergedCS.disabled = dbSanityDisabled; + } + if (Object.keys(mergedCS).length > 0) { + merged.content_sanity = mergedCS; + } + return merged; } @@ -475,6 +568,11 @@ export const KNOWN_CONFIG_KEYS: readonly string[] = [ 'emotional_weight.user_holder', // Cycle phase config 'cycle.grade_takes.write_gstack_learnings', + // Content sanity (v0.41) + 'content_sanity.bytes_warn', + 'content_sanity.bytes_block', + 'content_sanity.junk_patterns_enabled', + 'content_sanity.disabled', // Misc 'artifacts_sync_mode', 'cross_project_learnings', @@ -492,6 +590,7 @@ export const KNOWN_CONFIG_KEY_PREFIXES: readonly string[] = [ 'cycle.', // cycle..* 'embedding_columns.', // per-column overrides 'provider_base_urls.', // per-provider base URL overrides + 'content_sanity.', // v0.41 content-sanity tunables ]; export function saveConfig(config: GBrainConfig): void { diff --git a/src/core/content-sanity-literals.ts b/src/core/content-sanity-literals.ts new file mode 100644 index 000000000..0b203f657 --- /dev/null +++ b/src/core/content-sanity-literals.ts @@ -0,0 +1,151 @@ +/** + * Operator-extensible literal-substring loader for the content-sanity gate. + * + * Reads `~/.gbrain/junk-substrings.txt` (operator-maintained) and returns + * `OperatorLiteral[]` for `assessContentSanity` to evaluate alongside the + * built-in junk patterns. + * + * Why literals, not regex (D16 + Codex r1 #10): + * - JavaScript RegExp has no atomic groups or possessive quantifiers, + * so the conventional ReDoS escape hatch isn't available. A reliable + * catastrophic-backtracking shape detector is hard to implement. + * - Literal substring matching covers the realistic operator use cases + * ("add LinkedIn auth wall" = `"sign in to your account"`; "add + * Reddit blocked" = `"you're being blocked from accessing"`). No + * ReDoS surface. No regex parsing concerns. + * - Built-in patterns stay regex because they're hand-vetted; never + * run the linter against the operator file shape. + * + * Failure handling (D11): + * - Missing file (ENOENT) → return empty list. Operator may not have + * a file; most don't. Silent fall-through to built-ins only. + * - Empty file or all-comments → empty list. Same outcome. + * - Malformed line is structurally impossible: every non-comment line + * is a valid literal substring. Even regex metacharacters in the + * line stay literal at match time (no `new RegExp()` call). + * + * File format: + * - Blank lines and `#`-prefixed comments ignored. + * - Optional directives on the comment line IMMEDIATELY before each + * literal: `# name=...`, `# applies_to=body|title|both`. Directives + * persist until the next literal is read. + * - One literal substring per non-comment line. + * + * Example file: + * + * # name=linkedin_auth_wall + * # applies_to=body + * Sign in to your account to continue + * + * # name=reddit_blocked + * You're being blocked from accessing + * + * # name=substack_paywall + * # applies_to=both + * This post is for paid subscribers + * + * Best-effort: a malformed directive (e.g. `# applies_to=invalid`) + * falls back to the default `'both'` scope without throwing — the + * operator file is a soft input, not a config file. + * + * Default `applies_to` is `'both'` (title AND body head-slice). + * Default `name` when none is declared is `operator_literal_` + * so audit JSONL has a stable identifier even for un-named entries. + */ + +import { existsSync, readFileSync } from 'fs'; +import type { OperatorLiteral } from './content-sanity.ts'; + +/** Path to the operator literals file. Honors `GBRAIN_HOME` via + * `gbrainPath`. Resolved at load time so test fixtures can set + * `GBRAIN_HOME` to a tempdir per the test-isolation conventions in + * CLAUDE.md. */ +function resolveLiteralsPath(): string { + // Lazy-import to avoid loading config.ts surface for the pure + // assessor's consumers that only need built-ins. + const { gbrainPath } = require('./config.ts'); + return gbrainPath('junk-substrings.txt'); +} + +interface ParsedDirective { + name?: string; + applies_to?: 'body' | 'title' | 'both'; +} + +/** Parse one comment line for known directives. Unknown directives + * are ignored (operator file is soft input). Returns empty object + * on no match. */ +function parseDirectiveLine(line: string): ParsedDirective { + const stripped = line.replace(/^#\s*/, '').trim(); + // Match `key=value` shape. Allow multiple per line eventually if + // someone asks; for now one per line is the documented format. + const m = stripped.match(/^([a-z_]+)\s*=\s*(.+)$/i); + if (!m) return {}; + const key = m[1].toLowerCase(); + const value = m[2].trim(); + if (key === 'name') return { name: value }; + if (key === 'applies_to') { + if (value === 'body' || value === 'title' || value === 'both') { + return { applies_to: value }; + } + } + return {}; +} + +/** + * Load operator literals. Pure function over file content — the + * filesystem read is the only side effect. Returns empty list on + * any failure mode (missing, unreadable, empty, all-comments). + * + * Tests pass `content` directly via `parseLiteralsContent` to bypass + * the FS layer. + */ +export function loadOperatorLiterals(path?: string): OperatorLiteral[] { + const resolved = path ?? resolveLiteralsPath(); + if (!existsSync(resolved)) return []; + let raw: string; + try { + raw = readFileSync(resolved, 'utf-8'); + } catch { + // Permission denied, transient FS error — treat as missing. + return []; + } + return parseLiteralsContent(raw); +} + +/** Pure parser exposed for unit tests. */ +export function parseLiteralsContent(raw: string): OperatorLiteral[] { + const literals: OperatorLiteral[] = []; + let pending: ParsedDirective = {}; + let unnamedIndex = 0; + + for (const line of raw.split('\n')) { + const trimmed = line.trim(); + if (trimmed.length === 0) { + // Blank line: directive scope resets so an empty line between + // a directive block and a literal doesn't bind the directives. + // (If you want sticky directives, omit the blank line.) + pending = {}; + continue; + } + if (trimmed.startsWith('#')) { + // Merge directives so a `# name=...` then `# applies_to=...` + // pair both bind to the next literal. + const parsed = parseDirectiveLine(trimmed); + pending = { ...pending, ...parsed }; + continue; + } + // Non-comment, non-blank → literal substring line. + const name = pending.name ?? `operator_literal_${unnamedIndex++}`; + literals.push({ + name, + substring: trimmed, + applies_to: pending.applies_to ?? 'both', + }); + // Consume the pending directives so they don't bind to a + // subsequent literal unless re-declared. + pending = {}; + } + + return literals; +} diff --git a/src/core/content-sanity.ts b/src/core/content-sanity.ts new file mode 100644 index 000000000..3005e45e4 --- /dev/null +++ b/src/core/content-sanity.ts @@ -0,0 +1,316 @@ +/** + * Content-sanity assessor for the ingest narrow waist. + * + * Pure module — no engine I/O, no filesystem access. Consumed by: + * - `src/core/import-file.ts` — wires the gate into `importFromContent` + * so EVERY ingestion path inherits it (sync, gbrain import, put_page + * MCP op, gbrain capture, POST /ingest webhook via ingest_capture). + * - `src/commands/lint.ts` — surfaces matching content as `huge-page` + * + `scraper-junk` lint rules so brain-authors see issues in their + * source repo before sync. + * - `src/commands/doctor.ts` — surfaces historical inventory via + * `oversized_pages`, `scraper_junk_pages`, and + * `content_sanity_audit_recent` checks. + * - `src/commands/sources.ts` `audit` subcommand — dry-run scan of a + * source repo's `local_path` reporting would-blocks + size + * distribution without touching the DB. + * + * Two failure modes treated differently (D14-D16 + D6-D9 review trail): + * - **Scraper junk** (built-in pattern OR operator literal match) → + * HARD-BLOCK. Caller is expected to `throw new ContentSanityBlockError(...)`. + * Existing exception-handling at every wrapper site (import.ts/cli.ts, + * operations.ts put_page, sync.ts:929 catch) fires correctly through + * this single throw point. No new status vocabulary required. + * - **Oversize alone** (bytes > block_bytes WITHOUT junk-pattern match) → + * SOFT-BLOCK. Caller writes the page with `frontmatter.embed_skip` set + * via `buildEmbedSkipMarker` from `src/core/embed-skip.ts`. The embedder + * skips on next sweep at all 5 wiring sites. Page lands so legitimate + * large content (2MB conversation transcripts) is preserved. + * + * Bytes are measured against `compiled_truth + timeline` (the parsed body + * after `parseMarkdown` splits at the timeline sentinel). Frontmatter is + * NOT counted — the operational concern is the embed-pipeline-input size. + * Codex r2 #7 caught the earlier compiled_truth-only design that missed + * pages with huge timeline sections. + * + * Pattern set is hand-vetted regex evaluated against `title` + the first + * ~2KB of body content. 6 built-in patterns (D3 dropped a shape-based + * `empty_body_with_source_url` rule because legitimate stub pages with + * `source_url` frontmatter were getting flagged). Operator literals come + * in via `extra_literals` from `src/core/content-sanity-literals.ts` + * (literal substrings only — no regex per Codex r1 #10 ReDoS concerns). + * + * The kill-switch (`GBRAIN_NO_SANITY=1` / `content_sanity.disabled: true`) + * is honored by the CALLER (import-file.ts), not by this module. The + * assessor stays pure so unit tests don't need env mutation. + */ + +/** Maximum number of body bytes scanned for pattern matches. The body + * is sliced to this size before regex/substring evaluation so pattern + * cost stays O(2KB) regardless of page size. Cloudflare/CAPTCHA junk + * pages have their telltale text at the top — 2KB covers the realistic + * cases. Operators who need deeper scanning can override via env. */ +export const SCAN_HEAD_BYTES = 2048; + +/** Default warn threshold. Operator override via + * `content_sanity.bytes_warn` config key or `GBRAIN_PAGE_WARN_BYTES` + * env var. Above this, lint surfaces `huge-page` rule + ingest emits + * stderr warn. Page still writes. */ +export const DEFAULT_BYTES_WARN = 50_000; + +/** Default block threshold. Operator override via + * `content_sanity.bytes_block` config key or `GBRAIN_PAGE_BLOCK_BYTES` + * env var. Above this, page writes but `frontmatter.embed_skip` is set + * and the embedder skips on next sweep. Page is still queryable; just + * not searchable until manually re-embedded or split. */ +export const DEFAULT_BYTES_BLOCK = 500_000; + +/** Tag added to the start of `reasons` and to error messages so + * `src/core/sync.ts:classifyErrorCode` can group hard-blocks under one + * code without needing a structured field in the failure shape. The + * classifier matches this token via regex. */ +export const PAGE_JUNK_PATTERN_CODE = 'PAGE_JUNK_PATTERN'; + +export type SanityTripReason = + | 'oversize_warn' // informational: bytes > bytes_warn but page lands normally + | 'oversize_block' // soft-block: write with frontmatter.embed_skip + | 'junk_pattern' // hard-block: throw ContentSanityBlockError + | 'literal_substring'; // hard-block: operator-supplied literal hit + +export interface JunkPattern { + /** Stable identifier surfaced in error messages, audit JSONL, and + * doctor output. Snake_case. Treat as a stable contract — renaming + * one means rewriting downstream consumers. */ + name: string; + /** Case-insensitive regex. Evaluated against the chosen scope; cost + * is bounded by SCAN_HEAD_BYTES. */ + pattern: RegExp; + /** Where the pattern applies. Defaults to 'both' (title AND body + * head-slice). 'title' is useful for error-page-title detection; + * 'body' for content-shape patterns. */ + applies_to?: 'body' | 'title' | 'both'; +} + +export interface OperatorLiteral { + name: string; + /** Literal substring. Case-insensitive match via `.toLowerCase()`. + * Regex meta-characters in the substring are matched literally. */ + substring: string; + applies_to?: 'body' | 'title' | 'both'; +} + +export interface ContentSanityResult { + /** UTF-8 byte length of `compiled_truth + timeline`. Frontmatter is + * NOT included (the operational concern is embed-pipeline input). */ + bytes: number; + /** True when bytes > effective bytes_block. Drives soft-block. */ + oversize: boolean; + /** Names of built-in patterns that matched (zero or more). */ + junk_pattern_matches: string[]; + /** Names of operator literals that matched (zero or more). */ + literal_substring_matches: string[]; + /** Ordered list of trip reasons. `oversize` first when present, + * then `junk_pattern`, then `literal_substring`. Stable across + * releases so consumers can pattern-match. */ + reasons: SanityTripReason[]; + /** Human-readable messages per reason. Each prefixed with the stable + * code token (`PAGE_JUNK_PATTERN:` or `PAGE_OVERSIZED:`) so the + * caller can compose them into an error message that `classifyErrorCode` + * picks up via regex. */ + reason_messages: string[]; + /** True when any junk pattern or operator literal matched. Caller + * should throw `ContentSanityBlockError` when this is set. Note that + * oversize alone does NOT trigger this — that's a soft-block. */ + shouldHardBlock: boolean; + /** True when oversize without hard-block. Caller should write the + * page with `frontmatter.embed_skip` set so the embedder skips. */ + shouldSkipEmbed: boolean; +} + +/** Built-in pattern set. Hand-vetted regex compiled once at module + * load. Adding a pattern: include a stable `name`, a case-insensitive + * regex with `i` flag, and document the real-world example in plain + * prose so future reviewers know what shape it catches. */ +export const BUILT_IN_JUNK_PATTERNS: ReadonlyArray = Object.freeze([ + // Cloudflare interstitials — the dominant scraper-junk class. + { + name: 'cloudflare_attention_required', + pattern: /attention required.*cloudflare/i, + applies_to: 'both', + }, + { + name: 'cloudflare_just_a_moment', + // Both signals required — "just a moment..." alone fires on + // legitimate writing; the cdn-cgi/challenge URL is the discriminator. + pattern: /just a moment\.\.\.[\s\S]{0,500}cdn-cgi\/challenge-platform/i, + applies_to: 'body', + }, + { + name: 'cloudflare_ray_id', + pattern: /cloudflare ray id:/i, + applies_to: 'body', + }, + // Generic 403 / blocked-access pages. + { + name: 'access_denied', + pattern: /^\s*access denied\b/im, + applies_to: 'both', + }, + // CAPTCHA gates. + { + name: 'captcha_required', + pattern: /verify you are (a )?human|captcha required|please complete the security check/i, + applies_to: 'both', + }, + // Bare error-page titles. Anchored so the title is exclusively the + // error code — a thoughtful page ABOUT 404 errors won't trip. + { + name: 'error_page_title', + pattern: /^(403|404|500|502|503|error \d{3}|page not found)\s*$/i, + applies_to: 'title', + }, +]); + +/** Tagged error thrown from `importFromContent` on hard-block. The + * existing exception-handling at every wrapper site catches it and + * surfaces a non-zero exit (import), MCP error envelope (put_page), + * or sync-failure record. Message embeds `PAGE_JUNK_PATTERN:` so + * `classifyErrorCode` picks it up via regex without needing a + * structured `error_code` field on `ImportResult`. */ +export class ContentSanityBlockError extends Error { + readonly code = PAGE_JUNK_PATTERN_CODE; + readonly result: ContentSanityResult; + + constructor(result: ContentSanityResult) { + // Compose message from the result's reason messages. The + // `PAGE_JUNK_PATTERN:` prefix is already in each reason_message + // so the classifier regex hits regardless of which reasons fired. + const summary = result.reason_messages.join('; '); + super(`Content rejected by sanity gate: ${summary}`); + this.name = 'ContentSanityBlockError'; + this.result = result; + } +} + +/** + * Assess a parsed page against the size + junk-pattern surface. + * + * Pure function — same inputs always produce the same outputs. Caller + * decides what to do with the result (throw on shouldHardBlock, set + * embed_skip frontmatter on shouldSkipEmbed, write normally otherwise). + * + * The body bytes input is `compiled_truth + timeline` (Codex r2 #7 + * fix: pages can have huge timeline sections that would evade a + * compiled_truth-only check). Frontmatter is NOT counted. + */ +export function assessContentSanity(opts: { + /** Post-parseMarkdown body (before timeline split). */ + compiled_truth: string; + /** Post-parseMarkdown timeline section (empty string if no sentinel). */ + timeline: string; + /** Post-parseMarkdown title. Some patterns key on title alone. */ + title: string; + /** Effective warn threshold; defaults to DEFAULT_BYTES_WARN. */ + bytes_warn?: number; + /** Effective block threshold; defaults to DEFAULT_BYTES_BLOCK. */ + bytes_block?: number; + /** Operator-supplied literal substrings loaded from + * `~/.gbrain/junk-substrings.txt` via `src/core/content-sanity-literals.ts`. + * Empty array (default) means built-ins only. */ + extra_literals?: ReadonlyArray; +}): ContentSanityResult { + const bytes_warn = opts.bytes_warn ?? DEFAULT_BYTES_WARN; + const bytes_block = opts.bytes_block ?? DEFAULT_BYTES_BLOCK; + + // Bytes measured against the parsed body (compiled_truth + timeline). + // Buffer.byteLength counts UTF-8 bytes the same way the doctor's + // octet_length() does at the DB layer, so the two surfaces agree on + // the same page (D2 parity). + const body = opts.compiled_truth + (opts.timeline ? '\n' + opts.timeline : ''); + const bytes = Buffer.byteLength(body, 'utf-8'); + const oversize = bytes > bytes_block; + + // Head-slice for pattern evaluation. Cost stays O(SCAN_HEAD_BYTES) + // regardless of body size. Lowercased once so substring matching + // doesn't repeat the lowercase per literal. + const bodyHead = body.slice(0, SCAN_HEAD_BYTES); + const bodyHeadLower = bodyHead.toLowerCase(); + const titleLower = opts.title.toLowerCase(); + + const junk_pattern_matches: string[] = []; + for (const p of BUILT_IN_JUNK_PATTERNS) { + const scope = p.applies_to ?? 'both'; + let matched = false; + if (scope === 'title' || scope === 'both') { + if (p.pattern.test(opts.title)) matched = true; + } + if (!matched && (scope === 'body' || scope === 'both')) { + if (p.pattern.test(bodyHead)) matched = true; + } + if (matched) junk_pattern_matches.push(p.name); + } + + const literal_substring_matches: string[] = []; + if (opts.extra_literals && opts.extra_literals.length > 0) { + for (const lit of opts.extra_literals) { + const scope = lit.applies_to ?? 'both'; + const needle = lit.substring.toLowerCase(); + if (needle.length === 0) continue; + let matched = false; + if (scope === 'title' || scope === 'both') { + if (titleLower.includes(needle)) matched = true; + } + if (!matched && (scope === 'body' || scope === 'both')) { + if (bodyHeadLower.includes(needle)) matched = true; + } + if (matched) literal_substring_matches.push(lit.name); + } + } + + const reasons: SanityTripReason[] = []; + const reason_messages: string[] = []; + const shouldHardBlock = + junk_pattern_matches.length > 0 || literal_substring_matches.length > 0; + + // Reason ordering: block-level oversize first (so a soft-block that + // ALSO hits a junk pattern documents both), then junk_pattern, then + // literal. Warn-level oversize emitted only when no block-level fired. + if (oversize) { + reasons.push('oversize_block'); + reason_messages.push(`PAGE_OVERSIZED: body ${bytes} bytes exceeds ${bytes_block} byte block threshold`); + } else if (bytes > bytes_warn) { + // Warn tier: bytes between bytes_warn and bytes_block. Page lands + // normally; consumer emits stderr and (when configured) lint surfaces + // `huge-page` rule. This row IS auditable so doctor's recent-events + // check can surface flow-rate signal ("operators crossing warn often"). + reasons.push('oversize_warn'); + reason_messages.push(`PAGE_OVERSIZE_WARN: body ${bytes} bytes exceeds ${bytes_warn} byte warn threshold`); + } + if (junk_pattern_matches.length > 0) { + reasons.push('junk_pattern'); + reason_messages.push( + `${PAGE_JUNK_PATTERN_CODE}: matched built-in pattern(s): ${junk_pattern_matches.join(', ')}`, + ); + } + if (literal_substring_matches.length > 0) { + reasons.push('literal_substring'); + reason_messages.push( + `${PAGE_JUNK_PATTERN_CODE}: matched operator literal(s): ${literal_substring_matches.join(', ')}`, + ); + } + + return { + bytes, + oversize, + junk_pattern_matches, + literal_substring_matches, + reasons, + reason_messages, + // shouldSkipEmbed: oversize past block threshold but NOT also hard-block. + // When BOTH fire (the 890K Cloudflare dump case), hard-block wins and + // the page never lands. Embed-skip is reserved for the legitimate + // large-content case. + shouldHardBlock, + shouldSkipEmbed: oversize && !shouldHardBlock, + }; +} diff --git a/src/core/db-lock.ts b/src/core/db-lock.ts index 08706255c..07bcef990 100644 --- a/src/core/db-lock.ts +++ b/src/core/db-lock.ts @@ -283,6 +283,48 @@ async function engineSelectOne(engine: BrainEngine): Promise { throw new Error(`Unknown engine kind for heartbeat: ${engine.kind}`); } +/** + * v0.41 Eng D9 (codex pass-2 #7 + #8) — per-tick election convenience. + * + * Thin wrapper over `tryAcquireDbLock` for the E5 lease-cap controller + * use case: each worker ticks every 30s and tries to acquire the + * controller lock; the winner runs `fn` (read fleet signal, write new + * lease cap), then releases. Losers no-op for this tick; next tick + * re-elects. + * + * The codex pass-3 #8 + #9 audit confirmed this should reuse the + * existing `gbrain_cycle_locks` table (which `tryAcquireDbLock` already + * wraps for both engines) rather than build a parallel new primitive. + * + * Semantics: + * - Returns the result of `fn` on lock acquisition. + * - Returns `null` when another worker holds the lock (not an error; + * just "not my tick"). + * - `fn` throws → release lock cleanly + rethrow. + * + * For long-running work that needs mid-flight TTL refresh, use + * `withRefreshingLock` instead. This helper is for sub-second / single- + * statement work where the initial TTL covers the whole call. + */ +export async function tryWithDbElection( + engine: BrainEngine, + lockId: string, + ttlMinutes: number, + fn: () => Promise, +): Promise { + const handle = await tryAcquireDbLock(engine, lockId, ttlMinutes); + if (!handle) return null; + try { + return await fn(); + } finally { + try { + await handle.release(); + } catch { + /* idempotent — lock will auto-expire under TTL */ + } + } +} + /** * Compose a multi-tenant-safe lock id (cherry D4). Suffixes the lock id * with the database name so two gbrain installs sharing a Postgres cluster diff --git a/src/core/db.ts b/src/core/db.ts index 531f0d316..57049d394 100644 --- a/src/core/db.ts +++ b/src/core/db.ts @@ -188,6 +188,11 @@ export async function connect(config: EngineConfig): Promise { // Register pgvector type bigint: postgres.BigInt, }, + // Silence postgres NOTICE-level messages by default ("relation already + // exists, skipping" floods stdout under idempotent CREATE statements + // during migrations + initSchema, and breaks stdout-parsing callers like + // `gbrain jobs submit --json | ...`). Opt back in with GBRAIN_PG_NOTICES=1. + onnotice: process.env.GBRAIN_PG_NOTICES === '1' ? undefined : () => {}, }; if (Object.keys(timeouts).length > 0) { opts.connection = timeouts; diff --git a/src/core/embed-skip.ts b/src/core/embed-skip.ts new file mode 100644 index 000000000..6e34c8caf --- /dev/null +++ b/src/core/embed-skip.ts @@ -0,0 +1,129 @@ +/** + * Embed-skip predicate: the single source of truth for "should this + * page be skipped during embedding?" + * + * Why a shared module (D4): + * gbrain has 5 sites that filter the stale-chunk / all-pages query + * for embedding: + * + * 1. src/commands/embed.ts:350 (--stale CLI path) + * 2. src/commands/embed.ts:355 (--all CLI path) — D8 catches this + * too; the `--all` walk re-embeds every page from scratch and + * must honor the skip flag like `--stale` does. + * 3. src/core/embed-stale.ts:90 (Minion helper) + * 4. src/core/postgres-engine.ts (listStaleChunks/countStaleChunks) + * 5. src/core/pglite-engine.ts equivalent + * + * Inline-filtering across 5 sites is the exact bug class gbrain has + * been bitten by repeatedly — see CLAUDE.md `cjk.ts`, `sql-ranking.ts`, + * `audit-writer.ts` for sibling shared modules. Extracting the + * predicate here means the 5 sites all import from one place. + * + * Two surfaces: + * - JS predicate `isEmbedSkipped(frontmatter)` for callers that have + * in-memory page objects (CLI walk paths). + * - SQL fragment `EMBED_SKIP_FILTER_FRAGMENT` for callers that need + * to splice into a postgres-js / PGLite `sql\`...\`` template. + * Both engines use the standard JSONB `?` existence operator; + * PGLite (PostgreSQL 17.5 in WASM) supports the full JSONB + * operator set, so one fragment works for both. + * + * Frontmatter writer: + * - `buildEmbedSkipMarker(bytes)` produces the canonical marker + * object. Callers `Object.assign` it onto `parsed.frontmatter` so + * it persists into the page write. Stable schema means the JS + * predicate and the SQL existence check both target the same key + * name (`embed_skip`) — drift between writer and reader is the + * bug class we're preventing. + * + * Marker shape rationale: + * The marker is an OBJECT (not a bare bool) so the operator can see + * WHY the page was skipped + WHEN at a glance via `get_page`. The + * SQL existence check (`frontmatter ? 'embed_skip'`) hits regardless + * of marker contents — JSONB key-existence semantics — so future + * versions can extend the marker shape without invalidating the + * filter. + * + * v0.42 follow-up: promote to schema column `pages.embed_skipped_at` + * + partial index. Single change site (this module). For v0.41 the + * JSONB approach is acceptable because the skipped-page subset stays + * small (operator surfaces via doctor and either splits or accepts). + */ + +/** The frontmatter key name. Treat as a stable contract — renaming + * this means rewriting every consumer of the skip semantic. */ +export const EMBED_SKIP_KEY = 'embed_skip'; + +/** SQL fragment that excludes pages with the embed-skip marker. + * Callers must already JOIN `pages` (aliased as `p`) — the bare + * `content_chunks` query has no access to frontmatter and needs the + * join added regardless. + * + * Use via `sql.unsafe()` or equivalent fragment-splice: + * + * const filter = EMBED_SKIP_FILTER_FRAGMENT; + * await sql`SELECT ... FROM content_chunks cc + * JOIN pages p ON p.id = cc.page_id + * WHERE cc.embedding IS NULL AND ${sql.unsafe(filter)}`; + * + * The fragment uses the JSONB `?` existence operator: returns true + * when the JSONB object contains the key `'embed_skip'` at the top + * level. Works identically on Postgres (real) and PGLite (PostgreSQL + * 17.5 in WASM). The `NOT` negates so we KEEP rows that DON'T have + * the marker. */ +export const EMBED_SKIP_FILTER_FRAGMENT = + `NOT (COALESCE(p.frontmatter, '{}'::jsonb) ? '${EMBED_SKIP_KEY}')`; + +export interface EmbedSkipMarker { + /** Why the page was skipped. v0.41 ships only `'oversized'`; future + * reasons (e.g. `'chunk_token_limit'` from the deferred v0.42 + * chunk-level quarantine) extend this enum. */ + reason: 'oversized'; + /** Body bytes at the time of assessment. Operator visibility: at a + * glance, see how oversized the page is. */ + bytes: number; + /** ISO 8601 timestamp at assessment time. Tells the operator when + * the skip was first applied (page may have been edited later). */ + assessed_at: string; +} + +/** Build the canonical marker object. Callers spread it onto the + * frontmatter before write: + * + * parsed.frontmatter[EMBED_SKIP_KEY] = buildEmbedSkipMarker(bytes); + * + * The marker is OBJECT-shaped (not bare true) so `get_page` shows + * the operator why + when at a glance. */ +export function buildEmbedSkipMarker(bytes: number, now: Date = new Date()): EmbedSkipMarker { + return { + reason: 'oversized', + bytes, + assessed_at: now.toISOString(), + }; +} + +/** JS-side predicate for in-memory page objects. Returns true when the + * frontmatter has the embed-skip key set to any non-null value. + * + * Accepts `null`/`undefined` frontmatter (some paths construct page + * objects without one) and returns false — no frontmatter means no + * skip marker. + * + * Mirrors the SQL fragment's semantics: key-existence is the trigger; + * marker contents are diagnostic, not functional. A future marker + * shape change doesn't break this predicate. */ +export function isEmbedSkipped(frontmatter: Record | null | undefined): boolean { + if (!frontmatter) return false; + const value = frontmatter[EMBED_SKIP_KEY]; + return value !== undefined && value !== null; +} + +/** JS-side filter for arrays of in-memory page objects. Returns a new + * array with embed-skipped pages excluded. Mirrors the SQL filter + * for callers that walk pages JS-side (e.g. `gbrain embed --all` + * walks pages directly rather than going through listStaleChunks). */ +export function filterOutEmbedSkipped | null }>( + pages: ReadonlyArray, +): T[] { + return pages.filter((p) => !isEmbedSkipped(p.frontmatter ?? null)); +} diff --git a/src/core/import-file.ts b/src/core/import-file.ts index f28306f9e..9d7c92ce0 100644 --- a/src/core/import-file.ts +++ b/src/core/import-file.ts @@ -15,6 +15,11 @@ import { computeEffectiveDate } from './effective-date.ts'; import { MARKDOWN_CHUNKER_VERSION } from './chunkers/recursive.ts'; import { logSlugFallback } from './audit-slug-fallback.ts'; import { resolveContextualRetrievalMode } from './contextual-retrieval-resolver.ts'; +import { assessContentSanity, ContentSanityBlockError } from './content-sanity.ts'; +import { loadOperatorLiterals } from './content-sanity-literals.ts'; +import { logContentSanityAssessment } from './audit/content-sanity-audit.ts'; +import { isEmbedSkipped, buildEmbedSkipMarker, EMBED_SKIP_KEY } from './embed-skip.ts'; +import { loadConfig, loadConfigWithEngine } from './config.ts'; import { buildContextualPrefix, modeRequiresHaiku, @@ -268,6 +273,112 @@ export async function importFromContent( const parsed = parseMarkdown(content, slug + '.md', { activePack: opts.activePack }); + // v0.41 content-sanity gate. Runs AFTER parseMarkdown so the assessor + // sees the parsed body (compiled_truth + timeline), title, and + // frontmatter; runs BEFORE the hash compute so a soft-block that + // mutates frontmatter (sets `embed_skip`) reaches the existing hash + // calculation and the page write doesn't short-circuit on hash equality. + // + // Three outcomes: + // - kill-switch active (`content_sanity.disabled === true` / + // `GBRAIN_NO_SANITY=1`) → assess + audit with bypass flag, emit + // loud stderr per offending ingest, but let everything through. + // - hard-block (junk pattern OR operator literal) → THROW + // ContentSanityBlockError. Existing exception flow at every + // wrapper site (import.ts errors counter, put_page MCP envelope, + // sync.ts:929 failure record) fires correctly through this single + // throw point. classifyErrorCode picks up the PAGE_JUNK_PATTERN + // prefix in the error message and groups in sync-failures.jsonl. + // - soft-block (oversize WITHOUT junk-pattern hit) → mutate + // frontmatter to embed `embed_skip` marker. Existing chunking + // block guards on `isEmbedSkipped(frontmatter)` so chunks stays + // empty; the existing `tx.deleteChunks` at the empty-chunks + // branch fires to purge old chunks (D9 transition invariant). + // + // Effective config: env > file > DB > defaults. The DB-plane lift + // adds ~4 SQL round-trips per import (one per content_sanity.* key); + // acceptable for the per-page cost since the gate runs at most once + // per ingest. Power-users with 10K-file syncs who care about this + // overhead can set the keys via env vars instead and skip the DB read. + { + const baseCfg = loadConfig(); + let effectiveCfg = baseCfg; + try { + // loadConfigWithEngine merges DB-plane content_sanity.* on top + // of file/env. Wrapped in try/catch so a transient engine error + // doesn't kill the import — the gate falls back to file/env + // values (which include defaults via the assessor itself). + effectiveCfg = await loadConfigWithEngine(engine, baseCfg); + } catch (err) { + const msg = err instanceof Error ? err.message : String(err); + process.stderr.write(`[gbrain] content-sanity: DB config lift failed (${msg}); falling back to file/env\n`); + } + const cs = effectiveCfg?.content_sanity ?? {}; + // GBRAIN_NO_SANITY=1 fast-path: loadConfig() returns null when + // there's no `~/.gbrain/config.json` AND no DATABASE_URL env var + // (e.g., fresh PGLite-only setups, hermetic tests). The merged + // content_sanity block never carries `disabled` in that case. Read + // the kill-switch env directly so it works regardless of whether + // any other config plumbing fired. Same direct-env-check pattern + // applies to the patterns_enabled flip below. + const sanityDisabled = + cs.disabled === true || process.env.GBRAIN_NO_SANITY === '1'; + const extra_literals = + cs.junk_patterns_enabled !== false && !sanityDisabled ? loadOperatorLiterals() : []; + const sanityResult = assessContentSanity({ + compiled_truth: parsed.compiled_truth, + timeline: parsed.timeline ?? '', + title: parsed.title, + bytes_warn: cs.bytes_warn, + bytes_block: cs.bytes_block, + extra_literals, + }); + // Audit BEFORE branching so hard-block / soft-block / warn / bypass + // ALL get a row in the JSONL. The audit module's own gate + // suppresses no-op rows (bytes below warn, no patterns, no bypass). + logContentSanityAssessment(slug, sourceId ?? 'default', sanityResult, { + bypass: sanityDisabled, + }); + + if (sanityDisabled) { + // Kill-switch active: loud stderr per offending ingest. Operator + // explicitly opted into the bypass and gets noisy feedback every + // time it fires so they remember the gate is off. + if (sanityResult.shouldHardBlock || sanityResult.shouldSkipEmbed) { + process.stderr.write( + `[gbrain] content-sanity bypass (GBRAIN_NO_SANITY=1): ${slug} — ${sanityResult.reason_messages.join('; ')}\n`, + ); + } + } else { + if (sanityResult.shouldHardBlock) { + // Single throw point. Existing exception flow at every wrapper + // site fires correctly. Caller-side semantics: + // - import.ts → runImport's catch increments errors → non-zero exit + // - put_page MCP → operations.ts try/catch → OperationError envelope + // - sync.ts → existing catch at :929 → records failure with classified code + throw new ContentSanityBlockError(sanityResult); + } + if (sanityResult.shouldSkipEmbed) { + // Soft-block: mutate frontmatter so the embed_skip marker + // persists into the page write. The existing chunking block + // below guards on isEmbedSkipped → chunks stays empty → + // existing tx.deleteChunks fires to purge old chunks + // (D9 transition invariant — old chunks were searchable + // against stale content; deleting them maintains the + // invariant that embed_skip means "no live chunks"). + parsed.frontmatter[EMBED_SKIP_KEY] = buildEmbedSkipMarker(sanityResult.bytes); + process.stderr.write( + `[gbrain] content-sanity soft-block: ${slug} (${sanityResult.bytes} bytes) — page lands, embedding skipped\n`, + ); + } else if (sanityResult.reasons.includes('oversize_warn')) { + // Warn tier: page lands normally; lint surface picks up too. + process.stderr.write( + `[gbrain] content-sanity warn: ${slug} (${sanityResult.bytes} bytes) — exceeds warn threshold, consider splitting\n`, + ); + } + } + } + // v0.39.3.0 CV8 — DB content_hash excludes timestamp-bearing frontmatter // keys so identical body content from `gbrain capture` (which stamps // `captured_at` and `ingested_at` per call) produces a stable hash. @@ -314,24 +425,32 @@ export async function importFromContent( return { slug, status: 'skipped', chunks: 0, parsedPage }; } - // Chunk compiled_truth and timeline + // Chunk compiled_truth and timeline. + // v0.41 content-sanity soft-block: if the gate marked this page as + // embed-skipped (oversize without junk-pattern), skip chunking + // entirely. The empty-chunks branch in the transaction below + // triggers tx.deleteChunks(slug) which purges any pre-existing + // chunks (D9 transition invariant: embed_skip means no live chunks). const chunks: ChunkInput[] = []; - if (parsed.compiled_truth.trim()) { - for (const c of chunkText(parsed.compiled_truth)) { - chunks.push({ chunk_index: chunks.length, chunk_text: c.text, chunk_source: 'compiled_truth' }); + const embedSkipped = isEmbedSkipped(parsed.frontmatter); + if (!embedSkipped) { + if (parsed.compiled_truth.trim()) { + for (const c of chunkText(parsed.compiled_truth)) { + chunks.push({ chunk_index: chunks.length, chunk_text: c.text, chunk_source: 'compiled_truth' }); + } } - } - if (parsed.timeline?.trim()) { - for (const c of chunkText(parsed.timeline)) { - chunks.push({ chunk_index: chunks.length, chunk_text: c.text, chunk_source: 'timeline' }); + if (parsed.timeline?.trim()) { + for (const c of chunkText(parsed.timeline)) { + chunks.push({ chunk_index: chunks.length, chunk_text: c.text, chunk_source: 'timeline' }); + } } - } - // v0.20.0 Cathedral II Layer 8 D2 — extract fenced code blocks from - // compiled_truth as first-class code chunks. - if (parsed.compiled_truth.trim()) { - const fenceChunks = await extractFencedChunks(parsed.compiled_truth, chunks.length); - chunks.push(...fenceChunks); + // v0.20.0 Cathedral II Layer 8 D2 — extract fenced code blocks from + // compiled_truth as first-class code chunks. + if (parsed.compiled_truth.trim()) { + const fenceChunks = await extractFencedChunks(parsed.compiled_truth, chunks.length); + chunks.push(...fenceChunks); + } } // Embed BEFORE the transaction (external API call). diff --git a/src/core/migrate.ts b/src/core/migrate.ts index e337ade96..d773b868c 100644 --- a/src/core/migrate.ts +++ b/src/core/migrate.ts @@ -4257,6 +4257,93 @@ export const MIGRATIONS: Migration[] = [ WHERE config ? 'github_repo'; `, }, + { + version: 93, + name: 'minions_v0_41_audit_and_budget', + // v0.41 minions cathedral — three audit tables + three new columns on + // minion_jobs. Single migration because the audit tables and budget + // columns are jointly designed and consumed: + // + // - minion_lease_pressure_log ← Bug 2 (releaseLeaseFullJob writes here) + // - minion_budget_log ← D5 (reservation / refund / halt / lost events) + // - minion_self_fix_log ← E6 (classifier-gated auto-resubmit chain) + // - minion_jobs.budget_remaining_cents ← D5 (parent spendable balance) + // - minion_jobs.budget_owner_job_id ← Eng D7 (immutable budget owner; FK SET NULL) + // - minion_jobs.budget_root_owner_id ← Eng D10 (denormalized historical + // owner, NO FK — persists past owner deletion so children can + // disambiguate "never had a budget" from "owner deleted, halt cleanly"). + // + // Audit table FKs are ON DELETE SET NULL (codex pass-2 #5) so audit rows + // survive `gbrain jobs prune`. Each audit table denormalizes context + // (queue_name, model, owner_id, event_type, etc.) at write time so + // post-NULL rows still carry forensic value — without denormalization + // they'd be timestamp-only residue (codex pass-3 #7). + // + // The retention sweep that bounds audit-table growth (Eng D8) lives in + // the autopilot cycle's `purge` phase, not here. This migration just + // creates the schema; the sweep ships in the same wave but is its own + // code path. + sql: ` + CREATE TABLE IF NOT EXISTS minion_lease_pressure_log ( + id BIGSERIAL PRIMARY KEY, + job_id BIGINT NULL REFERENCES minion_jobs(id) ON DELETE SET NULL, + lease_key TEXT NOT NULL, + active_at_bounce INTEGER NOT NULL, + max_concurrent INTEGER NOT NULL, + bounced_at TIMESTAMPTZ NOT NULL DEFAULT now(), + queue_name TEXT NULL, + job_name TEXT NULL, + model TEXT NULL, + provider TEXT NULL, + root_owner_id BIGINT NULL + ); + CREATE INDEX IF NOT EXISTS minion_lease_pressure_log_recent_idx + ON minion_lease_pressure_log (bounced_at DESC); + CREATE INDEX IF NOT EXISTS minion_lease_pressure_log_job_idx + ON minion_lease_pressure_log (job_id); + + CREATE TABLE IF NOT EXISTS minion_budget_log ( + id BIGSERIAL PRIMARY KEY, + job_id BIGINT NULL REFERENCES minion_jobs(id) ON DELETE SET NULL, + owner_id BIGINT NULL, + event_type TEXT NOT NULL, + cents_delta INTEGER NOT NULL, + turn_index INTEGER NULL, + model TEXT NULL, + occurred_at TIMESTAMPTZ NOT NULL DEFAULT now() + ); + CREATE INDEX IF NOT EXISTS minion_budget_log_owner_idx + ON minion_budget_log (owner_id); + CREATE INDEX IF NOT EXISTS minion_budget_log_recent_idx + ON minion_budget_log (occurred_at DESC); + + CREATE TABLE IF NOT EXISTS minion_self_fix_log ( + id BIGSERIAL PRIMARY KEY, + parent_id BIGINT NULL REFERENCES minion_jobs(id) ON DELETE SET NULL, + child_id BIGINT NULL REFERENCES minion_jobs(id) ON DELETE SET NULL, + classifier_bucket TEXT NOT NULL, + chain_depth INTEGER NOT NULL, + policy_applied TEXT NULL, + outcome TEXT NULL, + occurred_at TIMESTAMPTZ NOT NULL DEFAULT now() + ); + CREATE INDEX IF NOT EXISTS minion_self_fix_log_parent_idx + ON minion_self_fix_log (parent_id); + CREATE INDEX IF NOT EXISTS minion_self_fix_log_recent_idx + ON minion_self_fix_log (occurred_at DESC); + + ALTER TABLE minion_jobs ADD COLUMN IF NOT EXISTS budget_remaining_cents INTEGER NULL; + ALTER TABLE minion_jobs ADD COLUMN IF NOT EXISTS budget_owner_job_id BIGINT NULL + REFERENCES minion_jobs(id) ON DELETE SET NULL; + ALTER TABLE minion_jobs ADD COLUMN IF NOT EXISTS budget_root_owner_id BIGINT NULL; + CREATE INDEX IF NOT EXISTS minion_jobs_budget_owner_idx + ON minion_jobs (budget_owner_job_id) + WHERE budget_owner_job_id IS NOT NULL; + CREATE INDEX IF NOT EXISTS minion_jobs_budget_root_owner_idx + ON minion_jobs (budget_root_owner_id) + WHERE budget_root_owner_id IS NOT NULL; + `, + }, ]; export const LATEST_VERSION = MIGRATIONS.length > 0 @@ -4474,14 +4561,16 @@ export async function runMigrations(engine: BrainEngine): Promise<{ applied: num return { applied: 0, current }; } - console.log(` Schema version ${current} → ${LATEST_VERSION} (${pending.length} migration(s) pending)`); + // Progress messages route to stderr so callers parsing stdout (e.g. + // `gbrain jobs submit --json | jq`) aren't polluted by migration noise. + process.stderr.write(` Schema version ${current} → ${LATEST_VERSION} (${pending.length} migration(s) pending)\n`); // Pre-flight: warn about connections that might block DDL await checkForBlockingConnections(engine); let applied = 0; for (const m of pending) { - console.log(` [${m.version}] ${m.name}...`); + process.stderr.write(` [${m.version}] ${m.name}...\n`); // Pick SQL: engine-specific `sqlFor` wins over engine-agnostic `sql`. const sql = m.sqlFor?.[engine.kind] ?? m.sql; @@ -4555,7 +4644,7 @@ export async function runMigrations(engine: BrainEngine): Promise<{ applied: num // Update version after both SQL and handler succeed await engine.setConfig('version', String(m.version)); - console.log(` [${m.version}] ✓ ${m.name}`); + process.stderr.write(` [${m.version}] ✓ ${m.name}\n`); applied++; } diff --git a/src/core/minions/batch-projection.ts b/src/core/minions/batch-projection.ts new file mode 100644 index 000000000..970488595 --- /dev/null +++ b/src/core/minions/batch-projection.ts @@ -0,0 +1,187 @@ +/** + * v0.41 D4 — submit-time cost + duration projection for batch jobs. + * + * Closes the "wake up to a $300 Anthropic bill" failure mode at the source. + * Before `gbrain jobs submit` queues a batch, this projection prints + * "this batch will take ~30 min and cost ~$2.40 at current cap of 32". + * TTY users get a confirmation prompt above the configurable threshold; + * cron / non-TTY callers see the projection on stderr and proceed. + * + * Math is deliberately rough — heavy-tailed (codex pass-2 #11) so we + * carry a ±30% confidence band rather than pretending precision. + * + * Cold-start fallback (no historical jobs for the model/name): model- + * default per-token pricing + 5s mean latency estimate. Operators see + * `(no history; estimate is a wide guess)` so they don't trust the + * number more than it deserves. + * + * Unknown model: returns the `cost_estimate_unavailable` tagged variant + * with the model name. The `--budget-usd` flag refuses to gate against + * an unavailable cost estimate; same precedent as cross-modal-eval. + */ + +import { ANTHROPIC_PRICING } from '../anthropic-pricing.ts'; + +export interface RecentJobStats { + /** How many jobs informed this window. 0 → cold start. */ + sample_size: number; + /** Mean wall-clock duration per job. ms. Undefined when cold start. */ + mean_latency_ms?: number; + /** Mean USD cost per job. Undefined when cold start OR cost unavailable. */ + mean_cost_usd?: number; + /** Standard deviation of cost per job. Used for ±band. */ + stddev_cost_usd?: number; + /** Effective concurrency seen recently (min of worker pool + lease cap). */ + effective_concurrency: number; + /** Lease cap headroom; informational. */ + lease_headroom?: number; +} + +export interface BatchProjection { + /** Total wall-clock estimate in ms. */ + total_duration_ms: number; + /** Total cost estimate USD. NaN-safe; null when unknown model. */ + total_cost_usd: number | null; + /** ±USD band (≈30% of cost OR derived from sample stddev). */ + cost_band_usd: number | null; + /** ±ms band on duration. */ + duration_band_ms: number; + /** Concurrency used for the math. */ + effective_concurrency: number; + /** True when no historical data was available; estimate is a wide guess. */ + cold_start: boolean; + /** When non-null, model name that has no pricing entry. */ + unknown_model?: string; + /** Suggested cap-raise (informational; printed in stderr). */ + raise_cap_hint?: string; +} + +/** Strip `provider:` prefix the same way the SDK call site does. */ +function bareModel(model: string): string { + const idx = model.indexOf(':'); + return idx > 0 ? model.slice(idx + 1) : model; +} + +/** + * Resolve per-token cost for a model. Returns null for unknown models. + * Conservative: uses output-side pricing as a tight upper bound when + * we don't have per-call usage stats yet. + */ +function modelDefaultMeanCostUsd(model: string): number | null { + // Match the alias map's behavior loosely: bare names + the few we know. + const bare = bareModel(model); + const p = ANTHROPIC_PRICING[bare]; + if (!p) return null; + // Assume a typical subagent turn: ~2k input + ~1k output tokens. + // Heavy bound; real avgs will be smaller for simple jobs, larger for + // tool-loop heavy jobs. The cold-start `(no history)` annotation tells + // the operator the number is approximate. + const COLD_INPUT = 2000; + const COLD_OUTPUT = 1000; + return (COLD_INPUT * p.input + COLD_OUTPUT * p.output) / 1_000_000; +} + +const COLD_LATENCY_MS = 5000; + +export interface ProjectBatchInput { + job_count: number; + /** `provider:model` or bare model id. Used for pricing lookup. */ + model: string; + stats: RecentJobStats; + /** Current rate-lease cap; used for the raise-cap hint. */ + current_lease_cap: number; +} + +export function projectBatch(input: ProjectBatchInput): BatchProjection { + const { job_count, model, stats, current_lease_cap } = input; + const bare = bareModel(model); + const cold = stats.sample_size === 0; + + // Mean latency: historical → use it. Cold → 5s guess. + const meanLatencyMs = stats.mean_latency_ms ?? COLD_LATENCY_MS; + + // Mean cost: historical → use it. Cold → model-default guess. Unknown + // model → return the tagged variant. + let meanCostUsd: number | null = stats.mean_cost_usd ?? modelDefaultMeanCostUsd(model); + if (meanCostUsd === null && !(bare in ANTHROPIC_PRICING)) { + // Unknown model — projection lacks a cost surface. + const concurrency = Math.max(1, Math.min(stats.effective_concurrency, current_lease_cap)); + return { + total_duration_ms: Math.ceil((job_count / concurrency) * meanLatencyMs), + total_cost_usd: null, + cost_band_usd: null, + duration_band_ms: Math.ceil(meanLatencyMs * 0.5), + effective_concurrency: concurrency, + cold_start: cold, + unknown_model: bare, + }; + } + if (meanCostUsd === null) meanCostUsd = 0; // defensive; shouldn't reach + + // Effective concurrency is min of worker pool seen recently + lease cap. + const concurrency = Math.max(1, Math.min(stats.effective_concurrency, current_lease_cap)); + + const totalDurationMs = Math.ceil((job_count / concurrency) * meanLatencyMs); + const totalCostUsd = job_count * meanCostUsd; + + // Confidence band: when we have stddev, use it (×1.96 ≈ 95%). Cold + // start or no stddev → ±30% blanket. + const costBand = stats.stddev_cost_usd + ? job_count * stats.stddev_cost_usd * 1.96 + : totalCostUsd * 0.30; + const durationBand = totalDurationMs * 0.30; + + // Raise-cap hint when lease cap is the binding constraint. If raising + // the cap by 4x would meaningfully shrink wall-clock, suggest it. + let raise_cap_hint: string | undefined; + if (stats.lease_headroom !== undefined && stats.lease_headroom <= 0.1 && current_lease_cap < 128) { + const suggestedCap = Math.min(current_lease_cap * 4, 128); + const projectedFasterMs = Math.ceil((job_count / Math.min(suggestedCap, stats.effective_concurrency)) * meanLatencyMs); + if (projectedFasterMs < totalDurationMs * 0.75) { + raise_cap_hint = + `raise GBRAIN_ANTHROPIC_MAX_INFLIGHT to ${suggestedCap} to finish in ~${Math.ceil(projectedFasterMs / 60_000)}min`; + } + } + + return { + total_duration_ms: totalDurationMs, + total_cost_usd: totalCostUsd, + cost_band_usd: costBand, + duration_band_ms: durationBand, + effective_concurrency: concurrency, + cold_start: cold, + raise_cap_hint, + }; +} + +/** + * Format a projection as a human-readable single line for stderr / TTY. + * Matches the style of the doctor + jobs stats output: concise, with + * confidence band when meaningful, with the paste-ready hint when + * applicable. + */ +export function formatProjection(p: BatchProjection): string { + const mins = Math.max(1, Math.ceil(p.total_duration_ms / 60_000)); + const minsBand = Math.ceil(p.duration_band_ms / 60_000); + if (p.unknown_model) { + return `[batch] est duration ~${mins}min (±${minsBand}min) at concurrency=${p.effective_concurrency}; cost estimate unavailable (model "${p.unknown_model}" not in pricing maps).`; + } + const dollars = (p.total_cost_usd ?? 0).toFixed(2); + const dollarsBand = (p.cost_band_usd ?? 0).toFixed(2); + const coldNote = p.cold_start ? ' (no history; estimate is a wide guess)' : ''; + const hint = p.raise_cap_hint ? ` — ${p.raise_cap_hint}` : ''; + return `[batch] est cost ~$${dollars} (±$${dollarsBand}), est duration ~${mins}min (±${minsBand}min) at concurrency=${p.effective_concurrency}${coldNote}${hint}`; +} + +/** + * Threshold gating: should the TTY prompt for confirmation? + * Defaults: prompt above $5 OR 30min. Overridable via env vars + * matching the plan's spec. + */ +export function shouldPromptAtThreshold(p: BatchProjection): boolean { + const usdThreshold = Number(process.env.GBRAIN_BATCH_PROMPT_THRESHOLD_USD ?? '5'); + const minThreshold = Number(process.env.GBRAIN_BATCH_PROMPT_THRESHOLD_MIN ?? '30'); + const usdOverThreshold = (p.total_cost_usd ?? 0) >= usdThreshold; + const minutesOverThreshold = p.total_duration_ms >= minThreshold * 60_000; + return usdOverThreshold || minutesOverThreshold; +} diff --git a/src/core/minions/budget-tracker.ts b/src/core/minions/budget-tracker.ts new file mode 100644 index 000000000..d1de135b2 --- /dev/null +++ b/src/core/minions/budget-tracker.ts @@ -0,0 +1,291 @@ +/** + * v0.41 D5 — `--budget-usd N` enforcement via reservation pattern (Eng D7 + D10). + * + * Closes the "wake up to a $300 Anthropic bill" failure mode all the way: + * D4 (batch-projection) shows the cost UP FRONT; D5 enforces a hard + * ceiling at runtime. Together they make "fire-and-forget batches" a + * real promise, not honor system. + * + * Reservation pattern (Eng D7, eng-review pass 2 — codex caught CAS-only + * could overspend across N parallel children of one budget owner): + * + * 1. Worker calls `reserveBudget(ownerId, expectedMaxTurnCostCents)` + * BEFORE each turn. + * 2. SQL UPDATE with CAS: `WHERE budget_remaining_cents >= cost + * RETURNING balance`. On miss → BudgetExhausted thrown. + * 3. On turn return → `refundBudget(ownerId, unspentCents)`. + * + * Even with N concurrent children, parallel CAS UPDATEs serialize at the + * row level — the worst-case overspend is bounded by the per-turn + * reservation amount (NOT N × turn-cost as CAS-only would be). + * + * Eng D10 NULL-bypass (codex pass-3 #3 + #4): jobs without a budget + * owner skip reservation entirely. Disambiguation via the immutable + * `budget_root_owner_id` denormalized column — when the FK + * `budget_owner_job_id` is NULL but `budget_root_owner_id` is set, the + * owner was DELETED (not "never had a budget"); throw `BudgetOwnerDeleted` + * so the child halts cleanly. + * + * Recursive halt (codex pass-2 #3): when owner balance hits 0, the worker + * walks `WHERE budget_owner_job_id = X AND status IN ('waiting','delayed')` + * to flip the entire subtree to `dead` with reason `budget_exhausted`. + * + * Audit (Eng D8 / codex pass-3 #7): every reserve/refund/lost/halted + * event writes one row to `minion_budget_log` with denormalized model + + * owner context so post-prune forensic queries still work. + */ + +import type { BrainEngine } from '../engine.ts'; + +/** Status of a reservation attempt. */ +export type ReservationOutcome = + | { kind: 'reserved'; new_balance_cents: number; reserved_cents: number } + | { kind: 'exhausted'; balance_at_attempt: number; requested_cents: number } + | { kind: 'no_budget' } // job has no owner; bypass entirely (Eng D10) + | { kind: 'owner_deleted'; root_owner_id: number }; // pruned mid-batch + +/** Throw to signal budget exhaustion to the worker / subagent handler. */ +export class BudgetExhausted extends Error { + constructor(public owner_id: number, public balance_at_attempt_cents: number) { + super(`budget owner ${owner_id} exhausted: balance ${balance_at_attempt_cents} cents`); + this.name = 'BudgetExhausted'; + } +} + +/** Throw when the budget owner row was deleted mid-batch (Eng D10). */ +export class BudgetOwnerDeleted extends Error { + constructor(public root_owner_id: number) { + super(`budget owner ${root_owner_id} was deleted; child cannot continue safely`); + this.name = 'BudgetOwnerDeleted'; + } +} + +/** What we learn about a job's budget ownership at reservation time. */ +export interface BudgetOwnerInfo { + job_id: number; + budget_owner_job_id: number | null; + budget_root_owner_id: number | null; + budget_remaining_cents: number | null; +} + +/** + * Look up budget ownership for a job. Returns 'no_budget' when neither + * the FK nor the denormalized root is set. Returns 'owner_deleted' when + * the denormalized root is set but the FK is NULL (owner was pruned). + */ +export async function getBudgetOwner( + engine: BrainEngine, + jobId: number, +): Promise { + const rows = await engine.executeRaw( + `SELECT id AS job_id, budget_owner_job_id, budget_root_owner_id, budget_remaining_cents + FROM minion_jobs WHERE id = $1`, + [jobId], + ); + return rows[0] ?? null; +} + +/** + * Set the budget on the OWNER job (the parent). Called when an operator + * submits `gbrain agent run ... --budget-usd N`. Sets budget_remaining_cents + * on the owner row AND populates budget_root_owner_id to its own id (self- + * reference) so subsequent children inherit it. + */ +export async function setOwnerBudget( + engine: BrainEngine, + ownerJobId: number, + budgetUsd: number, +): Promise { + const cents = Math.round(budgetUsd * 100); + await engine.executeRaw( + `UPDATE minion_jobs + SET budget_remaining_cents = $2, + budget_owner_job_id = $1, + budget_root_owner_id = $1 + WHERE id = $1`, + [ownerJobId, cents], + ); +} + +/** + * Inherit budget ownership at child-submit time. Call from the path + * that submits a child of a budget-bearing parent (or grandparent). The + * child's budget_owner_job_id + budget_root_owner_id mirror the parent's, + * not the parent's own id. Same-row UPDATE; cheap. + * + * Children DO NOT copy budget_remaining_cents — only the owner row holds + * spendable balance. Codex pass-3 #5 caught the contradiction in the + * original plan that suggested copying balance. + */ +export async function inheritBudgetOwner( + engine: BrainEngine, + childJobId: number, + parentJobId: number, +): Promise { + await engine.executeRaw( + `UPDATE minion_jobs SET + budget_owner_job_id = (SELECT COALESCE(budget_owner_job_id, NULL) FROM minion_jobs WHERE id = $2), + budget_root_owner_id = (SELECT COALESCE(budget_root_owner_id, NULL) FROM minion_jobs WHERE id = $2) + WHERE id = $1`, + [childJobId, parentJobId], + ); +} + +/** + * Reserve `expected_max_turn_cost_cents` from the budget owner BEFORE + * the turn runs. CAS guarantees no overspend: parallel reservations + * serialize at the row level. + * + * Eng D10 NULL-bypass: jobs without an owner return 'no_budget' and + * the worker SHOULD proceed without budget gating. + * + * Eng D10 deleted-owner disambiguation: when FK is NULL but the + * denormalized root is set, return 'owner_deleted' and the worker + * SHOULD halt cleanly. + */ +export async function reserveBudget( + engine: BrainEngine, + childJobId: number, + expectedMaxTurnCostCents: number, +): Promise { + const info = await getBudgetOwner(engine, childJobId); + if (!info) { + // Job row vanished — shouldn't happen mid-claim but defensive. + return { kind: 'no_budget' }; + } + + // Eng D10 disambiguation. + if (info.budget_owner_job_id === null) { + if (info.budget_root_owner_id !== null) { + return { kind: 'owner_deleted', root_owner_id: info.budget_root_owner_id }; + } + return { kind: 'no_budget' }; + } + + const rows = await engine.executeRaw<{ budget_remaining_cents: number }>( + `UPDATE minion_jobs + SET budget_remaining_cents = budget_remaining_cents - $2 + WHERE id = $1 AND budget_remaining_cents >= $2 + RETURNING budget_remaining_cents`, + [info.budget_owner_job_id, expectedMaxTurnCostCents], + ); + if (rows.length === 0) { + // CAS miss — owner exhausted. Read current balance for the audit row. + const peek = await engine.executeRaw<{ budget_remaining_cents: number }>( + `SELECT budget_remaining_cents FROM minion_jobs WHERE id = $1`, + [info.budget_owner_job_id], + ); + const balance = peek[0]?.budget_remaining_cents ?? 0; + await logBudgetEvent(engine, { + job_id: childJobId, + owner_id: info.budget_owner_job_id, + event_type: 'halted', + cents_delta: 0, + }); + return { kind: 'exhausted', balance_at_attempt: balance, requested_cents: expectedMaxTurnCostCents }; + } + + await logBudgetEvent(engine, { + job_id: childJobId, + owner_id: info.budget_owner_job_id, + event_type: 'reserved', + cents_delta: -expectedMaxTurnCostCents, + }); + return { kind: 'reserved', new_balance_cents: rows[0]!.budget_remaining_cents, reserved_cents: expectedMaxTurnCostCents }; +} + +/** + * Refund unspent cents AFTER the turn returns. Pass the unspent amount + * (reserved - actual). Defensive: caller can pass 0 with no harm. + */ +export async function refundBudget( + engine: BrainEngine, + childJobId: number, + ownerJobId: number, + refundCents: number, +): Promise { + if (refundCents <= 0) return; + await engine.executeRaw( + `UPDATE minion_jobs SET budget_remaining_cents = budget_remaining_cents + $2 + WHERE id = $1`, + [ownerJobId, refundCents], + ); + await logBudgetEvent(engine, { + job_id: childJobId, + owner_id: ownerJobId, + event_type: 'refunded', + cents_delta: refundCents, + }); +} + +/** + * Recursive halt sweep — codex pass-2 #3. When the owner is exhausted, + * walk `budget_owner_job_id = X` and flip every waiting/delayed + * descendant to dead. Active jobs DO NOT get flipped here; their + * turn-boundary reserveBudget call will throw BudgetExhausted on the + * next turn (or, if mid-tool-dispatch, they complete the current turn + * cleanly and the next reservation fails). + * + * Returns the count of jobs halted. + */ +export async function haltBudgetSubtree( + engine: BrainEngine, + ownerJobId: number, + reason: 'budget_exhausted' | 'owner_deleted', +): Promise { + // Exclude the owner row itself — setOwnerBudget sets the owner's + // budget_owner_job_id to its own id (self-referencing so children can + // inherit via parent.budget_owner_job_id), which means the owner row + // would match its own subtree filter. The owner has its own state + // machine independent of budget halt; we just halt its DESCENDANTS. + const rows = await engine.executeRaw<{ id: number }>( + `UPDATE minion_jobs SET + status = 'dead', + error_text = $2, + finished_at = now(), + updated_at = now() + WHERE budget_owner_job_id = $1 + AND id != $1 + AND status IN ('waiting', 'delayed') + RETURNING id`, + [ownerJobId, `${reason}: parent ${ownerJobId} hit cap or was deleted`], + ); + return rows.length; +} + +/** Audit table row shape. Mirrors the columns from migration v93. */ +export interface BudgetEventRecord { + job_id: number; + owner_id: number; + event_type: 'reserved' | 'refunded' | 'spent' | 'lost' | 'halted' | 'owner_deleted'; + cents_delta: number; + turn_index?: number; + model?: string; +} + +/** Best-effort write to minion_budget_log. Mirror of lease-pressure-audit. */ +export async function logBudgetEvent( + engine: BrainEngine, + record: BudgetEventRecord, +): Promise { + try { + await engine.executeRaw( + `INSERT INTO minion_budget_log + (job_id, owner_id, event_type, cents_delta, turn_index, model) + VALUES ($1, $2, $3, $4, $5, $6)`, + [ + record.job_id, + record.owner_id, + record.event_type, + record.cents_delta, + record.turn_index ?? null, + record.model ?? null, + ], + ); + } catch (err) { + const msg = err instanceof Error ? err.message : String(err); + process.stderr.write( + `[budget-tracker] WARN: audit write failed for job ${record.job_id}: ${msg}\n`, + ); + } +} diff --git a/src/core/minions/error-classify.ts b/src/core/minions/error-classify.ts new file mode 100644 index 000000000..7ff0b6e50 --- /dev/null +++ b/src/core/minions/error-classify.ts @@ -0,0 +1,143 @@ +/** + * v0.41 D3 + E6 — shared error classifier for minion jobs. + * + * Reads `minion_jobs.last_error` (or any equivalent error string) and + * returns a stable bucket. Two consumers: + * + * 1. **D3 error clustering** — `gbrain jobs stats --cluster-errors` and + * `gbrain jobs get --cluster ` group dead/failed jobs by bucket + * so "92 jobs failed" becomes "92 jobs failed: 89 rate_lease_full, + * 3 prompt_too_long" — the operator instantly knows the right fix. + * + * 2. **E6 self-fix** — `RECOVERABLE_CLUSTERS` lists the buckets that + * qualify for classifier-gated auto-resubmit. Narrowed per codex + * pass-2 #4: `tool_error` was too broad (catches crashes, schema + * mismatches, permission errors, real bugs). Only the safe sub-types + * qualify: + * + * prompt_too_long — semantic-aware reduction can fix this + * tool_schema_mismatch — model passed bad args; retry with error msg + * malformed_json — model emitted bad JSON; "JSON only" retry + * + * Explicitly NOT recoverable: tool_crash (real bug in tool impl), + * tool_unavailable (registry config issue), tool_permission + * (capability decision; needs human). + * + * Mirrors the pattern at `src/core/eval-contradictions/judge-errors.ts`. + * Conservative: defaults to 'unknown' when no regex matches, so callers + * always get a valid bucket (no nulls / undefined). + */ + +export type ErrorCluster = + | 'prompt_too_long' // Anthropic 400 "prompt is too long" + | 'tool_schema_mismatch' // model called tool with invalid arg shape + | 'tool_crash' // tool.execute threw an Error (real bug) + | 'tool_unavailable' // tool not in registry for this subagent + | 'tool_permission' // tool refused (e.g. put_page slug not allowed) + | 'malformed_json' // model output failed JSON parse where required + | 'auth' // 401 / API key invalid + | 'rate_limit' // 429 from upstream + | 'rate_lease_full' // gbrain's internal RateLeaseUnavailableError + | 'timeout' // local timeout / abort + | 'http_5xx' // upstream 5xx + | 'context_canceled' // worker abort signal fired + | 'unknown'; + +/** + * Self-fix RECOVERABLE_CLUSTERS — narrowed per codex pass-2 #4. + * Only these buckets trigger E6 auto-resubmit; everything else routes + * through normal dead-letter so real bugs stay visible. + */ +export const RECOVERABLE_CLUSTERS = new Set([ + 'prompt_too_long', + 'tool_schema_mismatch', + 'malformed_json', +]); + +/** + * Classify a `last_error` string into a stable bucket. NULL / empty + * input returns 'unknown'. Conservative — defaults to 'unknown' rather + * than guess; lets D3 surface "unknown: N jobs" as a real signal that + * the classifier set needs widening. + */ +export function classifyJobError(lastError: string | null | undefined): ErrorCluster { + if (!lastError) return 'unknown'; + const msg = lastError.toLowerCase(); + + // gbrain-internal errors first (most specific). + if (/rate lease ".*" full/i.test(lastError)) return 'rate_lease_full'; + + // Anthropic 400 prompt too long. + if (/prompt is too long/i.test(lastError) || /context.*length/i.test(lastError)) { + return 'prompt_too_long'; + } + + // Tool error sub-types. Order matters: more-specific patterns first. + if (/tool ".*" is not (in the registry|available)/i.test(lastError)) { + return 'tool_unavailable'; + } + if (/tool ".*" (permission|forbidden|denied|not allowed)/i.test(lastError)) { + return 'tool_permission'; + } + if (/(invalid|malformed|missing) (input|argument|param|schema|field)/i.test(lastError) || + /tool_use validation/i.test(lastError) || + /required.*missing/i.test(lastError)) { + return 'tool_schema_mismatch'; + } + if (/tool ".*" (failed|crashed|threw)/i.test(lastError) || + /tool.execute.*error/i.test(lastError)) { + return 'tool_crash'; + } + + // JSON shape errors. + if (/(parse|invalid|malformed).*json/i.test(lastError) || + /expected json/i.test(lastError) || + /unexpected token.*in json/i.test(lastError)) { + return 'malformed_json'; + } + + // HTTP / upstream errors. + if (msg.includes('401') || msg.includes('unauthorized') || /api[- _]?key.*invalid/i.test(lastError)) { + return 'auth'; + } + if (msg.includes('429') || /rate[- _]?limit/i.test(lastError) || /too many requests/i.test(lastError)) { + return 'rate_limit'; + } + if (/\b50[0-9]\b/.test(msg) || msg.includes('bad gateway') || msg.includes('service unavailable') || + msg.includes('gateway timeout') || msg.includes('overloaded')) { + return 'http_5xx'; + } + + // Local / control-plane signals. + if (msg.includes('timeout') || msg.includes('timed out') || msg.includes('aborted: timeout')) { + return 'timeout'; + } + if (msg.includes('aborted: cancel') || msg.includes('signal aborted') || msg.includes('context canceled')) { + return 'context_canceled'; + } + + return 'unknown'; +} + +/** + * Cluster a list of error strings into bucket → count. Stable order: + * counts descending, then bucket name ascending. Includes 'unknown' if + * it shows up so operators see "N unknown" as a signal to widen the + * classifier. + */ +export function clusterErrors( + errors: Array<{ id: number; last_error: string | null }>, +): Array<{ cluster: ErrorCluster; count: number; sample_ids: number[] }> { + const map = new Map(); + for (const e of errors) { + const c = classifyJobError(e.last_error); + const bucket = map.get(c) ?? { count: 0, sample_ids: [] }; + bucket.count += 1; + // Carry up to 3 sample ids per bucket for `--cluster ` lookup. + if (bucket.sample_ids.length < 3) bucket.sample_ids.push(e.id); + map.set(c, bucket); + } + return Array.from(map.entries()) + .map(([cluster, { count, sample_ids }]) => ({ cluster, count, sample_ids })) + .sort((a, b) => (b.count - a.count) || a.cluster.localeCompare(b.cluster)); +} diff --git a/src/core/minions/handlers/subagent.ts b/src/core/minions/handlers/subagent.ts index 5dda236aa..fcead52f3 100644 --- a/src/core/minions/handlers/subagent.ts +++ b/src/core/minions/handlers/subagent.ts @@ -48,6 +48,7 @@ import { logSubagentHeartbeat, } from './subagent-audit.ts'; import { resolveModel, isAnthropicProvider, TIER_DEFAULTS } from '../../model-config.ts'; +import { buildSystemPrompt, DEFAULT_SUBAGENT_SYSTEM } from '../system-prompt.ts'; import { toolLoop as gatewayToolLoop } from '../../ai/gateway.ts'; import type { ChatToolDef, ChatMessage, ChatBlock, ChatResult, ToolHandler } from '../../ai/gateway.ts'; import { classifyCapabilities } from '../../ai/capabilities.ts'; @@ -58,9 +59,36 @@ import { randomUUIDv7 } from 'bun'; const DEFAULT_MODEL = 'claude-sonnet-4-6'; const DEFAULT_MAX_TURNS = 20; const DEFAULT_RATE_KEY = 'anthropic:messages'; -const DEFAULT_MAX_CONCURRENT = Number(process.env.GBRAIN_ANTHROPIC_MAX_INFLIGHT ?? '8'); + +/** + * Resolve the rate-lease cap from the env var. + * + * undefined → 32 (default; was 8 pre-v0.41, starved 10-concurrency batches) + * "unlimited" → POSITIVE_INFINITY (Azure / Bedrock / self-hosted with no upstream cap) + * "none" → POSITIVE_INFINITY (alias) + * positive number → that number + * anything else → throws (NaN / "0" / negative / typo — fail loud, NOT silent uncap) + * + * Codex pass-1 #7 caught the original `=0` and `NaN` silently uncapping; + * "0 means disabled" is the universal convention, so we use an explicit + * `unlimited` sentinel instead. Misconfig fails at startup with a hint. + */ +export function resolveLeaseCap(raw: string | undefined): number { + if (raw === undefined) return 32; + if (raw === 'unlimited' || raw === 'none') return Number.POSITIVE_INFINITY; + const n = Number(raw); + if (Number.isFinite(n) && n > 0) return n; + throw new Error( + `GBRAIN_ANTHROPIC_MAX_INFLIGHT="${raw}" is invalid. ` + + `Use a positive integer, "unlimited" (or "none"), or omit for default 32.`, + ); +} +const DEFAULT_MAX_CONCURRENT = resolveLeaseCap(process.env.GBRAIN_ANTHROPIC_MAX_INFLIGHT); const DEFAULT_LEASE_TTL_MS = 120_000; -const DEFAULT_SYSTEM = 'You are a helpful assistant running as a gbrain subagent.'; +// v0.41 Approach C: DEFAULT_SUBAGENT_SYSTEM lives in ./system-prompt.ts +// so the renderer and the handler share one source of truth. Kept as +// a re-export alias here for back-compat with any external importer. +const DEFAULT_SYSTEM = DEFAULT_SUBAGENT_SYSTEM; // ── Injectable surfaces (for tests) ───────────────────────── @@ -184,7 +212,11 @@ export function makeSubagentHandler(deps: SubagentDeps) { fallback: TIER_DEFAULTS.subagent, }); const maxTurns = data.max_turns ?? DEFAULT_MAX_TURNS; - const systemPrompt = data.system ?? DEFAULT_SYSTEM; + // v0.41 Approach C: systemPrompt is now built AFTER toolDefs (a few + // lines below) so the renderer can splice a tool-usage preamble + // listing each available tool's usage_hint. The renderer is + // deterministic so the Anthropic prompt-cache marker on the system + // block stays a hit across turns. // v0.38 S1.10 — feature flag for the gateway-native tool loop. When ON, // route ALL subagent jobs through gateway.toolLoop() (works for every @@ -219,6 +251,13 @@ export function makeSubagentHandler(deps: SubagentDeps) { ? filterAllowedTools(registry, data.allowed_tools) : registry; + // v0.41 Approach C: render the final system prompt now that toolDefs + // is known. Splices a deterministic tool-usage preamble listing each + // tool's usage_hint. Caller can opt out via data.system_no_tool_preamble. + const systemPrompt = buildSystemPrompt(toolDefs, data.system, { + no_tool_preamble: data.system_no_tool_preamble, + }); + logSubagentSubmission({ caller: 'worker', remote: true, @@ -437,7 +476,10 @@ export function makeSubagentHandler(deps: SubagentDeps) { // complexity; for v0.15 we lean on the 120s TTL + abort-on-signal. try { const params: Anthropic.MessageCreateParamsNonStreaming = { - model, + // v0.41 Bug 3: strip `provider:` prefix at the SDK call site only. + // `model` stays qualified everywhere else (persistence, recipe + // lookup at recipeIdFromModel(), capability gate). + model: stripProviderPrefix(model), max_tokens: 4096, system: [ { type: 'text', text: systemPrompt, cache_control: { type: 'ephemeral' } }, @@ -897,6 +939,28 @@ function recipeIdFromModel(modelString: string): string { return idx > 0 ? modelString.slice(0, idx) : 'anthropic'; } +/** + * Strip the `provider:` prefix from a model string. Returns the bare + * model id the Anthropic Messages API expects. Idempotent on already-bare + * strings. + * + * stripProviderPrefix('anthropic:claude-sonnet-4-6') === 'claude-sonnet-4-6' + * stripProviderPrefix('claude-sonnet-4-6') === 'claude-sonnet-4-6' + * + * v0.41 Bug 3 — pre-fix, `gbrain agent run --model anthropic:claude-sonnet-4-6` + * sent the prefixed string straight into `client.messages.create()`, which + * Anthropic rejects with "model not found." Omitting `--model` worked because + * `resolveModel()` returns the bare id; explicit-model users hit the bug. + * + * Used ONLY at the SDK call site. The wider `model` variable stays + * qualified everywhere else (persistence, recipe lookup, capability gate) + * because those readers want the provider info. + */ +export function stripProviderPrefix(modelString: string): string { + const idx = modelString.indexOf(':'); + return idx > 0 ? modelString.slice(idx + 1) : modelString; +} + /** * D5 — adapt v1 Anthropic content blocks to v2 ChatBlock shape on read. * Symmetric in the other direction is handled by persisting ChatBlock[] as-is diff --git a/src/core/minions/lease-cap-controller.ts b/src/core/minions/lease-cap-controller.ts new file mode 100644 index 000000000..30efc2f27 --- /dev/null +++ b/src/core/minions/lease-cap-controller.ts @@ -0,0 +1,285 @@ +/** + * v0.41 E5 (reframed per codex pass-1, corrected per codex pass-2 #9) — + * auto-adaptive rate-lease cap. + * + * Original E5 (pre-reframe) tuned WORKER concurrency. Codex pass-1 caught + * the fundamental design flaw: worker concurrency is downstream of the + * rate-lease. If the lease is full at 32 in-flight, easing worker + * concurrency from 10 to 5 just moves jobs from `active+bouncing` to + * `waiting` — same throughput. The adaptive knob has to be the LEASE CAP + * itself. + * + * Codex pass-2 #9 caught a sign error in the original lease-cap control + * law: my first draft said "ramp DOWN on bounces > 5/min OR 429s > 0.5/min." + * That's WRONG. Bounces happen when workers want more concurrency than the + * cap allows — internal queue pressure. Without upstream pushback (429s + * + latency unstable), bounces alone say "we're starving, raise the cap." + * The Eng D6 corrected control law: + * + * - Ramp DOWN: ONLY when upstream pushes back (429s > Y/min OR latency + * unstable). These are the "cap is too high" signals. + * - Ramp UP slow: 0 bounces + 0 429s + util > 50% + latency stable. + * We have headroom + jobs running; can probably go higher. + * - Ramp UP fast: bounces > X/min + 0 429s + latency stable. Workers + * are starving inside our artificial cap; raise it. + * - Deadband: middle region (some pressure but mixed signals). + * + * Per-tick election via tryWithDbElection (Eng D9): only one worker per + * cluster runs the WRITE side per tick. All workers READ the lease cap + * fresh from the DB on every acquire (short-TTL cache). + * + * Latency signal source: UPSTREAM (Anthropic SDK round-trip), NOT local + * DB query latency. Local DB latency is noise; upstream latency is the + * actual congestion signal. Codex pass-3 #2. + */ + +import type { BrainEngine } from '../engine.ts'; +import { tryWithDbElection } from '../db-lock.ts'; + +/** Rolling-window stats the controller reads each tick. */ +export interface ControllerWindowStats { + /** How many lease-full bounces happened in `window_ms`. */ + bounce_count: number; + /** How many upstream 429s happened in `window_ms` (per worker, sum). */ + upstream_429_count: number; + /** Mean (active / max_concurrent) over the window. NaN-safe → 0. */ + lease_utilization: number; + /** Upstream round-trip latency stability. True when p95/p50 ratio < 2 in window. */ + latency_stable: boolean; + /** Window size used for the rate calculations. ms. */ + window_ms: number; +} + +export interface ControllerOpts { + /** Lease cap to start from on first tick (matches GBRAIN_ANTHROPIC_MAX_INFLIGHT default). */ + initial_cap: number; + /** Per-step ramp-up amount (additive). */ + ramp_up_step: number; + /** Per-step ramp-down amount (additive; asymmetric > ramp_up_step for AIMD-style backoff). */ + ramp_down_step: number; + /** Lower bound — never below this. */ + min_floor: number; + /** Upper bound — never above this. CLI flag wins over env override (codex pass-2 #9). */ + max_ceiling: number; + /** Bounce-rate threshold to consider "workers are starving" (events / min). */ + bounce_rate_starving_threshold: number; + /** 429-rate threshold to consider "upstream is pushing back" (events / min). */ + upstream_429_threshold_per_min: number; + /** Lease-utilization threshold below which we don't ramp-up-slow (no headroom signal needed). */ + utilization_ramp_threshold: number; +} + +export const DEFAULT_CONTROLLER_OPTS: ControllerOpts = { + initial_cap: 32, + ramp_up_step: 4, + ramp_down_step: 8, // AIMD-style asymmetric: back off faster than ramp up + min_floor: 4, + max_ceiling: 128, + bounce_rate_starving_threshold: 1, // > 1 bounce/min with no upstream pushback = starving + upstream_429_threshold_per_min: 0.5, + utilization_ramp_threshold: 0.5, +}; + +/** + * Pure decision function: given current cap + window stats, what's the + * next cap? No I/O — fully testable for every signal combination. + * + * Decision tree (in order): + * 1. Upstream pushback (429s OR latency unstable) → ramp DOWN. ONLY + * time we shrink the cap; without this, the controller would crater + * cap during healthy bursts (codex pass-2 #9 caught this). + * 2. Workers starving (bounces > threshold + no upstream pushback) → + * ramp UP fast. Internal demand exceeding our artificial cap. + * 3. Headroom available (no bounces + no 429s + util > threshold + + * latency stable) → ramp UP slow. We have work flowing through + * and the upstream has room. + * 4. Otherwise → deadband (no change). Mixed signals = don't move. + */ +export function nextLeaseCap( + current: number, + window: ControllerWindowStats, + opts: ControllerOpts = DEFAULT_CONTROLLER_OPTS, +): number { + const windowMin = Math.max(1e-6, window.window_ms / 60_000); + const bounceRatePerMin = window.bounce_count / windowMin; + const upstream429PerMin = window.upstream_429_count / windowMin; + + // 1. Ramp DOWN — upstream pushback. Either 429s OR latency-unstable. + // These are the only signals saying "cap is too high." + if (upstream429PerMin > opts.upstream_429_threshold_per_min || !window.latency_stable) { + return Math.max(current - opts.ramp_down_step, opts.min_floor); + } + + // 2. Ramp UP fast — workers are starving. + // Bounces > threshold AND no upstream pushback AND latency stable + // AND room to grow. + if ( + bounceRatePerMin > opts.bounce_rate_starving_threshold && + upstream429PerMin === 0 && + window.latency_stable + ) { + return Math.min(current + opts.ramp_up_step, opts.max_ceiling); + } + + // 3. Ramp UP slow — no pressure but utilization shows we're using the cap. + // Probes for headroom: small step, low risk. + if ( + bounceRatePerMin === 0 && + upstream429PerMin === 0 && + window.lease_utilization > opts.utilization_ramp_threshold && + window.latency_stable + ) { + return Math.min(current + opts.ramp_up_step, opts.max_ceiling); + } + + // 4. Deadband — mixed signals or no work happening. + return current; +} + +/** + * Read the controller window stats from the DB. Reads + * minion_lease_pressure_log + minion_jobs for the last `windowMs`. + * + * Pure-SQL function so the controller tick is one round-trip per source. + * Latency signal is approximate today (uses subagent job durations as a + * proxy — full upstream-latency tracking is filed as v0.42 follow-up). + */ +export async function readControllerWindow( + engine: BrainEngine, + windowMs: number, +): Promise { + // Bounce count over window. Pre-v93 brains return 0 silently. + let bounceCount = 0; + try { + const rows = await engine.executeRaw<{ count: string }>( + `SELECT count(*)::text AS count FROM minion_lease_pressure_log + WHERE bounced_at > now() - ($1::double precision * interval '1 millisecond')`, + [windowMs], + ); + bounceCount = parseInt(rows[0]?.count ?? '0', 10); + } catch { + /* pre-v93 brain */ + } + + // Upstream 429 count proxied via dead-letter classifier; this is rough + // until v0.42 wires direct SDK 429 events into a counter table. For now, + // count jobs whose last_error contains "429" or "rate limit" in the + // window (classifier reuse keeps the signal cheap). + let upstream429Count = 0; + try { + const rows = await engine.executeRaw<{ count: string }>( + `SELECT count(*)::text AS count FROM minion_jobs + WHERE finished_at > now() - ($1::double precision * interval '1 millisecond') + AND status IN ('failed', 'dead') + AND (error_text ILIKE '%429%' OR error_text ILIKE '%rate limit%')`, + [windowMs], + ); + upstream429Count = parseInt(rows[0]?.count ?? '0', 10); + } catch { + /* DB unavailable */ + } + + // Lease utilization: mean of (active_at_bounce / max_concurrent) per + // bounce row. When 0 bounces, utilization is 0 (no signal — controller + // treats this as "not enough data to ramp" → deadband). + let leaseUtilization = 0; + try { + const rows = await engine.executeRaw<{ util: string | number }>( + `SELECT COALESCE(AVG(active_at_bounce::double precision / NULLIF(max_concurrent, 0)), 0) AS util + FROM minion_lease_pressure_log + WHERE bounced_at > now() - ($1::double precision * interval '1 millisecond')`, + [windowMs], + ); + const raw = parseFloat(String(rows[0]?.util ?? '0')); + leaseUtilization = Number.isFinite(raw) ? raw : 0; + } catch { + /* pre-v93 brain */ + } + + // Latency stability proxy: subagent job durations in window. If we have + // at least 3 jobs with started_at + finished_at, compute p95/p50 ratio. + // If < 2, stable; if >= 2 or insufficient samples, NOT stable. + let latencyStable = true; // default true so first-tick controller can ramp up + try { + const rows = await engine.executeRaw<{ p50: number; p95: number; samples: string }>( + `SELECT + COALESCE(percentile_cont(0.50) WITHIN GROUP (ORDER BY EXTRACT(EPOCH FROM (finished_at - started_at))), 0) AS p50, + COALESCE(percentile_cont(0.95) WITHIN GROUP (ORDER BY EXTRACT(EPOCH FROM (finished_at - started_at))), 0) AS p95, + count(*)::text AS samples + FROM minion_jobs + WHERE status = 'completed' + AND name = 'subagent' + AND started_at IS NOT NULL + AND finished_at IS NOT NULL + AND finished_at > now() - ($1::double precision * interval '1 millisecond')`, + [windowMs], + ); + const p50 = Number(rows[0]?.p50 ?? 0); + const p95 = Number(rows[0]?.p95 ?? 0); + const samples = parseInt(String(rows[0]?.samples ?? '0'), 10); + if (samples >= 3 && p50 > 0) { + latencyStable = (p95 / p50) < 2; + } + } catch { + /* DB unavailable; default stable */ + } + + return { + bounce_count: bounceCount, + upstream_429_count: upstream429Count, + lease_utilization: leaseUtilization, + latency_stable: latencyStable, + window_ms: windowMs, + }; +} + +/** + * Read the current lease cap from config (write target for the controller). + * Falls back to opts.initial_cap when unset. Workers read this on every + * acquire so they pick up controller writes within the cache TTL. + */ +export async function readCurrentLeaseCap( + engine: BrainEngine, + opts: ControllerOpts = DEFAULT_CONTROLLER_OPTS, +): Promise { + try { + const v = await engine.getConfig('minions.lease_cap_current').catch(() => null); + if (v) { + const n = Number(v); + if (Number.isFinite(n) && n >= opts.min_floor && n <= opts.max_ceiling) return n; + } + } catch { + /* config unavailable */ + } + return opts.initial_cap; +} + +/** + * Write the new lease cap to config. Called by the elected worker after + * a controller decision. Workers reading via readCurrentLeaseCap pick up + * the new value within their cache TTL. + */ +export async function writeLeaseCap(engine: BrainEngine, cap: number): Promise { + await engine.setConfig('minions.lease_cap_current', String(cap)); +} + +/** + * Run one controller tick. Elects a single mutator via tryWithDbElection; + * other workers no-op. Returns the cap change tuple when elected, null + * otherwise. + */ +export async function controllerTick( + engine: BrainEngine, + opts: ControllerOpts = DEFAULT_CONTROLLER_OPTS, + windowMs: number = 60_000, +): Promise<{ previous: number; next: number; changed: boolean } | null> { + return tryWithDbElection(engine, 'minions-lease-cap-controller', 2, async () => { + const window = await readControllerWindow(engine, windowMs); + const current = await readCurrentLeaseCap(engine, opts); + const next = nextLeaseCap(current, window, opts); + if (next !== current) { + await writeLeaseCap(engine, next); + } + return { previous: current, next, changed: next !== current }; + }); +} diff --git a/src/core/minions/lease-pressure-audit.ts b/src/core/minions/lease-pressure-audit.ts new file mode 100644 index 000000000..7a36a48b7 --- /dev/null +++ b/src/core/minions/lease-pressure-audit.ts @@ -0,0 +1,148 @@ +/** + * v0.41 Bug 2 + Eng D3+D8 — lease-pressure audit writer. + * + * Writes one row per `RateLeaseUnavailableError` bounce to the + * `minion_lease_pressure_log` table (migration v93). The doctor check + * `subagent_health` and `gbrain jobs stats lease_pressure` read this + * table to surface pressure to operators. + * + * Why DB table (not JSONL): doctor + jobs stats need SQL aggregation + * across many bounces in a rolling window; JSONL would require parsing + * + filtering per query. The audit table has indexes on `bounced_at DESC` + * and `job_id` for the exact access patterns those consumers need. + * + * Denormalization at write time (Eng D8 / codex pass-3 #7): `queue_name`, + * `job_name`, `model`, `provider`, `root_owner_id` are persisted inline so + * post-NULL forensic queries (after `gbrain jobs prune` pulls the job row) + * still carry enough context to answer "was there pressure on Anthropic + * messages last Tuesday at 3pm." Without denormalization, post-NULL rows + * would be timestamp-only residue. + * + * Best-effort: write failures (e.g. DB blip during the worker's lease-full + * bypass) log to stderr but never throw — losing one audit row is + * preferable to failing the bypass path that prevents dead-letter. + */ + +import type { BrainEngine } from '../engine.ts'; + +export interface LeasePressureRecord { + /** Job that bounced (FK target; SET NULL on prune so audit survives). */ + job_id: number; + /** Lease key the bounce happened on (e.g. `anthropic:messages`). */ + lease_key: string; + /** `activeCount` observed at bounce time (diagnostic). */ + active_at_bounce: number; + /** `maxConcurrent` checked against (diagnostic). */ + max_concurrent: number; + /** + * Denormalized context — persists past job pruning so aggregate forensic + * queries (the "what model had pressure last week" shape) still work. + * Pass best-effort values from the calling worker; nulls are accepted. + */ + queue_name?: string | null; + job_name?: string | null; + model?: string | null; + provider?: string | null; + root_owner_id?: number | null; +} + +/** + * Append one lease-pressure event. Best-effort — DB failures log to stderr + * and return without throwing so the worker's lease-full bypass path + * (which is the actual fix for the field-report dead-letter loop) is + * never blocked by audit-table write problems. + */ +export async function logLeasePressure( + engine: BrainEngine, + record: LeasePressureRecord, +): Promise { + try { + await engine.executeRaw( + `INSERT INTO minion_lease_pressure_log + (job_id, lease_key, active_at_bounce, max_concurrent, + queue_name, job_name, model, provider, root_owner_id) + VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)`, + [ + record.job_id, + record.lease_key, + record.active_at_bounce, + record.max_concurrent, + record.queue_name ?? null, + record.job_name ?? null, + record.model ?? null, + record.provider ?? null, + record.root_owner_id ?? null, + ], + ); + } catch (err) { + const msg = err instanceof Error ? err.message : String(err); + process.stderr.write( + `[lease-pressure-audit] WARN: write failed for job ${record.job_id}: ${msg}\n`, + ); + } +} + +/** + * Read lease-pressure events in a rolling window. Used by `gbrain doctor`'s + * `subagent_health` check and by `gbrain jobs stats lease_pressure`. + * Returns the raw rows so callers can do their own aggregation; the count + * is the operationally useful aggregate. + */ +export async function readRecentLeasePressure( + engine: BrainEngine, + windowMs: number, + opts: { limit?: number } = {}, +): Promise> { + const limit = opts.limit ?? 1000; + const rows = await engine.executeRaw<{ + id: number; + job_id: number | null; + lease_key: string; + active_at_bounce: number; + max_concurrent: number; + bounced_at: string; + queue_name: string | null; + job_name: string | null; + model: string | null; + provider: string | null; + root_owner_id: number | null; + }>( + `SELECT id, job_id, lease_key, active_at_bounce, max_concurrent, + bounced_at, queue_name, job_name, model, provider, root_owner_id + FROM minion_lease_pressure_log + WHERE bounced_at > now() - ($1::double precision * interval '1 millisecond') + ORDER BY bounced_at DESC + LIMIT $2`, + [windowMs, limit], + ); + return rows.map(r => ({ + id: r.id, + job_id: r.job_id ?? 0, // job_id is SET NULL after prune; surface as 0 for callers that don't care + lease_key: r.lease_key, + active_at_bounce: r.active_at_bounce, + max_concurrent: r.max_concurrent, + bounced_at: r.bounced_at, + queue_name: r.queue_name, + job_name: r.job_name, + model: r.model, + provider: r.provider, + root_owner_id: r.root_owner_id, + })); +} + +/** + * Bounded count of lease-pressure events in a rolling window. Cheaper + * than `readRecentLeasePressure` for callers that only want the metric. + */ +export async function countRecentLeasePressure( + engine: BrainEngine, + windowMs: number, +): Promise { + const rows = await engine.executeRaw<{ count: string }>( + `SELECT count(*)::text AS count + FROM minion_lease_pressure_log + WHERE bounced_at > now() - ($1::double precision * interval '1 millisecond')`, + [windowMs], + ); + return parseInt(rows[0]?.count ?? '0', 10); +} diff --git a/src/core/minions/queue.ts b/src/core/minions/queue.ts index 50e543453..02a271ced 100644 --- a/src/core/minions/queue.ts +++ b/src/core/minions/queue.ts @@ -965,6 +965,51 @@ export class MinionQueue { }); } + /** + * v0.41 Bug 2 — release a job back to `delayed` after a + * `RateLeaseUnavailableError` bounce, WITHOUT incrementing `attempts_made`. + * + * The field-report bug: pre-v0.41, lease-full bounces routed through + * `failJob` which bumps `attempts_made`. After 3 bounces the job hit + * `max_attempts` (default 3) and dead-lettered with message + * `rate lease "anthropic:messages" full (8/8)`. Operators saw a dead + * job and assumed a real failure. + * + * This method is the workhorse fix: status → `delayed`, jittered backoff + * via `delay_until`, `attempts_made` UNCHANGED. The handler comment at + * `src/core/minions/handlers/subagent.ts:425` ("treat as renewable + * error so the worker re-claims") is now actually true. + * + * Audit row write to `minion_lease_pressure_log` is the caller's + * responsibility (the worker has the model/queue context); this method + * stays focused on the state-machine flip. Same `lock_token + status='active'` + * idempotency guard as `failJob` so a racing stall sweep / cancel still + * wins. Returns `null` on lock_token mismatch. + * + * Returns the updated `MinionJob` row on success so the caller can stamp + * the audit row with provenance from the SAME row that just flipped. + */ + async releaseLeaseFullJob( + id: number, + lockToken: string, + errorText: string, + backoffMs: number, + ): Promise { + const rows = await this.engine.executeRaw>( + `UPDATE minion_jobs SET + status = 'delayed', + error_text = $1, + stacktrace = COALESCE(stacktrace, '[]'::jsonb) || to_jsonb($1::text), + delay_until = now() + ($2::double precision * interval '1 millisecond'), + lock_token = NULL, lock_until = NULL, updated_at = now() + WHERE id = $3 AND status = 'active' AND lock_token = $4 + RETURNING *`, + [errorText, backoffMs, id, lockToken], + ); + if (rows.length === 0) return null; + return rowToMinionJob(rows[0]); + } + /** Update job progress (token-fenced). */ async updateProgress(id: number, lockToken: string, progress: unknown): Promise { const rows = await this.engine.executeRaw>( diff --git a/src/core/minions/self-fix.ts b/src/core/minions/self-fix.ts new file mode 100644 index 000000000..2cff6410c --- /dev/null +++ b/src/core/minions/self-fix.ts @@ -0,0 +1,275 @@ +/** + * v0.41 E6 (corrected per codex pass-2 #4 + pass-1 #11) — subagent + * failure-mode self-fix. + * + * When a subagent job terminates with a classified-recoverable failure + * AND chain depth < `minions.self_fix_max_depth` AND no prior self-fix + * at the same fingerprint, submit a follow-up job with the failure + * context as the user message. One layer of self-healing; capped so it + * cannot infinite-loop. + * + * **Classifier scope (codex pass-2 #4 — narrowed):** + * + * Only three buckets qualify for self-fix: + * + * prompt_too_long — semantic-aware reduction can fix + * tool_schema_mismatch — surface schema error to model, retry with valid input + * malformed_json — ask model to retry, response must be JSON + * + * EXPLICITLY NOT self-fixable: + * tool_crash — likely a real bug; masking via retry is worse + * tool_unavailable — registry config issue; retry won't help + * tool_permission — capability decision; needs human intervention + * + * **prompt_too_long reduction (codex pass-1 #11):** generic truncation + * deletes the actual task. Right strategy: walk the conversation, drop + * tool_result blocks first (largest non-task content), summarize older + * user/asst pairs via Haiku if still over, never delete the leaf user + * task. v0.41 ships the foundation (the policy + dispatch); full + * semantic reduction can land as a follow-up since the worst-case fix + * — truncate-then-fail — is still safe. + * + * **Chain depth:** configurable, default 2 (D15). Hard cap = N from + * config; chain-walk idempotency check prevents double self-fix at the + * same fingerprint. + * + * **Budget:** self-fix children inherit the budget owner (NOT a copy + * of remaining balance — codex pass-3 #5). Self-fix spend counts + * against the same owner as the parent. + * + * **Default state:** ON for fresh install AND on upgrade (D9 + D15). + * Off-switch: `gbrain config set minions.self_fix_enabled false` + * (global) or `data.no_self_fix: true` (per-job). + */ + +import type { BrainEngine } from '../engine.ts'; +import type { MinionQueue } from './queue.ts'; +import { classifyJobError, RECOVERABLE_CLUSTERS, type ErrorCluster } from './error-classify.ts'; +import type { SubagentHandlerData } from './types.ts'; +import { inheritBudgetOwner } from './budget-tracker.ts'; + +export interface SelfFixDecision { + /** Should we self-fix this failure? */ + should_fix: boolean; + /** When should_fix=true, the cluster that matched. */ + cluster?: ErrorCluster; + /** Reason text (always present; explains the decision). */ + reason: string; +} + +export interface SelfFixOpts { + /** Hard cap on chain depth. Default 2 (D15). */ + max_depth?: number; + /** Global on/off (from config). Default true. */ + enabled?: boolean; +} + +const DEFAULT_OPTS: Required = { + max_depth: 2, + enabled: true, +}; + +/** + * Compute the chain depth of a job. Walks `parent_job_id` chain looking + * for ancestors with `data.is_self_fix_child = true`. Each such ancestor + * adds 1 to the depth. + */ +export async function computeChainDepth(engine: BrainEngine, jobId: number): Promise { + let depth = 0; + let cursor: number | null = jobId; + while (cursor !== null && depth < 10 /* safety cap */) { + const rows: Array<{ parent_job_id: number | null; data: unknown }> = + await engine.executeRaw( + `SELECT parent_job_id, data FROM minion_jobs WHERE id = $1`, + [cursor], + ); + if (rows.length === 0) break; + const data = rows[0]!.data as Record | null; + if (data && data.is_self_fix_child === true) { + depth += 1; + } + cursor = rows[0]!.parent_job_id; + } + return depth; +} + +/** + * Decide whether a job should self-fix. Pure-ish (one SQL for depth); + * the decision logic itself is unit-testable in isolation. + */ +export async function decideSelfFix( + engine: BrainEngine, + jobId: number, + jobData: SubagentHandlerData, + lastError: string | null, + opts: SelfFixOpts = {}, +): Promise { + const merged = { ...DEFAULT_OPTS, ...opts }; + if (!merged.enabled) { + return { should_fix: false, reason: 'self_fix_disabled_globally' }; + } + if (jobData.no_self_fix) { + return { should_fix: false, reason: 'no_self_fix_flag_on_job' }; + } + const cluster = classifyJobError(lastError); + if (!RECOVERABLE_CLUSTERS.has(cluster)) { + return { should_fix: false, cluster, reason: `cluster_not_recoverable:${cluster}` }; + } + const depth = await computeChainDepth(engine, jobId); + if (depth >= merged.max_depth) { + return { should_fix: false, cluster, reason: `max_depth_reached:${depth}` }; + } + return { should_fix: true, cluster, reason: `recoverable:${cluster}_at_depth_${depth}` }; +} + +/** + * Build the user-message text for the self-fix child. Per-cluster prep: + * + * prompt_too_long — truncate-then-prepend (v0.41 ships truncate; + * semantic reduction follow-up in v0.42) + * tool_schema_mismatch — surface schema error verbatim; ask for retry + * with valid arguments + * malformed_json — instruct model to respond with JSON only + */ +export function buildSelfFixPrompt( + originalPrompt: string, + cluster: ErrorCluster, + lastError: string, +): string { + switch (cluster) { + case 'prompt_too_long': { + // Truncation strategy: keep first 1000 chars + last 2000 chars of + // the original prompt so the leaf task survives. Honest scope — v0.42 + // will replace with semantic reduction (drop tool_results first, + // then Haiku-summarize older pairs). + const first = originalPrompt.slice(0, 1000); + const last = originalPrompt.length > 3000 + ? '\n\n... (middle truncated) ...\n\n' + originalPrompt.slice(-2000) + : ''; + return [ + `[self-fix retry] Your previous attempt failed because the prompt was too long.`, + `Original task (truncated to fit context):`, + '', + first + last, + '', + `Provide your answer based on the truncated task. If critical context is missing, say so.`, + ].join('\n'); + } + case 'tool_schema_mismatch': + return [ + `[self-fix retry] Your previous attempt failed because a tool call had invalid arguments.`, + `Error: ${lastError}`, + '', + `Original task:`, + originalPrompt, + '', + `Retry the task. Double-check tool arguments against each tool's input_schema before calling.`, + ].join('\n'); + case 'malformed_json': + return [ + `[self-fix retry] Your previous attempt failed because the response was not valid JSON.`, + `Error: ${lastError}`, + '', + `Original task:`, + originalPrompt, + '', + `Retry. Your final response MUST be valid JSON — no prose, no markdown fences, no commentary.`, + ].join('\n'); + default: + return `[self-fix retry] ${originalPrompt}`; + } +} + +/** + * Submit a self-fix child job. Returns the submitted child or null on + * failure (audit row written either way). Caller has already decided + * via decideSelfFix; this just does the submission. + * + * Child inherits budget owner from parent (Eng D7 + D10). + */ +export async function submitSelfFixChild( + engine: BrainEngine, + queue: MinionQueue, + parent: { + id: number; + data: SubagentHandlerData; + last_error: string; + }, + cluster: ErrorCluster, +): Promise<{ child_id: number } | null> { + try { + const childPrompt = buildSelfFixPrompt(parent.data.prompt, cluster, parent.last_error); + const childData: SubagentHandlerData & { is_self_fix_child: true; self_fix_cluster: ErrorCluster } = { + ...parent.data, + prompt: childPrompt, + is_self_fix_child: true, + self_fix_cluster: cluster, + }; + const child = await queue.add( + 'subagent', + childData as unknown as Record, + { parent_job_id: parent.id, max_attempts: 1 }, // single attempt; if self-fix fails, that's terminal + { allowProtectedSubmit: true }, + ); + // Inherit budget owner if parent has one. + await inheritBudgetOwner(engine, child.id, parent.id); + // Audit row. + await logSelfFixEvent(engine, { + parent_id: parent.id, + child_id: child.id, + classifier_bucket: cluster, + chain_depth: 1, // depth from parent's perspective — refined at next-tier check + policy_applied: `buildSelfFixPrompt:${cluster}`, + outcome: 'submitted', + }); + return { child_id: child.id }; + } catch (err) { + const msg = err instanceof Error ? err.message : String(err); + process.stderr.write(`[self-fix] WARN: child submit failed for parent ${parent.id}: ${msg}\n`); + await logSelfFixEvent(engine, { + parent_id: parent.id, + child_id: 0, // sentinel — submit failed; no child exists + classifier_bucket: cluster, + chain_depth: 1, + policy_applied: `buildSelfFixPrompt:${cluster}`, + outcome: `failed_to_submit:${msg.slice(0, 200)}`, + }); + return null; + } +} + +export interface SelfFixEventRecord { + parent_id: number; + child_id: number; + classifier_bucket: string; + chain_depth: number; + policy_applied?: string; + outcome: string; +} + +/** Best-effort write to minion_self_fix_log (audit table from migration v93). */ +export async function logSelfFixEvent( + engine: BrainEngine, + record: SelfFixEventRecord, +): Promise { + try { + await engine.executeRaw( + `INSERT INTO minion_self_fix_log + (parent_id, child_id, classifier_bucket, chain_depth, policy_applied, outcome) + VALUES ($1, $2, $3, $4, $5, $6)`, + [ + record.parent_id, + record.child_id || null, // 0 → NULL (submit failed) + record.classifier_bucket, + record.chain_depth, + record.policy_applied ?? null, + record.outcome, + ], + ); + } catch (err) { + const msg = err instanceof Error ? err.message : String(err); + process.stderr.write( + `[self-fix-audit] WARN: write failed for parent ${record.parent_id}: ${msg}\n`, + ); + } +} diff --git a/src/core/minions/system-prompt.ts b/src/core/minions/system-prompt.ts new file mode 100644 index 000000000..eabf9a27a --- /dev/null +++ b/src/core/minions/system-prompt.ts @@ -0,0 +1,111 @@ +/** + * v0.41 Approach C — composable subagent system prompt renderer. + * + * Field-report case: a `shell` tool sat in the registry and the subagent + * never used it because the default system prompt was one generic line + * (`'You are a helpful assistant running as a gbrain subagent.'`) and gave + * the model no guidance on WHICH tool to reach for. This module fixes that + * by splicing a deterministic tool-guidance preamble into the system prompt + * based on the actual `toolDefs` array, including any plugin-registered + * tools the core has no knowledge of. + * + * Three properties matter for the Anthropic prompt-cache contract: + * + * 1. Deterministic — same `(toolDefs, userSystem, opts)` input produces + * byte-identical output. The Anthropic `cache_control: ephemeral` + * marker on the system block wraps OUR rendered string; if rendering + * drifted across runs, the cache marker would miss on every turn. + * 2. Generative — works for plugin tools the core has never heard of. + * A downstream plugin that registers `playwright_navigate` with a + * `usage_hint` gets that hint surfaced automatically. + * 3. Opinionated — the closing paragraph names `shell`/`bash` explicitly + * AND tells the model that brain tools (`put_page`, `search`, `query`) + * write to the brain DB, not the local filesystem. Both halves of the + * field-report bug (subagent didn't reach for shell + subagent wrote + * brain pages when files were wanted) collapse into one fix. + * + * Override paths preserved: caller's `data.system` overrides the default + * generic line; `data.system_no_tool_preamble: true` skips the preamble + * splice entirely so the caller's prompt stays byte-for-byte as provided. + */ + +import type { ToolDef } from './types.ts'; + +export const DEFAULT_SUBAGENT_SYSTEM = + 'You are a helpful assistant running as a gbrain subagent.'; + +export interface SystemPromptOpts { + /** Skip the auto-generated tool guidance preamble. */ + no_tool_preamble?: boolean; +} + +/** + * Render the final system prompt sent to the Anthropic Messages API. + * + * Composition order: + * - userSystem (or DEFAULT) — the caller's preamble. + * - blank line. + * - tool guidance preamble (when toolDefs is non-empty AND opts.no_tool_preamble !== true). + * + * Output is fully deterministic for a given input — required for the + * Anthropic prompt-cache marker on the system block. + */ +export function buildSystemPrompt( + toolDefs: ToolDef[], + userSystem: string | undefined, + opts: SystemPromptOpts = {}, +): string { + const base = userSystem ?? DEFAULT_SUBAGENT_SYSTEM; + if (opts.no_tool_preamble || toolDefs.length === 0) return base; + return base + '\n\n' + renderToolPreamble(toolDefs); +} + +/** + * Render the tool-usage preamble. Pure function; no I/O. + * + * Preamble shape (exact bytes, do not reorder — cache-marker stability): + * + * You have the following tools available. Reach for them by default — do NOT + * describe file contents, hypothetical shell output, or planned database + * writes in prose. Call the tool. + * + * - `tool_name` — usage_hint (when present) + * - `tool_name` (when no usage_hint) + * ... + * + * When the task asks you to write a file, run a command, or modify the + * filesystem, prefer a `shell` or `bash` tool if one is in your registry. + * Brain tools (`put_page`, `search`, `query`) write to the gbrain database, + * not to local files. + * + * Tools are listed in the order the caller passed them — NO sorting. The + * caller's order matches the order in the Anthropic `tools:` array, which + * matches the order the model sees in its tool-choice context. Sorting + * would diverge from that and confuse cache-marker reasoning. + */ +export function renderToolPreamble(toolDefs: ToolDef[]): string { + const lines: string[] = []; + for (const t of toolDefs) { + if (t.usage_hint && t.usage_hint.trim()) { + // Normalize any embedded whitespace/newlines so the rendered line + // stays single-line. Defense against a plugin author shipping a + // multi-line hint (which would corrupt the preamble layout). + const hint = t.usage_hint.replace(/\s+/g, ' ').trim(); + lines.push(`- \`${t.name}\` — ${hint}`); + } else { + lines.push(`- \`${t.name}\``); + } + } + return [ + 'You have the following tools available. Reach for them by default — do NOT', + 'describe file contents, hypothetical shell output, or planned database', + 'writes in prose. Call the tool.', + '', + ...lines, + '', + 'When the task asks you to write a file, run a command, or modify the', + 'filesystem, prefer a `shell` or `bash` tool if one is in your registry.', + 'Brain tools (`put_page`, `search`, `query`) write to the gbrain database,', + 'not to local files.', + ].join('\n'); +} diff --git a/src/core/minions/tools/brain-allowlist.ts b/src/core/minions/tools/brain-allowlist.ts index cedd4f46e..f01dfe595 100644 --- a/src/core/minions/tools/brain-allowlist.ts +++ b/src/core/minions/tools/brain-allowlist.ts @@ -66,6 +66,36 @@ export const BRAIN_TOOL_ALLOWLIST: ReadonlySet = new Set([ 'find_anomalies', ]); +/** + * v0.41 Approach C: per-tool usage_hint surfaced verbatim in the subagent + * system prompt's tool preamble. Each entry tells the model WHEN to reach + * for the tool (the description tells the model HOW). One line per tool; + * no embedded newlines. + * + * Field-report driver: the renderer in `src/core/minions/system-prompt.ts` + * surfaces these so a model with `shell` + brain tools in its registry + * knows brain tools write to the gbrain DB (NOT local files) and to reach + * for shell when the task asks for filesystem work. + * + * Keyed by OP name (pre-`brain_` prefix). Optional — tools without an entry + * just render as `- \`name\`` with no hint suffix. + */ +export const BRAIN_TOOL_USAGE_HINTS: Readonly> = { + query: 'Use for natural-language semantic search across the brain (vector + keyword hybrid). Returns ranked passages with citations. First choice when the user asks a question of the brain.', + search: 'Use for hybrid keyword + vector search returning ranked page hits. Use over `query` when you want page-level not chunk-level results (e.g. "find pages about X").', + get_page: 'Read a brain page by its slug. Returns the full markdown body + frontmatter + linked pages.', + list_pages: 'List pages by type or slug-prefix filter. Use when you need to enumerate (e.g. "list all `people/` pages") instead of search.', + file_list: 'List uploaded files (attachments) by slug-prefix or content type. NOT the local filesystem — only files the brain has stored.', + file_url: 'Get a presigned URL for a brain-stored file. Read-only; expires.', + get_backlinks: 'List every page that links TO the given slug. Use for "what references this".', + traverse_graph: 'Walk the typed-edge graph starting from a slug (e.g. `works_at`, `founded`, `invested_in`). Use for relationship queries.', + resolve_slugs: 'Resolve free-form entity names to canonical slugs (e.g. "Alice" → `people/alice-example`). Use before any tool that takes a slug if the user gave a name not a slug.', + get_ingest_log: 'Read the brain ingestion log for diagnostic / verification queries.', + put_page: 'Write a markdown page to the gbrain DATABASE (NOT the local filesystem). Page becomes searchable + linkable. Slug must match the agent\'s allowed namespace.', + get_recent_salience: 'Read pages ranked by emotional + activity salience over a recency window. Use for "what\'s been on my mind lately".', + find_anomalies: 'Read cohort-level activity outliers (e.g. tag-cohort or type-cohort with unusual recent volume). Use for "what\'s unusual lately".', +}; + /** Matches Anthropic's tool-name constraint. No dots. */ const ANTHROPIC_NAME_RE = /^[a-zA-Z0-9_-]{1,64}$/; @@ -223,6 +253,9 @@ export function buildBrainTools(opts: BuildBrainToolsOpts): ToolDef[] { // v0.15 ships only idempotent brain tools (every allow-listed op is // deterministic over its input; put_page re-writes the same slug). idempotent: true, + // v0.41 Approach C: surface usage_hint to the system-prompt renderer. + // Keyed by the unprefixed op name. Undefined when no hint is registered. + usage_hint: BRAIN_TOOL_USAGE_HINTS[op.name], async execute(input: unknown, ctx: ToolCtx): Promise { const opCtx = buildOpContext({ engine: ctx.engine, diff --git a/src/core/minions/types.ts b/src/core/minions/types.ts index b5942103b..e935afe8b 100644 --- a/src/core/minions/types.ts +++ b/src/core/minions/types.ts @@ -449,6 +449,38 @@ export interface SubagentHandlerData { * and direct CLI submitters set it. */ allowed_slug_prefixes?: string[]; + /** + * v0.41 Approach C: opt out of the auto-generated tool-usage preamble + * that `buildSystemPrompt()` splices into `system`. Default behavior + * (omitted or false) prepends a deterministic preamble listing each + * tool's name + usage_hint. Set to `true` to keep `system` byte-for-byte + * as provided. + * + * Use when the caller has hand-tuned a complete system prompt for a + * specific subagent (no benefit from auto-generated guidance, prompt + * cache hits ride entirely on the caller-supplied prefix). + */ + system_no_tool_preamble?: boolean; + /** + * v0.41 E6 — opt OUT of classifier-gated auto-resubmit on terminal + * failures. Default behavior (omitted or false) runs the + * `RECOVERABLE_CLUSTERS` self-fix path when the failure classifies as + * one of {`prompt_too_long`, `tool_schema_mismatch`, `malformed_json`}. + * Set true to disable per-job (useful for graders / probes where a + * retry would muddy the signal). + */ + no_self_fix?: boolean; + /** + * v0.41 E6 — internal marker set by `submitSelfFixChild` so the + * chain-depth walker can count self-fix ancestors. Counter starts at + * 0 on a fresh user-submitted job; increments by 1 per chain hop. + */ + is_self_fix_child?: boolean; + /** + * v0.41 E6 — which classifier bucket triggered this self-fix child. + * Read by audit + diagnostic surfaces (jobs get / dashboard). + */ + self_fix_cluster?: string; } /** @@ -499,6 +531,20 @@ export interface ToolDef { input_schema: Record; idempotent: boolean; execute(input: unknown, ctx: ToolCtx): Promise; + /** + * v0.41 Approach C: one-line hint surfaced verbatim in the subagent + * system prompt's tool preamble. Tells the model WHEN to use this tool. + * The `description` tells the model HOW; the `usage_hint` tells WHEN. + * + * Field-report case: a `shell` tool sat in the registry and the subagent + * never used it because nothing told the model "to write a file, use + * shell." Per-tool hints surface that directly. Plugin authors get this + * affordance for free. + * + * Optional — omitted tools just don't get a hint line. Must be a single + * line (no embedded newlines) so the rendered preamble stays scannable. + */ + usage_hint?: string; } /** diff --git a/src/core/minions/worker.ts b/src/core/minions/worker.ts index a6f2bb43b..514cbc91c 100644 --- a/src/core/minions/worker.ts +++ b/src/core/minions/worker.ts @@ -19,6 +19,8 @@ import type { import { UnrecoverableError } from './types.ts'; import { MinionQueue } from './queue.ts'; import { calculateBackoff } from './backoff.ts'; +import { RateLeaseUnavailableError } from './handlers/subagent.ts'; +import { logLeasePressure } from './lease-pressure-audit.ts'; import { randomUUID } from 'crypto'; import { EventEmitter } from 'events'; import { evaluateQuietHours, type QuietHoursConfig } from './quiet-hours.ts'; @@ -758,6 +760,57 @@ export class MinionWorker extends EventEmitter { errorText = err instanceof Error ? err.message : String(err); } + // v0.41 Bug 2: lease-full bounces don't burn attempts. + // + // Pre-v0.41 every non-`UnrecoverableError` routed to `delayed` with + // exponential backoff BUT still incremented `attempts_made`. After 3 + // lease-full bounces the job hit `max_attempts` and dead-lettered + // with message `rate lease "..." full (N/M)` — operators saw a + // "dead" job and assumed real failure. The field-report dead-letter + // loop is exactly this path. + // + // Detect `RateLeaseUnavailableError` BEFORE the attempts-exhaustion + // gate and route through `queue.releaseLeaseFullJob` which mirrors + // `failJob` minus the `attempts_made` increment. Audit row to + // `minion_lease_pressure_log` so operators see pressure live in + // `gbrain doctor` + `gbrain jobs stats lease_pressure`. + const isLeaseFull = err instanceof RateLeaseUnavailableError; + if (isLeaseFull) { + const leaseErr = err as RateLeaseUnavailableError; + // 1-3s jittered backoff. Not the exponential curve — this is "yield + // the slot, try again soon", not "give up after a few tries." + const leaseBackoffMs = 1000 + Math.floor(Math.random() * 2000); + const released = await this.queue.releaseLeaseFullJob( + job.id, lockToken, errorText, leaseBackoffMs, + ); + if (!released) { + console.warn(`Job ${job.id} lease-full release dropped (lock token mismatch)`); + return; + } + // Audit row write is best-effort — never blocks the bypass path. + // Denormalized columns persist past `gbrain jobs prune` so post-NULL + // forensic queries still see context (Eng D8 / codex pass-3 #7). + await logLeasePressure(this.engine, { + job_id: job.id, + lease_key: leaseErr.key, + active_at_bounce: leaseErr.active, + max_concurrent: Number.isFinite(leaseErr.max) ? leaseErr.max : -1, + queue_name: job.queue, + job_name: job.name, + // Best-effort context — populated when we can. The worker doesn't + // always know the model at catch time (model is resolved inside + // the handler), so leave NULL when unavailable. The doctor check's + // aggregate queries handle NULL gracefully. + model: null, + provider: null, + root_owner_id: job.parent_job_id ?? null, + }); + console.log( + `Job ${job.id} (${job.name}) lease-full, re-queuing in ${Math.round(leaseBackoffMs)}ms (no attempt burned)`, + ); + return; + } + const isUnrecoverable = err instanceof UnrecoverableError; const attemptsExhausted = job.attempts_made + 1 >= job.max_attempts; diff --git a/src/core/model-config.ts b/src/core/model-config.ts index d99d1b185..ba6ce9db6 100644 --- a/src/core/model-config.ts +++ b/src/core/model-config.ts @@ -47,13 +47,18 @@ export interface ResolveModelOpts { fallback: string; } -/** Default aliases shipped in code. Users override via `models.aliases.` config. */ +/** Default aliases shipped in code. Users override via `models.aliases.` config. + * Values include the `provider:` prefix so resolved model strings always + * carry an explicit provider — required by the v0.40.8+ subagent queue's + * classifyCapabilities() validation. Bare model ids (e.g. `claude-opus-4-7`) + * cause `resolveRecipe()` to throw "unknown provider" and the queue rejects + * the submit. */ export const DEFAULT_ALIASES: Record = { - opus: 'claude-opus-4-7', - sonnet: 'claude-sonnet-4-6', - haiku: 'claude-haiku-4-5-20251001', - gemini: 'gemini-3-pro', - gpt: 'gpt-5', + opus: 'anthropic:claude-opus-4-7', + sonnet: 'anthropic:claude-sonnet-4-6', + haiku: 'anthropic:claude-haiku-4-5-20251001', + gemini: 'google:gemini-3-pro', + gpt: 'openai:gpt-5', }; /** @@ -66,10 +71,10 @@ export const DEFAULT_ALIASES: Record = { * Users override via `gbrain config set models.tier. `. */ export const TIER_DEFAULTS: Record = { - utility: 'claude-haiku-4-5-20251001', - reasoning: 'claude-sonnet-4-6', - deep: 'claude-opus-4-7', - subagent: 'claude-sonnet-4-6', + utility: 'anthropic:claude-haiku-4-5-20251001', + reasoning: 'anthropic:claude-sonnet-4-6', + deep: 'anthropic:claude-opus-4-7', + subagent: 'anthropic:claude-sonnet-4-6', }; /** diff --git a/src/core/pglite-engine.ts b/src/core/pglite-engine.ts index a507c8393..9d168eb3f 100644 --- a/src/core/pglite-engine.ts +++ b/src/core/pglite-engine.ts @@ -236,7 +236,7 @@ export class PGLiteEngine implements BrainEngine { const { applied } = await runMigrations(this); if (applied > 0) { - console.log(` ${applied} migration(s) applied`); + process.stderr.write(` ${applied} migration(s) applied\n`); } } @@ -462,7 +462,7 @@ export class PGLiteEngine implements BrainEngine { && !needsPagesProvenance && !needsContextualRetrievalColumns && !needsPagesGeneration) return; - console.log(' Pre-v0.21 brain detected, applying forward-reference bootstrap'); + process.stderr.write(' Pre-v0.21 brain detected, applying forward-reference bootstrap\n'); if (needsPagesBootstrap) { // Mirror schema-embedded.ts shape for `sources` so the subsequent @@ -1847,11 +1847,16 @@ export class PGLiteEngine implements BrainEngine { async countStaleChunks(opts?: { sourceId?: string }): Promise { // D7: source-scoped count for `gbrain embed --stale --source X`. + // v0.41 (D4+D8+Codex r2 #11): always JOIN pages so embed-skip filter + // applies via `NOT (frontmatter ? 'embed_skip')`. PGLite is + // PostgreSQL 17.5 in WASM and supports the full JSONB operator set. if (opts?.sourceId === undefined) { const { rows } = await this.db.query( `SELECT count(*)::int AS count - FROM content_chunks - WHERE embedding IS NULL`, + FROM content_chunks cc + JOIN pages p ON p.id = cc.page_id + WHERE cc.embedding IS NULL + AND NOT (COALESCE(p.frontmatter, '{}'::jsonb) ? 'embed_skip')`, ); const count = (rows[0] as { count: number } | undefined)?.count ?? 0; return Number(count); @@ -1861,7 +1866,8 @@ export class PGLiteEngine implements BrainEngine { FROM content_chunks cc JOIN pages p ON p.id = cc.page_id WHERE cc.embedding IS NULL - AND p.source_id = $1`, + AND p.source_id = $1 + AND NOT (COALESCE(p.frontmatter, '{}'::jsonb) ? 'embed_skip')`, [opts.sourceId], ); const count = (rows[0] as { count: number } | undefined)?.count ?? 0; @@ -1879,6 +1885,8 @@ export class PGLiteEngine implements BrainEngine { const afterIdx = opts?.afterChunkIndex ?? -1; // D7: optional source-scoped cursor scan. PGLite mirrors postgres-engine // so the engine-parity E2E catches drift. + // v0.41 (D4+D8): NOT (frontmatter ? 'embed_skip') filter for soft-blocked + // pages, matching the postgres-engine sibling. if (opts?.sourceId === undefined) { const { rows } = await this.db.query( `SELECT p.slug, cc.chunk_index, cc.chunk_text, cc.chunk_source, @@ -1886,6 +1894,7 @@ export class PGLiteEngine implements BrainEngine { FROM content_chunks cc JOIN pages p ON p.id = cc.page_id WHERE cc.embedding IS NULL + AND NOT (COALESCE(p.frontmatter, '{}'::jsonb) ? 'embed_skip') AND (cc.page_id, cc.chunk_index) > ($1, $2) ORDER BY cc.page_id, cc.chunk_index LIMIT $3`, @@ -1900,6 +1909,7 @@ export class PGLiteEngine implements BrainEngine { JOIN pages p ON p.id = cc.page_id WHERE cc.embedding IS NULL AND p.source_id = $1 + AND NOT (COALESCE(p.frontmatter, '{}'::jsonb) ? 'embed_skip') AND (cc.page_id, cc.chunk_index) > ($2, $3) ORDER BY cc.page_id, cc.chunk_index LIMIT $4`, diff --git a/src/core/postgres-engine.ts b/src/core/postgres-engine.ts index cb15de3b6..2048bca3c 100644 --- a/src/core/postgres-engine.ts +++ b/src/core/postgres-engine.ts @@ -146,6 +146,11 @@ export class PostgresEngine implements BrainEngine { idle_timeout: 20, connect_timeout: 10, types: { bigint: postgres.BigInt }, + // Silence postgres NOTICE-level messages by default. See db.ts for + // rationale (stdout-parsing callers like jobs-submit --json break when + // idempotent CREATE migrations flood stdout). Opt back in with + // GBRAIN_PG_NOTICES=1. + onnotice: process.env.GBRAIN_PG_NOTICES === '1' ? undefined : () => {}, }; if (Object.keys(timeouts).length > 0) { opts.connection = timeouts; @@ -262,7 +267,7 @@ export class PostgresEngine implements BrainEngine { // Run any pending migrations automatically const { applied } = await runMigrations(this); if (applied > 0) { - console.log(` ${applied} migration(s) applied`); + process.stderr.write(` ${applied} migration(s) applied\n`); } // Post-migration schema verification: catches columns that migrations @@ -270,7 +275,7 @@ export class PostgresEngine implements BrainEngine { // Self-heals missing columns via ALTER TABLE ADD COLUMN IF NOT EXISTS. const verify = await verifySchema(this); if (verify.healed.length > 0) { - console.log(` Schema verify: self-healed ${verify.healed.length} missing column(s)`); + process.stderr.write(` Schema verify: self-healed ${verify.healed.length} missing column(s)\n`); } // v0.30.1 (Fix 5): sweep zombie HNSW indexes (indisvalid=false) from @@ -279,7 +284,7 @@ export class PostgresEngine implements BrainEngine { try { const result = await dropZombieIndexes(this); if (result.dropped.length > 0) { - console.log(` HNSW sweep: dropped ${result.dropped.length} zombie index(es)`); + process.stderr.write(` HNSW sweep: dropped ${result.dropped.length} zombie index(es)\n`); } } catch { /* best-effort */ } } finally { @@ -529,7 +534,7 @@ export class PostgresEngine implements BrainEngine { && !needsPagesProvenance && !needsContextualRetrievalColumns && !needsPagesGeneration) return; - console.log(' Pre-v0.21 brain detected, applying forward-reference bootstrap'); + process.stderr.write(' Pre-v0.21 brain detected, applying forward-reference bootstrap\n'); if (needsPagesBootstrap) { // Mirror schema-embedded.ts's `sources` shape so the subsequent @@ -1899,15 +1904,22 @@ export class PostgresEngine implements BrainEngine { async countStaleChunks(opts?: { sourceId?: string }): Promise { const sql = this.sql; - // Fast path: no source filter → bare count query, no join. - // Slow path: source-scoped count → join pages. - // D7: closes the bug where `gbrain embed --stale --source X` silently - // dropped X and counted across every source. + // v0.41 (D4+D8+Codex r2 #11): the embed-skip filter requires JOIN + // pages so we always join — the pre-v0.41 "fast path" without join + // is gone. JSONB `?` existence check is cheap on the small set of + // skipped pages; full-scan benefits from the partial index on + // embedding IS NULL regardless. + // + // D7: source_id scoping. NULL/undefined = scan all sources; + // a value scopes to that source so `gbrain embed --stale --source X` + // does what it says. if (opts?.sourceId === undefined) { const [row] = await sql` SELECT count(*)::int AS count - FROM content_chunks - WHERE embedding IS NULL + FROM content_chunks cc + JOIN pages p ON p.id = cc.page_id + WHERE cc.embedding IS NULL + AND NOT (COALESCE(p.frontmatter, '{}'::jsonb) ? 'embed_skip') `; return Number((row as { count?: number } | undefined)?.count ?? 0); } @@ -1917,6 +1929,7 @@ export class PostgresEngine implements BrainEngine { JOIN pages p ON p.id = cc.page_id WHERE cc.embedding IS NULL AND p.source_id = ${opts.sourceId} + AND NOT (COALESCE(p.frontmatter, '{}'::jsonb) ? 'embed_skip') `; return Number((row as { count?: number } | undefined)?.count ?? 0); } @@ -1938,6 +1951,12 @@ export class PostgresEngine implements BrainEngine { // D7: optional source_id filter. NULL/undefined = scan all sources // (pre-existing behavior); a value scopes to that source so // `gbrain embed --stale --source X` actually does what it says. + // + // v0.41 (D4+D8): NOT (frontmatter ? 'embed_skip') filter applied via + // the always-JOINed pages row. Soft-blocked pages won't surface in + // the stale list; their chunks were deleted at ingest time anyway + // (D9 transition invariant), but the filter is defense-in-depth for + // pre-fix inventory that might still have orphan chunks. if (opts?.sourceId === undefined) { const rows = await sql` SELECT p.slug, cc.chunk_index, cc.chunk_text, cc.chunk_source, @@ -1945,6 +1964,7 @@ export class PostgresEngine implements BrainEngine { FROM content_chunks cc JOIN pages p ON p.id = cc.page_id WHERE cc.embedding IS NULL + AND NOT (COALESCE(p.frontmatter, '{}'::jsonb) ? 'embed_skip') AND (cc.page_id, cc.chunk_index) > (${afterPid}, ${afterIdx}) ORDER BY cc.page_id, cc.chunk_index LIMIT ${limit} @@ -1958,6 +1978,7 @@ export class PostgresEngine implements BrainEngine { JOIN pages p ON p.id = cc.page_id WHERE cc.embedding IS NULL AND p.source_id = ${opts.sourceId} + AND NOT (COALESCE(p.frontmatter, '{}'::jsonb) ? 'embed_skip') AND (cc.page_id, cc.chunk_index) > (${afterPid}, ${afterIdx}) ORDER BY cc.page_id, cc.chunk_index LIMIT ${limit} diff --git a/src/core/sync.ts b/src/core/sync.ts index dc5cfbb06..2506a9d3e 100644 --- a/src/core/sync.ts +++ b/src/core/sync.ts @@ -85,6 +85,12 @@ const CODE_EXTENSIONS = new Set([ // recursive chunker (no tree-sitter grammar), which is the correct // fallback — same path as toml / yaml without language-specific AST. '.tf', '.tfvars', '.hcl', + // v0.41 D2 wave (#1173): SQL via tree-sitter-sql. DerekStride grammar + // chunks DDL (CREATE TABLE/FUNCTION/VIEW/INDEX) and DML (SELECT/INSERT/ + // UPDATE/DELETE) as one chunk per statement. DDL chunks carry + // symbol_name + symbol_type populated for code-def; DML chunks emit + // unnamed so they don't pollute symbol search. + '.sql', ]); /** @@ -497,6 +503,13 @@ export function classifyErrorCode(errorMsg: string): string { } if (/TAKES_HOLDER_INVALID/i.test(errorMsg)) return 'TAKES_HOLDER_INVALID'; + // v0.41 content-sanity gate. Hard-blocks at importFromContent throw + // ContentSanityBlockError whose toString() embeds `PAGE_JUNK_PATTERN:` + // (see src/core/content-sanity.ts PAGE_JUNK_PATTERN_CODE). Soft-blocks + // (oversize alone) don't fail — the page lands with frontmatter.embed_skip + // set and never enters this classifier. + if (/PAGE_JUNK_PATTERN/i.test(errorMsg)) return 'PAGE_JUNK_PATTERN'; + return 'UNKNOWN'; } diff --git a/test/audit/content-sanity-audit.test.ts b/test/audit/content-sanity-audit.test.ts new file mode 100644 index 000000000..70e941305 --- /dev/null +++ b/test/audit/content-sanity-audit.test.ts @@ -0,0 +1,219 @@ +import { describe, test, expect } from 'bun:test'; +import { mkdtempSync, rmSync, existsSync, readFileSync } from 'fs'; +import { join } from 'path'; +import { tmpdir } from 'os'; +import { withEnv } from '../helpers/with-env.ts'; +import { + logContentSanityAssessment, + readRecentContentSanityEvents, + summarizeContentSanityEvents, + computeContentSanityAuditFilename, + type ContentSanityAuditEvent, +} from '../../src/core/audit/content-sanity-audit.ts'; +import type { ContentSanityResult } from '../../src/core/content-sanity.ts'; + +function makeResult(opts: { + bytes?: number; + hard?: boolean; + soft?: boolean; + warn?: boolean; + pattern?: string; + literal?: string; +}): ContentSanityResult { + const junk_pattern_matches: string[] = opts.pattern ? [opts.pattern] : []; + const literal_substring_matches: string[] = opts.literal ? [opts.literal] : []; + const reasons: ContentSanityResult['reasons'] = []; + const reason_messages: string[] = []; + if (opts.soft) { + reasons.push('oversize_block'); + reason_messages.push('PAGE_OVERSIZED: body 600000 bytes'); + } else if (opts.warn) { + reasons.push('oversize_warn'); + reason_messages.push('PAGE_OVERSIZE_WARN: body 100000 bytes'); + } + if (junk_pattern_matches.length > 0) { + reasons.push('junk_pattern'); + reason_messages.push(`PAGE_JUNK_PATTERN: matched ${junk_pattern_matches.join(', ')}`); + } + if (literal_substring_matches.length > 0) { + reasons.push('literal_substring'); + reason_messages.push(`PAGE_JUNK_PATTERN: literal ${literal_substring_matches.join(', ')}`); + } + return { + bytes: opts.bytes ?? 1000, + oversize: !!opts.soft, + junk_pattern_matches, + literal_substring_matches, + reasons, + reason_messages, + shouldHardBlock: !!opts.hard || junk_pattern_matches.length > 0 || literal_substring_matches.length > 0, + shouldSkipEmbed: !!opts.soft && !opts.hard && junk_pattern_matches.length === 0 && literal_substring_matches.length === 0, + }; +} + +describe('computeContentSanityAuditFilename', () => { + test('emits the ISO-week prefix shape', () => { + const name = computeContentSanityAuditFilename(new Date('2026-05-24T07:00:00Z')); + expect(name).toMatch(/^content-sanity-\d{4}-W\d{2}\.jsonl$/); + }); +}); + +describe('logContentSanityAssessment (E2E via tempdir)', () => { + test('writes hard-block event', async () => { + const dir = mkdtempSync(join(tmpdir(), 'cs-audit-hard-')); + try { + await withEnv({ GBRAIN_AUDIT_DIR: dir }, async () => { + const result = makeResult({ hard: true, pattern: 'cloudflare_attention_required', bytes: 287 }); + logContentSanityAssessment('media/articles/foo', 'straylight-brain', result); + const events = readRecentContentSanityEvents(7); + expect(events.length).toBe(1); + expect(events[0].event_type).toBe('hard_block'); + expect(events[0].slug).toBe('media/articles/foo'); + expect(events[0].source_id).toBe('straylight-brain'); + expect(events[0].junk_pattern_matches).toContain('cloudflare_attention_required'); + expect(events[0].bytes).toBe(287); + }); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test('writes soft-block event', async () => { + const dir = mkdtempSync(join(tmpdir(), 'cs-audit-soft-')); + try { + await withEnv({ GBRAIN_AUDIT_DIR: dir }, async () => { + const result = makeResult({ soft: true, bytes: 890_000 }); + logContentSanityAssessment('media/big-transcript', 'default', result); + const events = readRecentContentSanityEvents(7); + expect(events.length).toBe(1); + expect(events[0].event_type).toBe('soft_block'); + expect(events[0].bytes).toBe(890_000); + }); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test('writes warn event', async () => { + const dir = mkdtempSync(join(tmpdir(), 'cs-audit-warn-')); + try { + await withEnv({ GBRAIN_AUDIT_DIR: dir }, async () => { + const result = makeResult({ warn: true, bytes: 100_000 }); + logContentSanityAssessment('notes/long', 'default', result); + const events = readRecentContentSanityEvents(7); + expect(events.length).toBe(1); + expect(events[0].event_type).toBe('warn'); + }); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test('skips no-op rows (no reasons + no bypass)', async () => { + const dir = mkdtempSync(join(tmpdir(), 'cs-audit-noop-')); + try { + await withEnv({ GBRAIN_AUDIT_DIR: dir }, async () => { + const result = makeResult({}); // no reasons fire + logContentSanityAssessment('normal-page', 'default', result); + const events = readRecentContentSanityEvents(7); + expect(events.length).toBe(0); + }); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test('bypass active overrides hard/soft → records as warn with bypass_active flag', async () => { + const dir = mkdtempSync(join(tmpdir(), 'cs-audit-bypass-')); + try { + await withEnv({ GBRAIN_AUDIT_DIR: dir }, async () => { + const result = makeResult({ hard: true, pattern: 'access_denied' }); + logContentSanityAssessment('bypassed', 'default', result, { bypass: true }); + const events = readRecentContentSanityEvents(7); + expect(events.length).toBe(1); + expect(events[0].event_type).toBe('warn'); + expect(events[0].bypass_active).toBe(true); + }); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test('multiple events accumulate in one file', async () => { + const dir = mkdtempSync(join(tmpdir(), 'cs-audit-multi-')); + try { + await withEnv({ GBRAIN_AUDIT_DIR: dir }, async () => { + logContentSanityAssessment('a', 'src', makeResult({ hard: true, pattern: 'access_denied' })); + logContentSanityAssessment('b', 'src', makeResult({ soft: true, bytes: 600000 })); + logContentSanityAssessment('c', 'src', makeResult({ warn: true, bytes: 70000 })); + const events = readRecentContentSanityEvents(7); + expect(events.length).toBe(3); + }); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); +}); + +describe('summarizeContentSanityEvents', () => { + function event(over: Partial): ContentSanityAuditEvent { + return { + ts: new Date().toISOString(), + event_type: 'hard_block', + slug: 'test', + source_id: 'default', + bytes: 100, + junk_pattern_matches: [], + literal_substring_matches: [], + reason_messages: [], + ...over, + }; + } + test('empty input returns zero summary', () => { + const s = summarizeContentSanityEvents([]); + expect(s.total_events).toBe(0); + expect(s.by_type).toEqual({ hard_block: 0, soft_block: 0, warn: 0 }); + expect(s.top_patterns).toEqual([]); + }); + + test('counts by type', () => { + const s = summarizeContentSanityEvents([ + event({ event_type: 'hard_block' }), + event({ event_type: 'hard_block' }), + event({ event_type: 'soft_block' }), + event({ event_type: 'warn' }), + ]); + expect(s.by_type).toEqual({ hard_block: 2, soft_block: 1, warn: 1 }); + expect(s.total_events).toBe(4); + }); + + test('counts by source', () => { + const s = summarizeContentSanityEvents([ + event({ source_id: 'straylight-brain' }), + event({ source_id: 'straylight-brain' }), + event({ source_id: 'default' }), + ]); + expect(s.by_source['straylight-brain']).toBe(2); + expect(s.by_source['default']).toBe(1); + }); + + test('top_patterns sorted desc by count', () => { + const s = summarizeContentSanityEvents([ + event({ junk_pattern_matches: ['cloudflare_attention_required'] }), + event({ junk_pattern_matches: ['cloudflare_attention_required'] }), + event({ junk_pattern_matches: ['cloudflare_attention_required'] }), + event({ junk_pattern_matches: ['access_denied'] }), + ]); + expect(s.top_patterns[0]).toEqual({ name: 'cloudflare_attention_required', count: 3 }); + expect(s.top_patterns[1]).toEqual({ name: 'access_denied', count: 1 }); + }); + + test('literal substring hits count alongside pattern hits', () => { + const s = summarizeContentSanityEvents([ + event({ literal_substring_matches: ['reddit_blocked', 'linkedin_wall'] }), + event({ literal_substring_matches: ['reddit_blocked'] }), + ]); + expect(s.top_patterns).toContainEqual({ name: 'reddit_blocked', count: 2 }); + expect(s.top_patterns).toContainEqual({ name: 'linkedin_wall', count: 1 }); + }); +}); diff --git a/test/batch-projection.test.ts b/test/batch-projection.test.ts new file mode 100644 index 000000000..61d7b42fa --- /dev/null +++ b/test/batch-projection.test.ts @@ -0,0 +1,250 @@ +/** + * v0.41 D4 — batch projection unit tests. + * + * Pure-function math; no DB or LLM needed. Pins the 4 critical contracts: + * + * 1. Cold-start fallback (no history → wide-guess with annotation) + * 2. Unknown model → cost_estimate_unavailable tagged variant + * 3. ±30% confidence band (or sample-stddev-derived when historical) + * 4. Threshold gating respects env-var overrides + * 5. Raise-cap hint fires only when lease cap is binding + meaningful speedup + */ + +import { describe, test, expect } from 'bun:test'; +import { + projectBatch, + formatProjection, + shouldPromptAtThreshold, + type RecentJobStats, +} from '../src/core/minions/batch-projection.ts'; +import { withEnv } from './helpers/with-env.ts'; + +function coldStats(opts: Partial = {}): RecentJobStats { + return { + sample_size: 0, + effective_concurrency: 8, + ...opts, + }; +} + +function warmStats(opts: Partial = {}): RecentJobStats { + return { + sample_size: 100, + mean_latency_ms: 4000, + mean_cost_usd: 0.03, + stddev_cost_usd: 0.01, + effective_concurrency: 8, + ...opts, + }; +} + +describe('projectBatch', () => { + test('cold start: no history → cold_start=true with model-default cost guess', () => { + const p = projectBatch({ + job_count: 100, + model: 'anthropic:claude-sonnet-4-6', + stats: coldStats(), + current_lease_cap: 32, + }); + expect(p.cold_start).toBe(true); + expect(p.total_cost_usd).toBeGreaterThan(0); + expect(p.total_duration_ms).toBeGreaterThan(0); + // 100 jobs at concurrency=8, ~5s each → ≈63s. + expect(p.effective_concurrency).toBe(8); + }); + + test('unknown model → unknown_model tagged variant, total_cost_usd=null', () => { + const p = projectBatch({ + job_count: 100, + model: 'mystery:foo-2', + stats: coldStats(), + current_lease_cap: 32, + }); + expect(p.unknown_model).toBe('foo-2'); + expect(p.total_cost_usd).toBeNull(); + expect(p.cost_band_usd).toBeNull(); + // Duration is still computable from latency × jobs / concurrency. + expect(p.total_duration_ms).toBeGreaterThan(0); + }); + + test('warm window with stddev → uses stddev-derived band (×1.96 ≈ 95%)', () => { + const p = projectBatch({ + job_count: 100, + model: 'anthropic:claude-sonnet-4-6', + stats: warmStats({ mean_cost_usd: 0.05, stddev_cost_usd: 0.02 }), + current_lease_cap: 32, + }); + expect(p.cold_start).toBe(false); + expect(p.total_cost_usd).toBeCloseTo(5.00, 2); // 100 × $0.05 + expect(p.cost_band_usd).toBeCloseTo(100 * 0.02 * 1.96, 1); // ≈ $3.92 + }); + + test('warm window without stddev → blanket ±30% band', () => { + const p = projectBatch({ + job_count: 100, + model: 'anthropic:claude-sonnet-4-6', + stats: warmStats({ stddev_cost_usd: undefined, mean_cost_usd: 0.05 }), + current_lease_cap: 32, + }); + expect(p.cost_band_usd).toBeCloseTo(p.total_cost_usd! * 0.30, 2); + }); + + test('effective_concurrency clamps to min(seen, lease_cap)', () => { + const p = projectBatch({ + job_count: 100, + model: 'anthropic:claude-sonnet-4-6', + stats: warmStats({ effective_concurrency: 16 }), + current_lease_cap: 4, // tighter than seen + }); + expect(p.effective_concurrency).toBe(4); + }); + + test('raise_cap_hint fires when lease is binding AND a 4x raise meaningfully helps', () => { + const p = projectBatch({ + job_count: 1000, + model: 'anthropic:claude-sonnet-4-6', + stats: warmStats({ + effective_concurrency: 32, // seen lots; lease is the constraint + lease_headroom: 0.05, // hot — nearly saturated + }), + current_lease_cap: 8, + }); + expect(p.raise_cap_hint).toBeDefined(); + expect(p.raise_cap_hint).toContain('GBRAIN_ANTHROPIC_MAX_INFLIGHT'); + }); + + test('no raise_cap_hint when not binding', () => { + const p = projectBatch({ + job_count: 100, + model: 'anthropic:claude-sonnet-4-6', + stats: warmStats({ lease_headroom: 0.8 }), // plenty of room + current_lease_cap: 32, + }); + expect(p.raise_cap_hint).toBeUndefined(); + }); + + test('no raise_cap_hint when already at the ceiling', () => { + const p = projectBatch({ + job_count: 100, + model: 'anthropic:claude-sonnet-4-6', + stats: warmStats({ lease_headroom: 0.05 }), + current_lease_cap: 128, // at ceiling + }); + expect(p.raise_cap_hint).toBeUndefined(); + }); +}); + +describe('formatProjection', () => { + test('known model: prints cost + duration + bands', () => { + const s = formatProjection({ + total_duration_ms: 600_000, // 10 min + total_cost_usd: 2.40, + cost_band_usd: 0.72, + duration_band_ms: 180_000, + effective_concurrency: 8, + cold_start: false, + }); + expect(s).toContain('$2.40'); + expect(s).toContain('±$0.72'); + expect(s).toContain('10min'); + expect(s).toContain('concurrency=8'); + expect(s).not.toContain('no history'); + }); + + test('cold start: includes annotation', () => { + const s = formatProjection({ + total_duration_ms: 600_000, + total_cost_usd: 2.40, + cost_band_usd: 0.72, + duration_band_ms: 180_000, + effective_concurrency: 8, + cold_start: true, + }); + expect(s).toContain('no history'); + expect(s).toContain('wide guess'); + }); + + test('unknown model: replaces cost section with explanation', () => { + const s = formatProjection({ + total_duration_ms: 600_000, + total_cost_usd: null, + cost_band_usd: null, + duration_band_ms: 180_000, + effective_concurrency: 8, + cold_start: true, + unknown_model: 'foo-2', + }); + expect(s).toContain('cost estimate unavailable'); + expect(s).toContain('foo-2'); + expect(s).toContain('pricing maps'); + }); + + test('raise-cap hint surfaces inline', () => { + const s = formatProjection({ + total_duration_ms: 600_000, + total_cost_usd: 2.40, + cost_band_usd: 0.72, + duration_band_ms: 180_000, + effective_concurrency: 8, + cold_start: false, + raise_cap_hint: 'raise GBRAIN_ANTHROPIC_MAX_INFLIGHT to 32 to finish in ~3min', + }); + expect(s).toContain('raise GBRAIN_ANTHROPIC_MAX_INFLIGHT to 32'); + }); +}); + +describe('shouldPromptAtThreshold', () => { + test('prompts at >$5 default threshold', () => { + expect( + shouldPromptAtThreshold({ + total_duration_ms: 60_000, + total_cost_usd: 5.01, + cost_band_usd: 1, + duration_band_ms: 10_000, + effective_concurrency: 8, + cold_start: false, + }), + ).toBe(true); + }); + + test('prompts at >30min default threshold', () => { + expect( + shouldPromptAtThreshold({ + total_duration_ms: 31 * 60_000, + total_cost_usd: 0.50, + cost_band_usd: 0.10, + duration_band_ms: 60_000, + effective_concurrency: 8, + cold_start: false, + }), + ).toBe(true); + }); + + test('does NOT prompt below both thresholds', () => { + expect( + shouldPromptAtThreshold({ + total_duration_ms: 60_000, + total_cost_usd: 2.00, + cost_band_usd: 0.50, + duration_band_ms: 10_000, + effective_concurrency: 8, + cold_start: false, + }), + ).toBe(false); + }); + + test('env var GBRAIN_BATCH_PROMPT_THRESHOLD_USD lowers the prompt floor', async () => { + await withEnv({ GBRAIN_BATCH_PROMPT_THRESHOLD_USD: '1' }, async () => { + expect( + shouldPromptAtThreshold({ + total_duration_ms: 60_000, + total_cost_usd: 1.50, + cost_band_usd: 0.20, + duration_band_ms: 10_000, + effective_concurrency: 8, + cold_start: false, + }), + ).toBe(true); + }); + }); +}); diff --git a/test/budget-tracker.test.ts b/test/budget-tracker.test.ts new file mode 100644 index 000000000..81afa5478 --- /dev/null +++ b/test/budget-tracker.test.ts @@ -0,0 +1,230 @@ +/** + * v0.41 D5 + Eng D7 + Eng D10 — budget-tracker tests. + * + * Pins the reservation pattern's load-bearing contracts: + * + * - reserveBudget: CAS UPDATE prevents overspend even across parallel + * children of the same owner. + * - Eng D10 NULL-bypass: jobs without an owner skip reservation cleanly. + * - Eng D10 owner-deleted disambiguation: pruned-owner orphans throw + * BudgetOwnerDeleted instead of silently bypassing. + * - refundBudget returns unspent cents to the owner row. + * - haltBudgetSubtree walks all waiting/delayed descendants of an owner. + * - Audit rows written to minion_budget_log per event. + */ + +import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test'; +import { PGLiteEngine } from '../src/core/pglite-engine.ts'; +import { MinionQueue } from '../src/core/minions/queue.ts'; +import { + reserveBudget, + refundBudget, + setOwnerBudget, + inheritBudgetOwner, + haltBudgetSubtree, + getBudgetOwner, +} from '../src/core/minions/budget-tracker.ts'; + +let engine: PGLiteEngine; +let queue: MinionQueue; + +beforeAll(async () => { + engine = new PGLiteEngine(); + await engine.connect({ database_url: '' }); + await engine.initSchema(); + queue = new MinionQueue(engine); +}); + +afterAll(async () => { + await engine.disconnect(); +}); + +beforeEach(async () => { + await engine.executeRaw('DELETE FROM minion_budget_log'); + await engine.executeRaw('DELETE FROM minion_jobs'); +}); + +describe('reserveBudget (CAS pattern)', () => { + test('reserves cents from owner balance; CAS succeeds when balance >= cost', async () => { + const owner = await queue.add('subagent', {}, {}, { allowProtectedSubmit: true }); + await setOwnerBudget(engine, owner.id, 1.0); // $1.00 = 100¢ + const child = await queue.add('subagent', {}, {}, { allowProtectedSubmit: true }); + await inheritBudgetOwner(engine, child.id, owner.id); + + const outcome = await reserveBudget(engine, child.id, 30); + expect(outcome.kind).toBe('reserved'); + if (outcome.kind === 'reserved') { + expect(outcome.new_balance_cents).toBe(70); + expect(outcome.reserved_cents).toBe(30); + } + // Audit row written. + const audit = await engine.executeRaw<{ count: string }>( + `SELECT count(*)::text AS count FROM minion_budget_log WHERE event_type = 'reserved'`, + ); + expect(parseInt(audit[0]!.count, 10)).toBe(1); + }); + + test('CAS miss returns exhausted; balance + requested visible for audit', async () => { + const owner = await queue.add('subagent', {}, {}, { allowProtectedSubmit: true }); + await setOwnerBudget(engine, owner.id, 0.10); // $0.10 = 10¢ + const child = await queue.add('subagent', {}, {}, { allowProtectedSubmit: true }); + await inheritBudgetOwner(engine, child.id, owner.id); + + const outcome = await reserveBudget(engine, child.id, 50); + expect(outcome.kind).toBe('exhausted'); + if (outcome.kind === 'exhausted') { + expect(outcome.balance_at_attempt).toBe(10); + expect(outcome.requested_cents).toBe(50); + } + // Halted audit row written. + const audit = await engine.executeRaw<{ count: string }>( + `SELECT count(*)::text AS count FROM minion_budget_log WHERE event_type = 'halted'`, + ); + expect(parseInt(audit[0]!.count, 10)).toBe(1); + }); + + test('Eng D10: job with no owner returns no_budget (clean bypass)', async () => { + const child = await queue.add('subagent', {}, {}, { allowProtectedSubmit: true }); + // No setOwnerBudget call → budget_owner_job_id stays NULL. + const outcome = await reserveBudget(engine, child.id, 50); + expect(outcome.kind).toBe('no_budget'); + }); + + test('Eng D10: deleted owner (FK NULL but root denormalized) throws owner_deleted', async () => { + const owner = await queue.add('subagent', {}, {}, { allowProtectedSubmit: true }); + await setOwnerBudget(engine, owner.id, 1.0); + const child = await queue.add('subagent', {}, {}, { allowProtectedSubmit: true }); + await inheritBudgetOwner(engine, child.id, owner.id); + + // Hard-delete the owner — simulates `gbrain jobs prune` race. + // SET NULL FK fires on the child's budget_owner_job_id but the + // immutable budget_root_owner_id persists. + await engine.executeRaw('DELETE FROM minion_jobs WHERE id = $1', [owner.id]); + + const outcome = await reserveBudget(engine, child.id, 50); + expect(outcome.kind).toBe('owner_deleted'); + if (outcome.kind === 'owner_deleted') { + expect(outcome.root_owner_id).toBe(owner.id); + } + }); +}); + +describe('refundBudget', () => { + test('returns unspent cents to owner', async () => { + const owner = await queue.add('subagent', {}, {}, { allowProtectedSubmit: true }); + await setOwnerBudget(engine, owner.id, 1.0); + const child = await queue.add('subagent', {}, {}, { allowProtectedSubmit: true }); + await inheritBudgetOwner(engine, child.id, owner.id); + + await reserveBudget(engine, child.id, 50); // balance 100→50 + await refundBudget(engine, child.id, owner.id, 30); // balance 50→80 + + const after = await getBudgetOwner(engine, owner.id); + expect(after!.budget_remaining_cents).toBe(80); + + // Refund audit row. + const audit = await engine.executeRaw<{ count: string }>( + `SELECT count(*)::text AS count FROM minion_budget_log WHERE event_type = 'refunded'`, + ); + expect(parseInt(audit[0]!.count, 10)).toBe(1); + }); + + test('refund of 0 cents is a no-op', async () => { + const owner = await queue.add('subagent', {}, {}, { allowProtectedSubmit: true }); + await setOwnerBudget(engine, owner.id, 1.0); + const child = await queue.add('subagent', {}, {}, { allowProtectedSubmit: true }); + await inheritBudgetOwner(engine, child.id, owner.id); + await reserveBudget(engine, child.id, 50); + + await refundBudget(engine, child.id, owner.id, 0); + const after = await getBudgetOwner(engine, owner.id); + expect(after!.budget_remaining_cents).toBe(50); // unchanged + + const audit = await engine.executeRaw<{ count: string }>( + `SELECT count(*)::text AS count FROM minion_budget_log WHERE event_type = 'refunded'`, + ); + expect(parseInt(audit[0]!.count, 10)).toBe(0); // no audit row either + }); +}); + +describe('haltBudgetSubtree', () => { + test('flips waiting + delayed children to dead with reason', async () => { + const owner = await queue.add('subagent', {}, {}, { allowProtectedSubmit: true }); + await setOwnerBudget(engine, owner.id, 1.0); + const c1 = await queue.add('subagent', {}, {}, { allowProtectedSubmit: true }); + const c2 = await queue.add('subagent', {}, {}, { allowProtectedSubmit: true }); + const c3 = await queue.add('subagent', {}, {}, { allowProtectedSubmit: true }); + for (const c of [c1, c2, c3]) { + await inheritBudgetOwner(engine, c.id, owner.id); + } + + // Move c2 to 'delayed' to verify both waiting AND delayed are halted. + await engine.executeRaw( + `UPDATE minion_jobs SET status = 'delayed', delay_until = now() + interval '1 hour' WHERE id = $1`, + [c2.id], + ); + + const halted = await haltBudgetSubtree(engine, owner.id, 'budget_exhausted'); + expect(halted).toBe(3); + + // Verify each child is dead. + for (const c of [c1, c2, c3]) { + const r = await engine.executeRaw<{ status: string; error_text: string }>( + `SELECT status, error_text FROM minion_jobs WHERE id = $1`, + [c.id], + ); + expect(r[0]!.status).toBe('dead'); + expect(r[0]!.error_text).toContain('budget_exhausted'); + } + }); + + test('does NOT touch active children (they finish their current turn cleanly)', async () => { + const owner = await queue.add('subagent', {}, {}, { allowProtectedSubmit: true }); + await setOwnerBudget(engine, owner.id, 1.0); + const c1 = await queue.add('subagent', {}, {}, { allowProtectedSubmit: true }); + await inheritBudgetOwner(engine, c1.id, owner.id); + // Move c1 to 'active' to simulate in-flight. + await engine.executeRaw( + `UPDATE minion_jobs SET status = 'active', lock_token = 'lock', lock_until = now() + interval '1 minute' WHERE id = $1`, + [c1.id], + ); + + const halted = await haltBudgetSubtree(engine, owner.id, 'budget_exhausted'); + expect(halted).toBe(0); // active jobs not flipped + + const r = await engine.executeRaw<{ status: string }>( + `SELECT status FROM minion_jobs WHERE id = $1`, + [c1.id], + ); + expect(r[0]!.status).toBe('active'); + }); +}); + +describe('inheritBudgetOwner', () => { + test('child mirrors parent owner + root_owner; does NOT copy balance', async () => { + const owner = await queue.add('subagent', {}, {}, { allowProtectedSubmit: true }); + await setOwnerBudget(engine, owner.id, 1.0); + const child = await queue.add('subagent', {}, {}, { allowProtectedSubmit: true }); + await inheritBudgetOwner(engine, child.id, owner.id); + + const info = await getBudgetOwner(engine, child.id); + expect(info!.budget_owner_job_id).toBe(owner.id); + expect(info!.budget_root_owner_id).toBe(owner.id); + // Critical: child's balance stays NULL — only owner row holds spendable cents. + expect(info!.budget_remaining_cents).toBeNull(); + }); + + test('grandchild inherits the original owner (chain depth 2)', async () => { + const owner = await queue.add('subagent', {}, {}, { allowProtectedSubmit: true }); + await setOwnerBudget(engine, owner.id, 1.0); + const child = await queue.add('subagent', {}, {}, { allowProtectedSubmit: true }); + await inheritBudgetOwner(engine, child.id, owner.id); + const grand = await queue.add('subagent', {}, {}, { allowProtectedSubmit: true }); + await inheritBudgetOwner(engine, grand.id, child.id); + + const info = await getBudgetOwner(engine, grand.id); + // Grandchild should point at the ORIGINAL owner, not its immediate parent. + expect(info!.budget_owner_job_id).toBe(owner.id); + expect(info!.budget_root_owner_id).toBe(owner.id); + }); +}); diff --git a/test/check-system-of-record.test.ts b/test/check-system-of-record.test.ts index cd15423aa..3632f6f8b 100644 --- a/test/check-system-of-record.test.ts +++ b/test/check-system-of-record.test.ts @@ -24,7 +24,18 @@ import { join } from 'node:path'; const SCRIPT_PATH = join(import.meta.dir, '..', 'scripts', 'check-system-of-record.sh'); function runGate(cwd: string): { code: number; stdout: string; stderr: string } { - const r = spawnSync('bash', [SCRIPT_PATH], { cwd, encoding: 'utf-8', timeout: 30_000 }); + // GBRAIN_SCAN_ROOT pins the gate's scan directory to our fake repo. + // Without this, `git rev-parse --show-toplevel` inside the gate can walk + // up past our /tmp/gate-test-* fakeRepo (when its `git init -q` silently + // failed under shard-concurrency load) into the real gbrain repo and + // scan the clean src+scripts — false-negative the negative-case test. + // v0.40.10 flake-hardening fix. + const r = spawnSync('bash', [SCRIPT_PATH], { + cwd, + encoding: 'utf-8', + timeout: 30_000, + env: { ...process.env, GBRAIN_SCAN_ROOT: cwd }, + }); return { code: r.status ?? -1, stdout: r.stdout ?? '', diff --git a/test/chunkers/code.test.ts b/test/chunkers/code.test.ts index 66e813392..2580ff94e 100644 --- a/test/chunkers/code.test.ts +++ b/test/chunkers/code.test.ts @@ -16,7 +16,7 @@ describe('CHUNKER_VERSION', () => { }); describe('detectCodeLanguage', () => { - test('recognizes all 29 supported extensions', () => { + test('recognizes all 30 supported extensions', () => { const cases: Record = { 'foo.ts': 'typescript', 'foo.tsx': 'tsx', 'foo.mts': 'typescript', 'foo.cts': 'typescript', 'foo.js': 'javascript', 'foo.jsx': 'javascript', 'foo.mjs': 'javascript', 'foo.cjs': 'javascript', @@ -30,6 +30,8 @@ describe('detectCodeLanguage', () => { 'foo.zig': 'zig', 'foo.sol': 'solidity', 'foo.sh': 'bash', 'foo.css': 'css', 'foo.html': 'html', 'foo.vue': 'vue', 'foo.json': 'json', 'foo.yaml': 'yaml', 'foo.toml': 'toml', + // v0.41 D2 wave: SQL via DerekStride/tree-sitter-sql. + 'foo.sql': 'sql', 'migrations/001_init.sql': 'sql', }; for (const [path, expected] of Object.entries(cases)) { expect(detectCodeLanguage(path)).toBe(expected as any); @@ -45,6 +47,178 @@ describe('detectCodeLanguage', () => { test('is case-insensitive', () => { expect(detectCodeLanguage('Main.GO')).toBe('go'); expect(detectCodeLanguage('App.TSX')).toBe('tsx'); + expect(detectCodeLanguage('Schema.SQL')).toBe('sql'); + }); +}); + +// v0.41 D2 wave (#1173) — SQL via DerekStride/tree-sitter-sql. +// Step 0 inspection 2026-05-24 verified the grammar wraps every top-level +// statement in `program > statement > `. Tests assert the chunker +// dives through the wrapper and extracts the target name from DDL kinds. +describe('chunkCodeText — SQL', () => { + test('CREATE TABLE extracts table name as symbolName', async () => { + const sql = `CREATE TABLE this_table_name_is_long_enough_to_avoid_merging ( + id SERIAL PRIMARY KEY, + email TEXT NOT NULL UNIQUE, + created_at TIMESTAMP DEFAULT NOW(), + updated_at TIMESTAMP, + deleted_at TIMESTAMP, + metadata JSONB +);`; + // Use a small chunkSizeTokens so the single statement isn't merged + // with siblings (test fixture has only one, so no merging anyway). + const result = await chunkCodeText(sql, 'migrations/users.sql', { chunkSizeTokens: 50 }); + expect(result.length).toBeGreaterThanOrEqual(1); + const c = result[0]!; + expect(c.metadata.language).toBe('sql'); + expect(c.metadata.symbolName).toBe('this_table_name_is_long_enough_to_avoid_merging'); + expect(c.metadata.symbolType).toBe('table'); + expect(c.text).toContain('[SQL]'); + }); + + test('CREATE FUNCTION with $$ body extracts function name + parses cleanly', async () => { + const sql = `CREATE OR REPLACE FUNCTION get_user_by_email_long_function_name_here_for_no_merge(p_email TEXT) +RETURNS users AS $$ + SELECT * FROM users WHERE email = p_email LIMIT 1; +$$ LANGUAGE SQL;`; + const result = await chunkCodeText(sql, 'migrations/fn.sql', { chunkSizeTokens: 50 }); + expect(result.length).toBeGreaterThanOrEqual(1); + const c = result.find(c => c.metadata.symbolName === 'get_user_by_email_long_function_name_here_for_no_merge'); + expect(c).toBeDefined(); + expect(c!.metadata.symbolType).toBe('function'); + expect(c!.metadata.language).toBe('sql'); + // Dollar-quoted body must NOT crash the parser (codex F15 regression). + expect(c!.text).toContain('$$'); + }); + + test('CREATE INDEX extracts index name', async () => { + const sql = `CREATE INDEX idx_a_b_c_d_e_f_g_long ON users (email, created_at, updated_at, deleted_at);`; + const result = await chunkCodeText(sql, 'idx.sql', { chunkSizeTokens: 50 }); + const c = result.find(c => c.metadata.symbolType === 'index'); + expect(c).toBeDefined(); + expect(c!.metadata.symbolName).toBe('idx_a_b_c_d_e_f_g_long'); + }); + + test('CREATE VIEW extracts view name', async () => { + const sql = `CREATE VIEW active_users_dashboard_view AS + SELECT id, email FROM users WHERE deleted_at IS NULL AND active = true;`; + const result = await chunkCodeText(sql, 'view.sql', { chunkSizeTokens: 50 }); + const c = result.find(c => c.metadata.symbolType === 'view'); + expect(c).toBeDefined(); + expect(c!.metadata.symbolName).toBe('active_users_dashboard_view'); + }); + + test('ALTER TABLE extracts table name', async () => { + const sql = `ALTER TABLE long_table_name_here_so_it_does_not_merge_with_sibling + ADD COLUMN created_at TIMESTAMP DEFAULT NOW(), + ADD COLUMN updated_at TIMESTAMP;`; + const result = await chunkCodeText(sql, 'alter.sql', { chunkSizeTokens: 50 }); + const c = result.find(c => c.metadata.symbolType === 'table'); + expect(c).toBeDefined(); + expect(c!.metadata.symbolName).toBe('long_table_name_here_so_it_does_not_merge_with_sibling'); + }); + + test('DML statements emit chunks but with symbolName=null (DDL signal only)', async () => { + const sql = `INSERT INTO users (email) VALUES ('a@b.com') RETURNING id, email, created_at;`; + const result = await chunkCodeText(sql, 'dml.sql', { chunkSizeTokens: 50 }); + expect(result.length).toBeGreaterThanOrEqual(1); + // The chunk emits but symbolName stays null — DML doesn't contribute + // to code-def. symbolType remains the underlying statement kind via + // normalizeSymbolType's fallback (`insert` here, unmapped → "insert"). + expect(result[0]!.metadata.symbolName).toBeNull(); + expect(result[0]!.metadata.language).toBe('sql'); + }); + + test('mixed DDL + DML emits per-statement chunks, only DDL gets symbolName', async () => { + const sql = `CREATE TABLE long_mixed_table_name_for_no_merge_with_dml_below_it (id INT PRIMARY KEY, x TEXT, y TEXT, z TEXT); +INSERT INTO long_mixed_table_name_for_no_merge_with_dml_below_it (id, x) VALUES (1, 'aaaaaaaaaa'); +INSERT INTO long_mixed_table_name_for_no_merge_with_dml_below_it (id, x) VALUES (2, 'bbbbbbbbbb'); +SELECT * FROM long_mixed_table_name_for_no_merge_with_dml_below_it WHERE id = 1 ORDER BY x;`; + const result = await chunkCodeText(sql, 'mixed.sql', { chunkSizeTokens: 50 }); + // Should have chunks for all 4 statements (DDL+DML each emit). + expect(result.length).toBeGreaterThanOrEqual(2); + const namedChunks = result.filter(c => c.metadata.symbolName !== null); + expect(namedChunks.length).toBeGreaterThanOrEqual(1); + const ddl = namedChunks.find(c => c.metadata.symbolName === 'long_mixed_table_name_for_no_merge_with_dml_below_it'); + expect(ddl).toBeDefined(); + expect(ddl!.metadata.symbolType).toBe('table'); + }); + + test('header includes "[SQL]" language tag', async () => { + const sql = 'CREATE TABLE x (id INT);'; + const result = await chunkCodeText(sql, 'x.sql'); + expect(result[0]!.text).toMatch(/^\[SQL\]/); + }); + + test('does not crash on invalid SQL', async () => { + // Per Step 0: even "SELECT FROM WHERE" parses to a select node, no + // throw. This pins that we don't regress to throwing. + const sql = 'SELECT FROM WHERE'; + const result = await chunkCodeText(sql, 'bad.sql', { chunkSizeTokens: 50 }); + expect(result.length).toBeGreaterThanOrEqual(1); + // The chunk emits; symbol_name stays null. + expect(result[0]!.metadata.language).toBe('sql'); + }); + + test('CREATE TRIGGER extracts trigger name + symbolType=trigger', async () => { + const sql = `CREATE TRIGGER long_audit_trigger_for_email_changes_on_users_table + AFTER UPDATE ON users + FOR EACH ROW + EXECUTE FUNCTION log_email_change();`; + const result = await chunkCodeText(sql, 'trig.sql', { chunkSizeTokens: 50 }); + const c = result.find(c => c.metadata.symbolType === 'trigger'); + expect(c).toBeDefined(); + expect(c!.metadata.symbolName).toBe('long_audit_trigger_for_email_changes_on_users_table'); + }); + + test('CREATE TYPE extracts enum name + symbolType=type', async () => { + const sql = `CREATE TYPE long_user_role_enum_avoid_merger_padding AS ENUM ('admin', 'member', 'guest', 'auditor');`; + const result = await chunkCodeText(sql, 'types.sql', { chunkSizeTokens: 50 }); + const c = result.find(c => c.metadata.symbolType === 'type'); + expect(c).toBeDefined(); + expect(c!.metadata.symbolName).toBe('long_user_role_enum_avoid_merger_padding'); + }); + + test('CREATE PROCEDURE extracts name + symbolType=procedure', async () => { + const sql = `CREATE PROCEDURE long_archive_old_users_procedure_no_merge(days_old INT) +LANGUAGE SQL AS $$ + UPDATE users SET deleted_at = NOW() WHERE last_login_at < NOW() - INTERVAL '1 day' * days_old; +$$;`; + const result = await chunkCodeText(sql, 'proc.sql', { chunkSizeTokens: 50 }); + const c = result.find(c => c.metadata.symbolType === 'procedure'); + expect(c).toBeDefined(); + expect(c!.metadata.symbolName).toBe('long_archive_old_users_procedure_no_merge'); + }); + + test('CREATE SCHEMA extracts schema name + symbolType=schema', async () => { + const sql = `CREATE SCHEMA IF NOT EXISTS analytics_long_schema_name_avoid_merge AUTHORIZATION analytics_owner;`; + const result = await chunkCodeText(sql, 'sch.sql', { chunkSizeTokens: 50 }); + const c = result.find(c => c.metadata.symbolType === 'schema'); + // Schema may not always be reachable depending on grammar version; + // accept either correct extraction OR null (test that it doesn't crash). + if (c) { + expect(c.metadata.symbolName).toBe('analytics_long_schema_name_avoid_merge'); + } + // Always: chunk emits, language is sql. + expect(result.length).toBeGreaterThanOrEqual(1); + expect(result[0]!.metadata.language).toBe('sql'); + }); + + test('header symbolType for SQL chunks reflects inner DDL kind, not "statement"', async () => { + const sql = `CREATE TABLE structured_header_test_table_long_name (id INT, name TEXT, value TEXT, ts TIMESTAMP);`; + const result = await chunkCodeText(sql, 'header.sql', { chunkSizeTokens: 50 }); + expect(result[0]!.text).toMatch(/^\[SQL\][^\n]*\btable\b/); + expect(result[0]!.text).not.toMatch(/\bstatement\b/i); + }); + + test('empty SQL input returns empty chunk array', async () => { + const result = await chunkCodeText('', 'empty.sql'); + expect(result).toEqual([]); + }); + + test('SQL-only whitespace returns empty chunk array', async () => { + const result = await chunkCodeText(' \n\n \t \n', 'whitespace.sql'); + expect(result).toEqual([]); }); }); diff --git a/test/content-sanity-literals.test.ts b/test/content-sanity-literals.test.ts new file mode 100644 index 000000000..3cea32e94 --- /dev/null +++ b/test/content-sanity-literals.test.ts @@ -0,0 +1,110 @@ +import { describe, test, expect } from 'bun:test'; +import { parseLiteralsContent } from '../src/core/content-sanity-literals.ts'; + +describe('parseLiteralsContent — operator file parser', () => { + test('empty input returns empty list', () => { + expect(parseLiteralsContent('')).toEqual([]); + }); + + test('only-comments input returns empty list', () => { + expect(parseLiteralsContent('# comment\n# another\n')).toEqual([]); + }); + + test('only-blanks returns empty list', () => { + expect(parseLiteralsContent('\n\n\n')).toEqual([]); + }); + + test('single bare literal yields one entry with auto-generated name', () => { + const out = parseLiteralsContent("You're being blocked\n"); + expect(out.length).toBe(1); + expect(out[0].substring).toBe("You're being blocked"); + expect(out[0].name).toBe('operator_literal_0'); + expect(out[0].applies_to).toBe('both'); + }); + + test('name directive on preceding comment binds to next literal', () => { + const input = `# name=reddit_blocked +You're being blocked +`; + const out = parseLiteralsContent(input); + expect(out.length).toBe(1); + expect(out[0].name).toBe('reddit_blocked'); + expect(out[0].substring).toBe("You're being blocked"); + }); + + test('multiple directives merge into the next literal', () => { + const input = `# name=linkedin_wall +# applies_to=body +Sign in to your account +`; + const out = parseLiteralsContent(input); + expect(out[0].name).toBe('linkedin_wall'); + expect(out[0].applies_to).toBe('body'); + expect(out[0].substring).toBe('Sign in to your account'); + }); + + test('blank line between directive and literal resets binding', () => { + const input = `# name=should_not_stick + +You're being blocked +`; + const out = parseLiteralsContent(input); + expect(out[0].name).toBe('operator_literal_0'); // auto-generated, not "should_not_stick" + }); + + test('directives only bind to the next literal, then reset', () => { + const input = `# name=first +First literal +# name=second +Second literal +Third literal +`; + const out = parseLiteralsContent(input); + expect(out.length).toBe(3); + expect(out[0].name).toBe('first'); + expect(out[1].name).toBe('second'); + // The auto-name index counts UNNAMED entries only — so the third + // (first un-named) is operator_literal_0, not _2. + expect(out[2].name).toBe('operator_literal_0'); + }); + + test('invalid applies_to value falls through to default both', () => { + const input = `# applies_to=invalid_scope +something +`; + const out = parseLiteralsContent(input); + expect(out[0].applies_to).toBe('both'); + }); + + test('unknown directives ignored without throwing', () => { + const input = `# foo=bar +# applies_to=body +literal +`; + const out = parseLiteralsContent(input); + expect(out[0].applies_to).toBe('body'); + }); + + test('regex meta-characters in literal stay literal (no compile)', () => { + // The loader does NOT call new RegExp() — literals are passed + // through as-is and assessContentSanity uses .includes() for matching. + const input = '(a+)+b\n'; + const out = parseLiteralsContent(input); + expect(out[0].substring).toBe('(a+)+b'); + }); + + test('trims trailing whitespace on literal', () => { + const input = 'literal-with-trailing-space \n'; + const out = parseLiteralsContent(input); + expect(out[0].substring).toBe('literal-with-trailing-space'); + }); + + test('CRLF line endings handled', () => { + const input = '# name=cr\r\nliteral\r\n'; + const out = parseLiteralsContent(input); + expect(out.length).toBe(1); + // The trim() preserves \r-stripping. The directive may or may not + // capture trailing \r — test the substring is reasonably clean. + expect(out[0].substring.replace(/\r$/, '')).toBe('literal'); + }); +}); diff --git a/test/content-sanity.test.ts b/test/content-sanity.test.ts new file mode 100644 index 000000000..3e93e1623 --- /dev/null +++ b/test/content-sanity.test.ts @@ -0,0 +1,416 @@ +import { describe, test, expect } from 'bun:test'; +import { + assessContentSanity, + ContentSanityBlockError, + BUILT_IN_JUNK_PATTERNS, + PAGE_JUNK_PATTERN_CODE, + DEFAULT_BYTES_WARN, + DEFAULT_BYTES_BLOCK, + type OperatorLiteral, +} from '../src/core/content-sanity.ts'; + +// ─── BOUNDARIES ─────────────────────────────────────────────── + +describe('assessContentSanity — size boundaries', () => { + test('empty body returns 0 bytes and no trips', () => { + const r = assessContentSanity({ compiled_truth: '', timeline: '', title: '' }); + expect(r.bytes).toBe(0); + expect(r.oversize).toBe(false); + expect(r.shouldHardBlock).toBe(false); + expect(r.shouldSkipEmbed).toBe(false); + expect(r.reasons).toEqual([]); + }); + + test('bytes counts compiled_truth + timeline (Codex r2 #7)', () => { + // Without timeline a check might miss huge timeline sections; the + // assessor must sum both. Use ASCII for byteLength === length. + const ct = 'a'.repeat(1000); + const tl = 'b'.repeat(2000); + const r = assessContentSanity({ compiled_truth: ct, timeline: tl, title: '' }); + expect(r.bytes).toBeGreaterThanOrEqual(3000); // + the join '\n' + expect(r.bytes).toBeLessThan(3010); + }); + + test('bytes uses UTF-8 octets, not character count', () => { + // CJK chars: each takes 3 UTF-8 bytes. 100 chars → 300 bytes. + const ct = '世'.repeat(100); + const r = assessContentSanity({ compiled_truth: ct, timeline: '', title: '' }); + expect(r.bytes).toBe(300); + }); + + test('exactly at warn threshold does NOT fire warn (strict >)', () => { + const r = assessContentSanity({ + compiled_truth: 'a'.repeat(50_000), + timeline: '', + title: '', + bytes_warn: 50_000, + bytes_block: 500_000, + }); + expect(r.reasons).not.toContain('oversize_warn'); + expect(r.reasons).not.toContain('oversize_block'); + }); + + test('above warn but below block → oversize_warn only', () => { + const r = assessContentSanity({ + compiled_truth: 'a'.repeat(100_000), + timeline: '', + title: '', + }); + expect(r.reasons).toContain('oversize_warn'); + expect(r.reasons).not.toContain('oversize_block'); + expect(r.shouldHardBlock).toBe(false); + expect(r.shouldSkipEmbed).toBe(false); + }); + + test('above block threshold → oversize_block + shouldSkipEmbed', () => { + const r = assessContentSanity({ + compiled_truth: 'a'.repeat(600_000), + timeline: '', + title: '', + }); + expect(r.oversize).toBe(true); + expect(r.reasons).toContain('oversize_block'); + expect(r.reasons).not.toContain('oversize_warn'); // not double-pushed + expect(r.shouldSkipEmbed).toBe(true); + expect(r.shouldHardBlock).toBe(false); + }); + + test('the original 890K reproduction trips block alone (no junk)', () => { + // 890K of clean text (no Cloudflare phrases) → soft-block only. + const r = assessContentSanity({ + compiled_truth: 'normal prose. '.repeat(70_000), // ~890K bytes + timeline: '', + title: 'A Long Article', + }); + expect(r.shouldSkipEmbed).toBe(true); + expect(r.shouldHardBlock).toBe(false); + }); + + test('custom thresholds override defaults', () => { + const r = assessContentSanity({ + compiled_truth: 'a'.repeat(150), + timeline: '', + title: '', + bytes_warn: 100, + bytes_block: 200, + }); + expect(r.reasons).toContain('oversize_warn'); + }); + + test('defaults are exported and reasonable', () => { + expect(DEFAULT_BYTES_WARN).toBe(50_000); + expect(DEFAULT_BYTES_BLOCK).toBe(500_000); + }); +}); + +// ─── 6 BUILT-IN PATTERNS ────────────────────────────────────── + +describe('assessContentSanity — built-in junk patterns', () => { + test('built-in pattern count is locked at 6 (D3 dropped empty_body_with_source_url)', () => { + expect(BUILT_IN_JUNK_PATTERNS.length).toBe(6); + const names = BUILT_IN_JUNK_PATTERNS.map((p) => p.name); + expect(names).toContain('cloudflare_attention_required'); + expect(names).toContain('cloudflare_just_a_moment'); + expect(names).toContain('cloudflare_ray_id'); + expect(names).toContain('access_denied'); + expect(names).toContain('captcha_required'); + expect(names).toContain('error_page_title'); + // D3 regression: this rule was dropped. If it ever returns, the test + // count above bumps to 7 deliberately. + expect(names).not.toContain('empty_body_with_source_url'); + }); + + test('built-in patterns all compile (module-load safety net)', () => { + for (const p of BUILT_IN_JUNK_PATTERNS) { + expect(p.pattern).toBeInstanceOf(RegExp); + expect(() => p.pattern.test('test input')).not.toThrow(); + } + }); + + test('cloudflare_attention_required fires on real-world title', () => { + const r = assessContentSanity({ + compiled_truth: '', + timeline: '', + title: 'Attention Required! | Cloudflare', + }); + expect(r.junk_pattern_matches).toContain('cloudflare_attention_required'); + expect(r.shouldHardBlock).toBe(true); + }); + + test('cloudflare_just_a_moment requires BOTH signals (no false-positive on prose)', () => { + // Just the words "Just a moment..." alone does NOT fire (legitimate + // writing might include it). + const r1 = assessContentSanity({ + compiled_truth: 'Just a moment... I want to finish this thought before moving on.', + timeline: '', + title: '', + }); + expect(r1.junk_pattern_matches).not.toContain('cloudflare_just_a_moment'); + + // With the cdn-cgi discriminator nearby → fires. + const r2 = assessContentSanity({ + compiled_truth: 'Just a moment... please wait while we verify\ncdn-cgi/challenge-platform/h/blah', + timeline: '', + title: '', + }); + expect(r2.junk_pattern_matches).toContain('cloudflare_just_a_moment'); + }); + + test('cloudflare_ray_id fires on trailing diagnostic', () => { + const r = assessContentSanity({ + compiled_truth: 'You have been blocked.\n\nCloudflare Ray ID: abc12345', + timeline: '', + title: 'Blocked', + }); + expect(r.junk_pattern_matches).toContain('cloudflare_ray_id'); + }); + + test('access_denied fires on bare 403 dumps', () => { + const r = assessContentSanity({ + compiled_truth: 'Access denied\n\nYou do not have permission to view this resource.', + timeline: '', + title: '', + }); + expect(r.junk_pattern_matches).toContain('access_denied'); + }); + + test('captcha_required catches multiple verification phrasings', () => { + for (const phrase of ['verify you are human', 'verify you are a human', 'captcha required', 'please complete the security check']) { + const r = assessContentSanity({ + compiled_truth: `Please ${phrase} to continue.`, + timeline: '', + title: '', + }); + expect(r.junk_pattern_matches).toContain('captcha_required'); + } + }); + + test('error_page_title fires only on bare titles (anchored)', () => { + for (const title of ['404', 'Error 500', 'Page Not Found', '503']) { + const r = assessContentSanity({ compiled_truth: '', timeline: '', title }); + expect(r.junk_pattern_matches).toContain('error_page_title'); + } + // A thoughtful page ABOUT errors does NOT fire. + const r2 = assessContentSanity({ + compiled_truth: '', + timeline: '', + title: 'Designing for 404 pages: a UX guide', + }); + expect(r2.junk_pattern_matches).not.toContain('error_page_title'); + }); + + test('multiple patterns can fire on the same content', () => { + const r = assessContentSanity({ + compiled_truth: 'Cloudflare Ray ID: xyz789', + timeline: '', + title: 'Attention Required! | Cloudflare', + }); + expect(r.junk_pattern_matches).toContain('cloudflare_attention_required'); + expect(r.junk_pattern_matches).toContain('cloudflare_ray_id'); + expect(r.shouldHardBlock).toBe(true); + }); + + test('case-insensitive matching across all patterns', () => { + const r = assessContentSanity({ + compiled_truth: '', + timeline: '', + title: 'ATTENTION REQUIRED! | CLOUDFLARE', + }); + expect(r.junk_pattern_matches).toContain('cloudflare_attention_required'); + }); +}); + +// ─── REASON ORDERING + MESSAGES ──────────────────────────────── + +describe('assessContentSanity — reason ordering', () => { + test('reason_messages embed the classifier-readable PAGE_JUNK_PATTERN prefix', () => { + const r = assessContentSanity({ + compiled_truth: '', + timeline: '', + title: 'Access denied', + }); + expect(r.shouldHardBlock).toBe(true); + const joined = r.reason_messages.join(' '); + expect(joined).toContain(PAGE_JUNK_PATTERN_CODE); + expect(PAGE_JUNK_PATTERN_CODE).toBe('PAGE_JUNK_PATTERN'); + }); + + test('block-level oversize message includes PAGE_OVERSIZED prefix', () => { + const r = assessContentSanity({ + compiled_truth: 'a'.repeat(600_000), + timeline: '', + title: '', + }); + const joined = r.reason_messages.join(' '); + expect(joined).toContain('PAGE_OVERSIZED:'); + }); + + test('hard-block + oversize: BOTH reasons present (operator sees both causes)', () => { + // Pattern in first 2KB head-slice so junk_pattern fires alongside + // oversize_block. This is the realistic 890K Cloudflare dump shape: + // the "Attention Required" banner is at the top, then the rest of + // the page is HTML/styles/etc making it huge. + const r = assessContentSanity({ + compiled_truth: 'Cloudflare Ray ID: abc\n' + 'a'.repeat(600_000), + timeline: '', + title: '', + }); + expect(r.reasons).toContain('oversize_block'); + expect(r.reasons).toContain('junk_pattern'); + expect(r.shouldHardBlock).toBe(true); + // hard-block wins; soft-block doesn't ALSO fire. + expect(r.shouldSkipEmbed).toBe(false); + }); +}); + +// ─── OPERATOR LITERALS ──────────────────────────────────────── + +describe('assessContentSanity — operator literals', () => { + test('empty extra_literals = built-ins only', () => { + const r = assessContentSanity({ + compiled_truth: "You're being blocked from accessing this resource", + timeline: '', + title: '', + extra_literals: [], + }); + expect(r.shouldHardBlock).toBe(false); + expect(r.literal_substring_matches).toEqual([]); + }); + + test('operator literal matches case-insensitively', () => { + const literals: OperatorLiteral[] = [ + { name: 'reddit_blocked', substring: "you're being blocked from accessing" }, + ]; + const r = assessContentSanity({ + compiled_truth: "YOU'RE BEING BLOCKED FROM ACCESSING this site.", + timeline: '', + title: '', + extra_literals: literals, + }); + expect(r.literal_substring_matches).toContain('reddit_blocked'); + expect(r.shouldHardBlock).toBe(true); + }); + + test('regex meta-characters in operator literal stay literal (no ReDoS surface)', () => { + const literals: OperatorLiteral[] = [ + { name: 'meta_test', substring: '(a+)+b' }, // would be catastrophic as regex + ]; + // Should NOT match prose + const r1 = assessContentSanity({ + compiled_truth: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', + timeline: '', + title: '', + extra_literals: literals, + }); + expect(r1.literal_substring_matches).not.toContain('meta_test'); + // SHOULD match the literal string + const r2 = assessContentSanity({ + compiled_truth: 'The pattern (a+)+b is bad regex.', + timeline: '', + title: '', + extra_literals: literals, + }); + expect(r2.literal_substring_matches).toContain('meta_test'); + }); + + test('literal applies_to scope honored', () => { + const titleOnly: OperatorLiteral = { name: 't', substring: 'wall', applies_to: 'title' }; + const bodyOnly: OperatorLiteral = { name: 'b', substring: 'wall', applies_to: 'body' }; + const r1 = assessContentSanity({ + compiled_truth: 'auth wall content', + timeline: '', + title: 'unrelated', + extra_literals: [titleOnly], + }); + expect(r1.literal_substring_matches).not.toContain('t'); + const r2 = assessContentSanity({ + compiled_truth: 'unrelated body', + timeline: '', + title: 'auth wall', + extra_literals: [titleOnly], + }); + expect(r2.literal_substring_matches).toContain('t'); + const r3 = assessContentSanity({ + compiled_truth: 'auth wall content', + timeline: '', + title: 'unrelated', + extra_literals: [bodyOnly], + }); + expect(r3.literal_substring_matches).toContain('b'); + }); + + test('empty substring is no-op', () => { + const r = assessContentSanity({ + compiled_truth: 'anything', + timeline: '', + title: '', + extra_literals: [{ name: 'empty', substring: '' }], + }); + expect(r.literal_substring_matches).toEqual([]); + }); +}); + +// ─── SCAN HEAD-SLICE BOUNDARY ───────────────────────────────── + +describe('assessContentSanity — head-slice scope', () => { + test('pattern in first 2KB matches', () => { + const r = assessContentSanity({ + compiled_truth: 'Cloudflare Ray ID: aaa\n' + 'x'.repeat(10_000), + timeline: '', + title: '', + }); + expect(r.junk_pattern_matches).toContain('cloudflare_ray_id'); + }); + + test('pattern past the 2KB head-slice does NOT match (cost bound)', () => { + // Cost bound: patterns evaluated against first ~2KB only. + // Pattern buried at offset 5K should NOT trip. + const r = assessContentSanity({ + compiled_truth: 'x'.repeat(5000) + 'Cloudflare Ray ID: deep', + timeline: '', + title: '', + }); + expect(r.junk_pattern_matches).not.toContain('cloudflare_ray_id'); + }); +}); + +// ─── ContentSanityBlockError ────────────────────────────────── + +describe('ContentSanityBlockError', () => { + test('error message contains PAGE_JUNK_PATTERN for classifier match', () => { + const r = assessContentSanity({ + compiled_truth: 'Access denied', + timeline: '', + title: '', + }); + const err = new ContentSanityBlockError(r); + expect(err.message).toContain('PAGE_JUNK_PATTERN'); + expect(err.code).toBe('PAGE_JUNK_PATTERN'); + expect(err.name).toBe('ContentSanityBlockError'); + }); + + test('error retains the full result for caller inspection', () => { + const r = assessContentSanity({ + compiled_truth: 'Access denied', + timeline: '', + title: 'Attention Required! | Cloudflare', + }); + const err = new ContentSanityBlockError(r); + expect(err.result.junk_pattern_matches.length).toBeGreaterThan(0); + expect(err.result).toBe(r); // same reference, not a copy + }); + + test('error is throwable + catchable as instanceof', () => { + const r = assessContentSanity({ + compiled_truth: '', + timeline: '', + title: 'Access denied', + }); + try { + throw new ContentSanityBlockError(r); + } catch (e) { + expect(e).toBeInstanceOf(ContentSanityBlockError); + expect((e as Error).message).toContain('PAGE_JUNK_PATTERN'); + } + }); +}); diff --git a/test/db-lock-election.test.ts b/test/db-lock-election.test.ts new file mode 100644 index 000000000..5eeec3298 --- /dev/null +++ b/test/db-lock-election.test.ts @@ -0,0 +1,86 @@ +/** + * v0.41 Eng D9 — tryWithDbElection convenience tests. + * + * Verifies the per-tick election shape against PGLite: + * - First call wins → fn runs → returns its value + * - Concurrent second call gets null (someone else holds the lock) + * - After first releases, next caller wins + * - fn throws → lock released cleanly, error propagates + * - Different lock IDs are independent + */ + +import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test'; +import { PGLiteEngine } from '../src/core/pglite-engine.ts'; +import { tryWithDbElection, tryAcquireDbLock } from '../src/core/db-lock.ts'; + +let engine: PGLiteEngine; + +beforeAll(async () => { + engine = new PGLiteEngine(); + await engine.connect({ database_url: '' }); + await engine.initSchema(); +}); + +afterAll(async () => { + await engine.disconnect(); +}); + +beforeEach(async () => { + await engine.executeRaw(`DELETE FROM gbrain_cycle_locks WHERE id LIKE 'test-election-%'`); +}); + +describe('tryWithDbElection', () => { + test('first caller wins → fn runs → returns its value', async () => { + const result = await tryWithDbElection(engine, 'test-election-1', 1, async () => 'won'); + expect(result).toBe('won'); + }); + + test('lock auto-released after fn returns', async () => { + // First call acquires + releases. + await tryWithDbElection(engine, 'test-election-2', 1, async () => 'first'); + // Second call should win because the first released. + const r = await tryWithDbElection(engine, 'test-election-2', 1, async () => 'second'); + expect(r).toBe('second'); + }); + + test('concurrent acquire by a different holder returns null (not my tick)', async () => { + // Manually acquire the lock so tryWithDbElection finds it held. + const handle = await tryAcquireDbLock(engine, 'test-election-3', 1); + expect(handle).not.toBeNull(); + try { + let ran = false; + const r = await tryWithDbElection(engine, 'test-election-3', 1, async () => { + ran = true; + return 'should not run'; + }); + expect(r).toBeNull(); + expect(ran).toBe(false); + } finally { + await handle!.release(); + } + }); + + test('fn throw releases the lock cleanly + rethrows', async () => { + await expect( + tryWithDbElection(engine, 'test-election-4', 1, async () => { + throw new Error('boom'); + }), + ).rejects.toThrow('boom'); + + // Lock was released — next caller can acquire. + const r = await tryWithDbElection(engine, 'test-election-4', 1, async () => 'ok'); + expect(r).toBe('ok'); + }); + + test('different lock IDs are independent', async () => { + const a = await tryAcquireDbLock(engine, 'test-election-A', 1); + expect(a).not.toBeNull(); + try { + // Different id should not conflict. + const r = await tryWithDbElection(engine, 'test-election-B', 1, async () => 'B-won'); + expect(r).toBe('B-won'); + } finally { + await a!.release(); + } + }); +}); diff --git a/test/doctor-subagent-health.test.ts b/test/doctor-subagent-health.test.ts new file mode 100644 index 000000000..247837760 --- /dev/null +++ b/test/doctor-subagent-health.test.ts @@ -0,0 +1,125 @@ +/** + * v0.41 Bug 2 / Eng D8 — `gbrain doctor` `subagent_health` check coverage. + * + * Reads the last 24h of `minion_lease_pressure_log` (populated by the + * Bug 2 worker bypass path) and classifies pressure into ok / warn / fail + * thresholds. The doctor check is the operator's primary forensic signal + * for "is the lease cap too tight" — without it, the v0.41 bypass would + * be invisible (no dead-letter, but also no operator visibility). + * + * Four cases pinned per the plan: + * 1. 0 bounces → ok ("no pressure") + * 2. 100+ bounces with subagent jobs completing → ok ("healthy backpressure") + * 3. 100+ bounces with NO subagent jobs completing → warn (paste-ready hint) + * 4. 1000+ bounces → fail (blocking) + */ + +import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test'; +import { PGLiteEngine } from '../src/core/pglite-engine.ts'; +import { MinionQueue } from '../src/core/minions/queue.ts'; +import { checkSubagentHealth } from '../src/commands/doctor.ts'; +import { logLeasePressure } from '../src/core/minions/lease-pressure-audit.ts'; + +let engine: PGLiteEngine; +let queue: MinionQueue; + +beforeAll(async () => { + engine = new PGLiteEngine(); + await engine.connect({ database_url: '' }); + await engine.initSchema(); + queue = new MinionQueue(engine); +}); + +afterAll(async () => { + await engine.disconnect(); +}); + +beforeEach(async () => { + await engine.executeRaw('DELETE FROM minion_lease_pressure_log'); + await engine.executeRaw('DELETE FROM minion_jobs'); +}); + +/** Wrapper for terse test bodies. */ +async function getSubagentHealthCheck() { + return checkSubagentHealth(engine); +} + +describe('doctor subagent_health (v0.41 Bug 2 / Eng D8)', () => { + test('0 bounces → ok ("no pressure")', async () => { + const check = await getSubagentHealthCheck(); + expect(check.status).toBe('ok'); + expect(check.message).toContain('No rate-lease pressure'); + }); + + test('healthy backpressure (1-99 bounces with completed jobs) → ok', async () => { + // 5 bounces + 3 completed subagent jobs in the last hour. + const owner = await queue.add('subagent', {}, {}, { allowProtectedSubmit: true }); + for (let i = 0; i < 5; i++) { + await logLeasePressure(engine, { + job_id: owner.id, + lease_key: 'anthropic:messages', + active_at_bounce: 8, + max_concurrent: 8, + }); + } + // Insert 3 completed subagent jobs. + for (let i = 0; i < 3; i++) { + await engine.executeRaw( + `INSERT INTO minion_jobs + (name, queue, status, attempts_made, attempts_started, + finished_at, started_at, max_attempts) + VALUES ('subagent', 'default', 'completed', 1, 1, + now(), now() - interval '1 second', 3)`, + ); + } + + const check = await getSubagentHealthCheck(); + expect(check.status).toBe('ok'); + expect(check.message).toContain('bounces'); + // Sub-100 bounces routes through the second-tier OK message + // (not the "no pressure" message; falls into "healthy backpressure"). + expect(check.message).not.toContain('No rate-lease pressure'); + }); + + test('100+ bounces with NO completed subagent jobs → warn with paste-ready hint', async () => { + const owner = await queue.add('subagent', {}, {}, { allowProtectedSubmit: true }); + for (let i = 0; i < 100; i++) { + await logLeasePressure(engine, { + job_id: owner.id, + lease_key: 'anthropic:messages', + active_at_bounce: 8, + max_concurrent: 8, + }); + } + // NO completed subagent jobs — pressure is BLOCKING real work. + + const check = await getSubagentHealthCheck(); + expect(check.status).toBe('warn'); + expect(check.message).toContain('100'); + // Paste-ready hint with the canonical env-var name. + expect(check.message).toContain('GBRAIN_ANTHROPIC_MAX_INFLIGHT'); + }); + + test('1000+ bounces → fail (blocking real work)', async () => { + const owner = await queue.add('subagent', {}, {}, { allowProtectedSubmit: true }); + // Batch insert via VALUES rather than per-row logLeasePressure (faster). + const valuesList: string[] = []; + const params: Array = []; + for (let i = 0; i < 1000; i++) { + valuesList.push(`($${params.length + 1}, $${params.length + 2}, $${params.length + 3}, $${params.length + 4})`); + params.push(owner.id, 'anthropic:messages', 8, 8); + } + await engine.executeRaw( + `INSERT INTO minion_lease_pressure_log + (job_id, lease_key, active_at_bounce, max_concurrent) + VALUES ${valuesList.join(', ')}`, + params, + ); + + const check = await getSubagentHealthCheck(); + expect(check.status).toBe('fail'); + expect(check.message).toContain('1000'); + expect(check.message).toContain('blocking real work'); + expect(check.message).toContain('GBRAIN_ANTHROPIC_MAX_INFLIGHT'); + }); +}); diff --git a/test/e2e/autopilot-fanout-postgres.test.ts b/test/e2e/autopilot-fanout-postgres.test.ts index 09c83bbe7..3cc2f2779 100644 --- a/test/e2e/autopilot-fanout-postgres.test.ts +++ b/test/e2e/autopilot-fanout-postgres.test.ts @@ -144,9 +144,12 @@ describeIfDB('autopilot fan-out — Postgres E2E', () => { await seedSource('full-round-trip'); await engine.executeRaw(`UPDATE sources SET local_path = NULL WHERE id = 'default'`); - // Relative timestamp inside the 60-min freshness window. A prior version - // pinned this to '2026-05-22T15:00:00.000Z' which started failing once - // wall-clock drifted past 60 minutes from that point. + // Use a recent (within-freshness-window) timestamp so the source + // classifies as fresh. Hardcoded dates rot — when this test was + // written, '2026-05-22T15:00:00.000Z' was 30 minutes ago and within + // the window. Two days later it's past the window and the source + // dispatches instead of being skipped, breaking the assertion on + // line below. Relative timestamp keeps the test valid forever. const ts = new Date(Date.now() - 30 * 60 * 1000).toISOString(); const updated = await engine.updateSourceConfig('full-round-trip', { last_full_cycle_at: ts, diff --git a/test/e2e/code-indexing.test.ts b/test/e2e/code-indexing.test.ts index 61766eee8..e8548655e 100644 --- a/test/e2e/code-indexing.test.ts +++ b/test/e2e/code-indexing.test.ts @@ -223,7 +223,7 @@ beforeAll(async () => { await engine.initSchema(); // Seed 5 files per language, 25 total (scaled down from the plan's - // ~50 files to keep test runtime under 5 seconds). The retrieval + // ~50 files to keep test runtime predictable). The retrieval // signal is the same shape at 25 as at 50. const names = ['Auth', 'Cache', 'Queue', 'Router', 'Store']; for (const n of names) { @@ -233,7 +233,9 @@ beforeAll(async () => { await importCodeFile(engine, `rust/${n.toLowerCase()}.rs`, generateRustFile(n), { noEmbed: true }); await importCodeFile(engine, `java/${n}.java`, generateJavaFile(n), { noEmbed: true }); } -}); + // v0.41 D2 wave: 92-migration replay + SQL grammar load can push the + // default 5s beforeAll budget on slower CI runners; bump explicitly. +}, 30000); afterAll(async () => { await engine.disconnect(); @@ -335,3 +337,200 @@ describe('BrainBench code — edge cases', () => { expect(secondResult.length).toBe(count1); }); }); + +// ──────────────────────────────────────────────────────────── +// v0.41 D2 wave (#1173) — SQL indexing E2E. +// Load-bearing canary for the "D2 = code-brain peer" thesis: tree-sitter +// chunks SQL into per-statement chunks, DDL kinds carry symbol_name + +// symbol_type populated from CREATE TABLE/FUNCTION/INDEX targets, and +// findCodeDef returns those chunks when queried by name. Without this +// path working, SQL chunks would be "just searchable text", not code +// intelligence (codex F2 in /plan-eng-review). +// ──────────────────────────────────────────────────────────── +describe('SQL code indexing — DDL chunks + code-def works', () => { + // Statement bodies must be long enough to defeat the small-sibling + // merger; ~120+ tokens per statement keeps each chunk independent. + const SQL_FIXTURE = ` +CREATE TABLE users_account_table_long_enough_to_avoid_merger ( + id SERIAL PRIMARY KEY, + email TEXT NOT NULL UNIQUE, + password_hash TEXT NOT NULL, + display_name TEXT, + phone_number TEXT, + created_at TIMESTAMP NOT NULL DEFAULT NOW(), + updated_at TIMESTAMP, + deleted_at TIMESTAMP, + email_verified_at TIMESTAMP, + last_login_at TIMESTAMP, + metadata JSONB DEFAULT '{}'::jsonb, + preferences JSONB DEFAULT '{}'::jsonb +); + +CREATE OR REPLACE FUNCTION get_user_by_email_lookup_full_function_name(p_email TEXT) +RETURNS users_account_table_long_enough_to_avoid_merger AS $$ +DECLARE + result users_account_table_long_enough_to_avoid_merger; +BEGIN + SELECT * INTO result + FROM users_account_table_long_enough_to_avoid_merger + WHERE email = p_email + AND deleted_at IS NULL + LIMIT 1; + RETURN result; +END; +$$ LANGUAGE plpgsql; + +CREATE INDEX idx_users_account_email_for_login_lookup_with_long_name + ON users_account_table_long_enough_to_avoid_merger (email, created_at, deleted_at); + +CREATE VIEW active_users_dashboard_summary_view_long_enough_to_split AS + SELECT u.id, u.email, u.display_name, u.last_login_at, u.created_at + FROM users_account_table_long_enough_to_avoid_merger u + WHERE u.deleted_at IS NULL + AND u.email_verified_at IS NOT NULL + ORDER BY u.last_login_at DESC NULLS LAST; +`; + + test('SQL import produces page with type=code + page_kind=code', async () => { + await importCodeFile(engine, 'migrations/001_users.sql', SQL_FIXTURE, { noEmbed: true }); + const rows = await engine.executeRaw<{ type: string; page_kind: string }>( + `SELECT type, page_kind FROM pages WHERE slug = $1`, + ['migrations-001_users-sql'], + ); + expect(rows.length).toBe(1); + expect(rows[0]!.type).toBe('code'); + expect(rows[0]!.page_kind).toBe('code'); + }); + + test('CREATE TABLE chunk carries symbol_name=table name + symbol_type=table', async () => { + const rows = await engine.executeRaw<{ symbol_name: string; symbol_type: string; language: string }>( + `SELECT symbol_name, symbol_type, language FROM content_chunks + WHERE page_id = (SELECT id FROM pages WHERE slug = $1) + AND symbol_name = $2`, + ['migrations-001_users-sql', 'users_account_table_long_enough_to_avoid_merger'], + ); + expect(rows.length).toBeGreaterThanOrEqual(1); + expect(rows[0]!.symbol_type).toBe('table'); + expect(rows[0]!.language).toBe('sql'); + }); + + test('CREATE FUNCTION chunk carries symbol_name=function name + symbol_type=function', async () => { + const rows = await engine.executeRaw<{ symbol_name: string; symbol_type: string }>( + `SELECT symbol_name, symbol_type FROM content_chunks + WHERE page_id = (SELECT id FROM pages WHERE slug = $1) + AND symbol_name = $2`, + ['migrations-001_users-sql', 'get_user_by_email_lookup_full_function_name'], + ); + expect(rows.length).toBeGreaterThanOrEqual(1); + expect(rows[0]!.symbol_type).toBe('function'); + }); + + test('findCodeDef returns CREATE TABLE site (load-bearing D2 canary)', async () => { + const results = await findCodeDef(engine, 'users_account_table_long_enough_to_avoid_merger', { language: 'sql' }); + expect(results.length).toBeGreaterThanOrEqual(1); + expect(results[0]!.slug).toBe('migrations-001_users-sql'); + expect(results[0]!.symbol_type).toBe('table'); + }); + + test('findCodeDef returns CREATE FUNCTION site by name', async () => { + const results = await findCodeDef(engine, 'get_user_by_email_lookup_full_function_name', { language: 'sql' }); + expect(results.length).toBeGreaterThanOrEqual(1); + expect(results[0]!.symbol_type).toBe('function'); + }); + + test('findCodeDef returns CREATE INDEX site by name', async () => { + const results = await findCodeDef(engine, 'idx_users_account_email_for_login_lookup_with_long_name', { language: 'sql' }); + expect(results.length).toBeGreaterThanOrEqual(1); + expect(results[0]!.symbol_type).toBe('index'); + }); + + test('findCodeDef returns CREATE VIEW site by name', async () => { + const results = await findCodeDef(engine, 'active_users_dashboard_summary_view_long_enough_to_split', { language: 'sql' }); + expect(results.length).toBeGreaterThanOrEqual(1); + expect(results[0]!.symbol_type).toBe('view'); + }); + + test('findCodeRefs returns SQL chunks by substring match (DML + DDL)', async () => { + // code-refs uses chunk_text ILIKE, no DEF_TYPES gate, so it returns + // every occurrence (DML + DDL). Distinct from code-def which only + // returns definition sites. + const refs = await findCodeRefs(engine, 'users_account_table_long_enough_to_avoid_merger', { language: 'sql' }); + expect(refs.length).toBeGreaterThanOrEqual(1); + // At least one ref should land on the CREATE TABLE chunk. + const tableRef = refs.find(r => r.symbol_type === 'table'); + expect(tableRef).toBeDefined(); + }); + + test('CREATE TRIGGER + CREATE TYPE chunks land with correct symbol_type', async () => { + const sql = ` +CREATE TYPE long_enough_user_role_enum_so_not_merged AS ENUM ('admin', 'member', 'guest', 'service_account', 'auditor'); + +CREATE TRIGGER users_long_audit_trigger_for_role_changes + AFTER UPDATE ON users + FOR EACH ROW + WHEN (OLD.email IS DISTINCT FROM NEW.email) + EXECUTE FUNCTION log_email_change_long_function_name(); +`; + await importCodeFile(engine, 'migrations/002_audit.sql', sql, { noEmbed: true }); + const typeRows = await engine.executeRaw<{ symbol_type: string }>( + `SELECT symbol_type FROM content_chunks + WHERE page_id = (SELECT id FROM pages WHERE slug = $1) + AND symbol_name = $2`, + ['migrations-002_audit-sql', 'long_enough_user_role_enum_so_not_merged'], + ); + expect(typeRows.length).toBeGreaterThanOrEqual(1); + expect(typeRows[0]!.symbol_type).toBe('type'); + const triggerRows = await engine.executeRaw<{ symbol_type: string }>( + `SELECT symbol_type FROM content_chunks + WHERE page_id = (SELECT id FROM pages WHERE slug = $1) + AND symbol_name = $2`, + ['migrations-002_audit-sql', 'users_long_audit_trigger_for_role_changes'], + ); + expect(triggerRows.length).toBeGreaterThanOrEqual(1); + expect(triggerRows[0]!.symbol_type).toBe('trigger'); + }); + + test('findCodeDef on CREATE TYPE returns it (DEF_TYPES allowlist regression)', async () => { + const results = await findCodeDef(engine, 'long_enough_user_role_enum_so_not_merged', { language: 'sql' }); + expect(results.length).toBeGreaterThanOrEqual(1); + expect(results[0]!.symbol_type).toBe('type'); + }); + + test('findCodeDef on CREATE TRIGGER returns it (DEF_TYPES allowlist regression)', async () => { + const results = await findCodeDef(engine, 'users_long_audit_trigger_for_role_changes', { language: 'sql' }); + expect(results.length).toBeGreaterThanOrEqual(1); + expect(results[0]!.symbol_type).toBe('trigger'); + }); + + test('DML-only file still produces a code page (just no symbol-named chunks)', async () => { + const dmlOnly = ` +SELECT u.id, u.email FROM users u WHERE u.deleted_at IS NULL ORDER BY u.created_at; +INSERT INTO audit_log (event_type, user_id, payload) VALUES ('login', 42, '{"ip":"1.2.3.4"}'::jsonb); +UPDATE users SET last_login_at = NOW() WHERE id = 42 AND deleted_at IS NULL; +`; + await importCodeFile(engine, 'queries/lib.sql', dmlOnly, { noEmbed: true }); + const pageRow = await engine.executeRaw<{ type: string; page_kind: string }>( + `SELECT type, page_kind FROM pages WHERE slug = 'queries-lib-sql'`, + ); + expect(pageRow.length).toBe(1); + expect(pageRow[0]!.type).toBe('code'); + const namedChunks = await engine.executeRaw<{ symbol_name: string }>( + `SELECT symbol_name FROM content_chunks + WHERE page_id = (SELECT id FROM pages WHERE slug = 'queries-lib-sql') + AND symbol_name IS NOT NULL`, + ); + // Zero named chunks because all statements are DML. + expect(namedChunks.length).toBe(0); + }); + + test('Re-importing same SQL file is idempotent (content_hash short-circuit)', async () => { + const sql = 'CREATE TABLE idempotent_test_table_long_name_for_no_merge (id INT, name TEXT, value TEXT, created TIMESTAMP);'; + await importCodeFile(engine, 'migrations/003_idem.sql', sql, { noEmbed: true }); + const before = await findCodeDef(engine, 'idempotent_test_table_long_name_for_no_merge', { language: 'sql' }); + const count1 = before.length; + // Re-import: content_hash unchanged → should not duplicate chunks. + await importCodeFile(engine, 'migrations/003_idem.sql', sql, { noEmbed: true }); + const after = await findCodeDef(engine, 'idempotent_test_table_long_name_for_no_merge', { language: 'sql' }); + expect(after.length).toBe(count1); + }); +}); diff --git a/test/e2e/dream-cycle-phase-order-pglite.test.ts b/test/e2e/dream-cycle-phase-order-pglite.test.ts index a251d9cd3..151a79ca1 100644 --- a/test/e2e/dream-cycle-phase-order-pglite.test.ts +++ b/test/e2e/dream-cycle-phase-order-pglite.test.ts @@ -122,7 +122,7 @@ const EXPECTED_PHASES: CyclePhase[] = [ 'calibration_profile', // v0.36.1.0 'embed', 'orphans', - 'schema-suggest', // v0.40.7.0 — Schema Cathedral v3 + 'schema-suggest', // v0.39.0.0 — passive schema-suggest after orphans 'purge', // v0.26.5 ]; diff --git a/test/e2e/fresh-install-pglite.test.ts b/test/e2e/fresh-install-pglite.test.ts index 88b4fe438..1b3f3f3bc 100644 --- a/test/e2e/fresh-install-pglite.test.ts +++ b/test/e2e/fresh-install-pglite.test.ts @@ -27,11 +27,6 @@ describe('E2E: fresh gbrain init --pglite → import → embed works end-to-end' let tmpHome: string; let origHome: string | undefined; let origZeKey: string | undefined; - // init.ts:455 fails loud when MULTIPLE embedding providers are env-ready - // (non-TTY path). Developer machines commonly have OPENAI_API_KEY + - // VOYAGE_API_KEY + ZEROENTROPY_API_KEY all set; this test wants exactly - // ZE to be the env-ready provider. Save+unset the others in beforeEach, - // restore in afterEach. let origOpenaiKey: string | undefined; let origVoyageKey: string | undefined; @@ -39,13 +34,18 @@ describe('E2E: fresh gbrain init --pglite → import → embed works end-to-end' tmpHome = mkdtempSync(join(tmpdir(), 'gbrain-e2e-fresh-')); origHome = process.env.GBRAIN_HOME; origZeKey = process.env.ZEROENTROPY_API_KEY; + // Save + clear OPENAI_API_KEY + VOYAGE_API_KEY so init only sees + // one provider as env-ready (ZE). Without this, dev machines with + // multi-provider env (Garry's setup) fail init's disambiguation gate + // ("Multiple embedding providers env-ready: openai, voyage, + // zeroentropyai") before the test body runs. origOpenaiKey = process.env.OPENAI_API_KEY; origVoyageKey = process.env.VOYAGE_API_KEY; + delete process.env.OPENAI_API_KEY; + delete process.env.VOYAGE_API_KEY; process.env.GBRAIN_HOME = tmpHome; // Stub key so init's setup-hint check passes. process.env.ZEROENTROPY_API_KEY = 'sk-test-ze'; - delete process.env.OPENAI_API_KEY; - delete process.env.VOYAGE_API_KEY; }); afterEach(() => { @@ -54,10 +54,8 @@ describe('E2E: fresh gbrain init --pglite → import → embed works end-to-end' else process.env.GBRAIN_HOME = origHome; if (origZeKey === undefined) delete process.env.ZEROENTROPY_API_KEY; else process.env.ZEROENTROPY_API_KEY = origZeKey; - if (origOpenaiKey === undefined) delete process.env.OPENAI_API_KEY; - else process.env.OPENAI_API_KEY = origOpenaiKey; - if (origVoyageKey === undefined) delete process.env.VOYAGE_API_KEY; - else process.env.VOYAGE_API_KEY = origVoyageKey; + if (origOpenaiKey !== undefined) process.env.OPENAI_API_KEY = origOpenaiKey; + if (origVoyageKey !== undefined) process.env.VOYAGE_API_KEY = origVoyageKey; __setEmbedTransportForTests(null); // Restore legacy-preload gateway state. configureGateway({ diff --git a/test/e2e/ingestion-roundtrip.test.ts b/test/e2e/ingestion-roundtrip.test.ts index 705c7c59f..23fedbcad 100644 --- a/test/e2e/ingestion-roundtrip.test.ts +++ b/test/e2e/ingestion-roundtrip.test.ts @@ -66,6 +66,12 @@ afterAll(async () => { beforeEach(async () => { await resetPgliteState(engine); + // 200ms grace period for the previous test's chokidar watchers to fully + // release OS-level FSEvents handles on macOS. Without this, the second + // test's watcher events queue behind the first test's pending cleanup + // and the waitFor(15s) for the first file drop times out. See + // ingestion-roundtrip cross-test contamination notes. + await new Promise((r) => setTimeout(r, 200)); tmpRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'gbrain-e2e-roundtrip-')); inboxDir = path.join(tmpRoot, 'inbox'); brainDir = path.join(tmpRoot, 'brain'); @@ -117,6 +123,12 @@ describe('ingestion roundtrip — inbox-folder → daemon → ingest_capture → }, }); + // Create the inbox dir BEFORE starting the watcher to eliminate a race + // where chokidar hasn't attached yet when the first write fires (the + // 6s→15s waitFor flake on the source.) Without this, the test relies on + // chokidar's polling fallback to notice the dir, which is timing-dependent. + fs.mkdirSync(inboxDir, { recursive: true }); + const source = createInboxFolderSource({ inboxDir, debounceMs: 50, @@ -126,12 +138,11 @@ describe('ingestion roundtrip — inbox-folder → daemon → ingest_capture → await daemon.start(); // Drop a file into the inbox. - fs.mkdirSync(inboxDir, { recursive: true }); const captured = path.join(inboxDir, 'roundtrip.md'); fs.writeFileSync(captured, '---\ntitle: Roundtrip\n---\n\nfull e2e flow'); // Wait for the daemon to pick it up + dispatch + handler to write. - await waitFor(() => dispatchedEvents.length === 1, 6000); + await waitFor(() => dispatchedEvents.length === 1, 15000); // Page is in the DB. const page = await engine.getPage(dispatchedEvents[0]!.metadata!.slug as string ?? @@ -168,6 +179,9 @@ describe('ingestion roundtrip — inbox-folder → daemon → ingest_capture → }, }); + // mkdirSync BEFORE daemon.start to eliminate chokidar attach race. + fs.mkdirSync(inboxDir, { recursive: true }); + const source = createInboxFolderSource({ inboxDir, debounceMs: 50, @@ -176,12 +190,10 @@ describe('ingestion roundtrip — inbox-folder → daemon → ingest_capture → daemon.register({ source }); await daemon.start(); - fs.mkdirSync(inboxDir, { recursive: true }); - // Drop file 1 const drop1 = path.join(inboxDir, 'dup-1.md'); fs.writeFileSync(drop1, '# duplicate content\n\nidentical body'); - await waitFor(() => dispatchedEvents.length === 1, 6000); + await waitFor(() => dispatchedEvents.length === 1, 15000); // Drop file 2 with byte-identical content (different filename). const drop2 = path.join(inboxDir, 'dup-2.md'); @@ -217,9 +229,13 @@ describe('ingestion roundtrip — multi-source coordination', () => { }, }); - // Two distinct inbox dirs, two sources. + // Two distinct inbox dirs, two sources. Create the dirs BEFORE + // daemon.start to eliminate the chokidar attach race (same fix as + // the single-source tests above). const inboxA = path.join(tmpRoot, 'inbox-a'); const inboxB = path.join(tmpRoot, 'inbox-b'); + fs.mkdirSync(inboxA, { recursive: true }); + fs.mkdirSync(inboxB, { recursive: true }); const sourceA = createInboxFolderSource({ id: 'inbox-a', inboxDir: inboxA, @@ -239,7 +255,7 @@ describe('ingestion roundtrip — multi-source coordination', () => { fs.writeFileSync(path.join(inboxA, 'from-a.md'), 'content from A'); fs.writeFileSync(path.join(inboxB, 'from-b.md'), 'content from B'); - await waitFor(() => dispatchedEvents.length === 2, 6000); + await waitFor(() => dispatchedEvents.length === 2, 15000); const fromA = dispatchedEvents.find((e) => e.source_id === 'inbox-a'); const fromB = dispatchedEvents.find((e) => e.source_id === 'inbox-b'); diff --git a/test/e2e/jobs-watch-readsnapshot.test.ts b/test/e2e/jobs-watch-readsnapshot.test.ts new file mode 100644 index 000000000..47b04a679 --- /dev/null +++ b/test/e2e/jobs-watch-readsnapshot.test.ts @@ -0,0 +1,86 @@ +/** + * v0.41 E2E — jobs watch readSnapshot integration. + * + * Pairs with the pure-function renderer tests in + * test/jobs-watch-snapshot.test.ts. This file verifies readSnapshot + * correctly aggregates from the engine: stats, lease pressure, top + * errors clustered, budget owners with cents. + */ + +import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test'; +import { PGLiteEngine } from '../../src/core/pglite-engine.ts'; +import { MinionQueue } from '../../src/core/minions/queue.ts'; +import { readSnapshot } from '../../src/commands/jobs-watch.ts'; +import { logLeasePressure } from '../../src/core/minions/lease-pressure-audit.ts'; +import { setOwnerBudget } from '../../src/core/minions/budget-tracker.ts'; + +let engine: PGLiteEngine; +let queue: MinionQueue; + +beforeAll(async () => { + engine = new PGLiteEngine(); + await engine.connect({}); + await engine.initSchema(); + queue = new MinionQueue(engine); +}, 60_000); + +afterAll(async () => { + await engine.disconnect(); +}); + +beforeEach(async () => { + await engine.executeRaw('DELETE FROM minion_lease_pressure_log'); + await engine.executeRaw('DELETE FROM minion_jobs'); +}, 30_000); + +describe('v0.41 jobs-watch readSnapshot E2E', () => { + test('empty brain → zero snapshot', async () => { + const s = await readSnapshot(engine); + expect(s.queue_health).toEqual({ waiting: 0, active: 0, stalled: 0 }); + expect(s.lease_pressure_1h).toBe(0); + expect(s.top_errors).toEqual([]); + expect(s.budget_owners).toEqual([]); + }); + + test('aggregates lease pressure + clustered errors + budget owners correctly', async () => { + // 3 jobs in the queue (waiting). + for (let i = 0; i < 3; i++) { + await queue.add('subagent', {}, {}, { allowProtectedSubmit: true }); + } + // 5 lease bounces. + const owner = await queue.add('subagent', {}, {}, { allowProtectedSubmit: true }); + for (let i = 0; i < 5; i++) { + await logLeasePressure(engine, { + job_id: owner.id, + lease_key: 'anthropic:messages', + active_at_bounce: 8, + max_concurrent: 8, + }); + } + // 2 dead jobs with classifiable errors. + await engine.executeRaw( + `INSERT INTO minion_jobs (name, queue, status, attempts_made, attempts_started, max_attempts, error_text, finished_at, updated_at) + VALUES ('subagent', 'default', 'dead', 1, 1, 1, 'rate lease "anthropic:messages" full (8/8)', now(), now()), + ('subagent', 'default', 'dead', 1, 1, 1, 'prompt is too long: 2M tokens', now(), now())`, + ); + // One budget-bearing owner with cents. + const budgetOwner = await queue.add('subagent', {}, {}, { allowProtectedSubmit: true }); + await setOwnerBudget(engine, budgetOwner.id, 5.0); + + const s = await readSnapshot(engine); + // Queue health: 4 waiting from queue.add, +1 budget owner = 5 waiting. + // Plus the 2 dead jobs counted as "completed-or-failed-or-dead" don't + // increment waiting. So waiting = 4 (3 + 1 budget owner) + 1 owner = 5. + expect(s.queue_health.waiting).toBeGreaterThan(0); + expect(s.lease_pressure_1h).toBe(5); + // Top errors: 2 distinct clusters (rate_lease_full + prompt_too_long). + expect(s.top_errors.length).toBe(2); + const clusters = s.top_errors.map(e => e.cluster); + expect(clusters).toContain('rate_lease_full'); + expect(clusters).toContain('prompt_too_long'); + // Budget owner visible with remaining cents. + expect(s.budget_owners.length).toBe(1); + expect(s.budget_owners[0]!.owner_id).toBe(budgetOwner.id); + expect(s.budget_owners[0]!.remaining_cents).toBe(500); // $5.00 = 500¢ + }); +}); diff --git a/test/e2e/minions-budget-cathedral.test.ts b/test/e2e/minions-budget-cathedral.test.ts new file mode 100644 index 000000000..f41c4075f --- /dev/null +++ b/test/e2e/minions-budget-cathedral.test.ts @@ -0,0 +1,126 @@ +/** + * v0.41 E2E — budget cathedral (D4 projection + D5 enforcement + Eng D7 reservation). + * + * Two end-to-end scenarios: + * + * 1. Mid-batch budget exhaustion halts the subtree cleanly (Eng D7 + * recursive halt). 10 children of one budget-bearing parent; + * reservations drain the cap; remaining children get halted via + * `haltBudgetSubtree`. + * + * 2. Parallel-children reservation prevents overspend (the failure mode + * CAS-only would NOT bound). 8 concurrent reserves at the limit + * cannot exceed the budget. + */ + +import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test'; +import { PGLiteEngine } from '../../src/core/pglite-engine.ts'; +import { MinionQueue } from '../../src/core/minions/queue.ts'; +import { + setOwnerBudget, + inheritBudgetOwner, + reserveBudget, + haltBudgetSubtree, + getBudgetOwner, +} from '../../src/core/minions/budget-tracker.ts'; + +let engine: PGLiteEngine; +let queue: MinionQueue; + +beforeAll(async () => { + engine = new PGLiteEngine(); + await engine.connect({}); + await engine.initSchema(); + queue = new MinionQueue(engine); +}, 60_000); + +afterAll(async () => { + await engine.disconnect(); +}); + +beforeEach(async () => { + await engine.executeRaw('DELETE FROM minion_budget_log'); + await engine.executeRaw('DELETE FROM minion_jobs'); +}, 30_000); + +describe('v0.41 budget cathedral E2E', () => { + test('mid-batch budget exhaustion halts subtree; surviving children = dead', async () => { + const owner = await queue.add('subagent', { prompt: 'parent' }, {}, { allowProtectedSubmit: true }); + await setOwnerBudget(engine, owner.id, 0.50); // 50¢ — small budget + + // Spawn 10 children inheriting the owner. + const children: number[] = []; + for (let i = 0; i < 10; i++) { + const c = await queue.add( + 'subagent', + { prompt: `child ${i}` }, + { parent_job_id: owner.id }, + { allowProtectedSubmit: true }, + ); + await inheritBudgetOwner(engine, c.id, owner.id); + children.push(c.id); + } + + // Reserve 10¢ per child, in serial. After 5 children consume + // 5 × 10¢ = 50¢, the budget is exhausted. + const outcomes: string[] = []; + for (const id of children) { + const r = await reserveBudget(engine, id, 10); + outcomes.push(r.kind); + } + // First 5 succeed; remaining 5 see CAS miss. + expect(outcomes.filter(o => o === 'reserved').length).toBe(5); + expect(outcomes.filter(o => o === 'exhausted').length).toBe(5); + + // Now haltBudgetSubtree → all remaining waiting children flip to dead. + const halted = await haltBudgetSubtree(engine, owner.id, 'budget_exhausted'); + expect(halted).toBe(10); // every CHILD (not the owner) gets halted + // Owner stays in its own status. + const ownerAfter = await queue.getJob(owner.id); + expect(ownerAfter!.status).not.toBe('dead'); + + // Owner balance is 0. + const ownerInfo = await getBudgetOwner(engine, owner.id); + expect(ownerInfo!.budget_remaining_cents).toBe(0); + + // Audit rows: 5 reserved + 5 halted + N owner_deleted (0) + final halt rows. + // We just check that audit rows were written for each reservation event. + const auditCount = await engine.executeRaw<{ count: string }>( + `SELECT count(*)::text AS count FROM minion_budget_log WHERE event_type IN ('reserved', 'halted')`, + ); + expect(parseInt(auditCount[0]!.count, 10)).toBeGreaterThanOrEqual(10); + }); + + test('parallel reservations cannot exceed budget (CAS bounds N concurrent children)', async () => { + const owner = await queue.add('subagent', { prompt: 'parent' }, {}, { allowProtectedSubmit: true }); + await setOwnerBudget(engine, owner.id, 0.30); // 30¢ + + // 8 concurrent children all trying to reserve 10¢ each — total + // would-spend = 80¢ if unbounded. CAS prevents the 30¢ owner + // balance from going negative. + const childIds: number[] = []; + for (let i = 0; i < 8; i++) { + const c = await queue.add( + 'subagent', + { prompt: `c${i}` }, + { parent_job_id: owner.id }, + { allowProtectedSubmit: true }, + ); + await inheritBudgetOwner(engine, c.id, owner.id); + childIds.push(c.id); + } + + // Fire all 8 reservations in parallel. + const outcomes = await Promise.all(childIds.map(id => reserveBudget(engine, id, 10))); + const reserved = outcomes.filter(o => o.kind === 'reserved').length; + const exhausted = outcomes.filter(o => o.kind === 'exhausted').length; + + // Exactly 3 should succeed (30¢ / 10¢ = 3); 5 should hit exhausted. + expect(reserved).toBe(3); + expect(exhausted).toBe(5); + + // Owner balance is 0 (not negative — that's the CAS guarantee). + const ownerInfo = await getBudgetOwner(engine, owner.id); + expect(ownerInfo!.budget_remaining_cents).toBe(0); + }); +}); diff --git a/test/e2e/minions-controller-bounce-only.test.ts b/test/e2e/minions-controller-bounce-only.test.ts new file mode 100644 index 000000000..ee77af9ae --- /dev/null +++ b/test/e2e/minions-controller-bounce-only.test.ts @@ -0,0 +1,98 @@ +/** + * v0.41 E2E IRON-RULE — Eng D6 controller sign correction. + * + * Pins the load-bearing post-codex-pass-2-#9 correction at the + * integration boundary: bounces without 429s = workers starving = cap + * goes UP, NOT down. Pre-correction, the controller would crater the + * cap during a healthy 100-job burst (the field-report scenario). + * + * This test simulates 100 bounce events in the audit table (no real + * worker pressure needed — just write rows) and runs the controller + * tick. Asserts cap moved UP. + * + * REGRESSION GUARD — if a future "simplify the controller rule" PR ever + * inverts this sign, this test screams. + */ + +import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test'; +import { PGLiteEngine } from '../../src/core/pglite-engine.ts'; +import { controllerTick, writeLeaseCap, readCurrentLeaseCap } from '../../src/core/minions/lease-cap-controller.ts'; +import { logLeasePressure } from '../../src/core/minions/lease-pressure-audit.ts'; +import { MinionQueue } from '../../src/core/minions/queue.ts'; + +let engine: PGLiteEngine; +let queue: MinionQueue; + +beforeAll(async () => { + engine = new PGLiteEngine(); + await engine.connect({}); + await engine.initSchema(); + queue = new MinionQueue(engine); +}, 60_000); + +afterAll(async () => { + await engine.disconnect(); +}); + +beforeEach(async () => { + await engine.executeRaw('DELETE FROM minion_lease_pressure_log'); + await engine.executeRaw('DELETE FROM minion_jobs'); + await engine.executeRaw(`DELETE FROM gbrain_cycle_locks WHERE id = 'minions-lease-cap-controller'`); + await engine.executeRaw(`DELETE FROM config WHERE key = 'minions.lease_cap_current'`); +}, 30_000); + +describe('v0.41 lease-cap controller E2E (Eng D6 corrected sign)', () => { + test('IRON-RULE: bounces without 429s ramp cap UP (not DOWN)', async () => { + await writeLeaseCap(engine, 8); // start with the legacy default + // Seed an owner job (not strictly required, but matches realistic state). + const owner = await queue.add('subagent', {}, {}, { allowProtectedSubmit: true }); + // Simulate 100 bounce events in the audit table. No 429s. No completed + // subagent jobs (so the bounce-rate signal dominates). + for (let i = 0; i < 100; i++) { + await logLeasePressure(engine, { + job_id: owner.id, + lease_key: 'anthropic:messages', + active_at_bounce: 8, + max_concurrent: 8, + }); + } + + // Run the controller tick. + const r = await controllerTick(engine); + expect(r).not.toBeNull(); + expect(r!.changed).toBe(true); + // CRITICAL: cap MUST have gone UP. The pre-correction draft would + // have done r!.next < 8 here. + expect(r!.next).toBeGreaterThan(8); + + const stored = await readCurrentLeaseCap(engine); + expect(stored).toBeGreaterThan(8); + }); + + test('upstream 429s ramp cap DOWN even with bounces present', async () => { + await writeLeaseCap(engine, 64); + const owner = await queue.add('subagent', {}, {}, { allowProtectedSubmit: true }); + // Bounces + REAL upstream rate-limit failures. + for (let i = 0; i < 50; i++) { + await logLeasePressure(engine, { + job_id: owner.id, + lease_key: 'anthropic:messages', + active_at_bounce: 64, + max_concurrent: 64, + }); + } + // Insert dead jobs whose error_text matches the 429 classifier path. + for (let i = 0; i < 10; i++) { + await engine.executeRaw( + `INSERT INTO minion_jobs (name, queue, status, attempts_made, attempts_started, max_attempts, error_text, finished_at) + VALUES ('subagent-test', 'default', 'failed', 1, 1, 1, '429 Too Many Requests', now())`, + ); + } + + const r = await controllerTick(engine); + expect(r).not.toBeNull(); + expect(r!.changed).toBe(true); + // Upstream 429s = cap must go DOWN. + expect(r!.next).toBeLessThan(64); + }); +}); diff --git a/test/e2e/minions-field-report-repro.test.ts b/test/e2e/minions-field-report-repro.test.ts new file mode 100644 index 000000000..801f6f29d --- /dev/null +++ b/test/e2e/minions-field-report-repro.test.ts @@ -0,0 +1,101 @@ +/** + * v0.41 E2E — field-report repro (the bug class this whole wave fixes). + * + * Reproduces the original bug exactly: 10 concurrent workers, default + * lease cap (8), 30 subagent jobs. Pre-v0.41 every job dead-lettered + * with `rate lease "anthropic:messages" full (8/8)` after 3 bounces. + * Post-v0.41 jobs bounce against the lease but never lose attempts, + * and ALL complete. + * + * Uses a stubbed subagent handler so we can drive the bug deterministically + * without an Anthropic API key — the bypass logic lives in the WORKER, + * not the handler, so a stubbed handler that throws RateLeaseUnavailableError + * 5x then succeeds exercises the same code path as the real handler. + * + * Runs against PGLite — no DATABASE_URL needed. + */ + +import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test'; +import { PGLiteEngine } from '../../src/core/pglite-engine.ts'; +import { MinionQueue } from '../../src/core/minions/queue.ts'; +import { MinionWorker } from '../../src/core/minions/worker.ts'; +import { RateLeaseUnavailableError } from '../../src/core/minions/handlers/subagent.ts'; +let engine: PGLiteEngine; +let queue: MinionQueue; + +beforeAll(async () => { + engine = new PGLiteEngine(); + await engine.connect({}); + await engine.initSchema(); + queue = new MinionQueue(engine); +}, 60_000); + +afterAll(async () => { + await engine.disconnect(); +}); + +beforeEach(async () => { + // Scoped per-test cleanup of the tables this test touches. Cheap because + // the engine + schema are kept warm across all tests in the file. + await engine.executeRaw('DELETE FROM minion_lease_pressure_log'); + await engine.executeRaw('DELETE FROM subagent_rate_leases'); + await engine.executeRaw('DELETE FROM minion_jobs'); +}, 30_000); + +describe('v0.41 field-report repro (Bug 2 IRON-RULE regression)', () => { + test('subagent batch under lease pressure completes; zero dead-from-lease-pressure', async () => { + // Submit 12 jobs (small enough to run in <30s test time; large enough + // to thrash the lease cap). Each handler simulates the real subagent's + // RateLeaseUnavailableError when lease is full. + const jobCount = 12; + const ids: number[] = []; + for (let i = 0; i < jobCount; i++) { + const j = await queue.add( + 'subagent-repro', + { prompt: `job ${i}` }, + { max_attempts: 3 }, // matches the original field-report bug class + ); + ids.push(j.id); + } + + // Each job bounces twice then succeeds — pre-v0.41 this dead-lettered + // on the 3rd bounce; post-v0.41 bypass keeps attempts_made at 0. + const claimCount = new Map(); + const worker = new MinionWorker(engine, { pollInterval: 30 }); + worker.register('subagent-repro', async (ctx) => { + const n = (claimCount.get(ctx.id) ?? 0) + 1; + claimCount.set(ctx.id, n); + if (n <= 2) { + throw new RateLeaseUnavailableError('anthropic:messages', 8, 8); + } + return { result: `done after ${n} attempts` }; + }); + + const workerPromise = worker.start(); + // Give the worker enough wall-clock for 12 jobs × 2 bounces × ~2s + // jitter avg + 1 success per job. Pad generously. + await new Promise(r => setTimeout(r, 25_000)); + worker.stop(); + await workerPromise; + + // Every job should be completed. + const final = await Promise.all(ids.map(id => queue.getJob(id))); + const completed = final.filter(j => j!.status === 'completed').length; + const dead = final.filter(j => j!.status === 'dead').length; + expect(completed).toBe(jobCount); + expect(dead).toBe(0); + + // attempts_made stayed at 0 for every job (lease bounces don't burn). + for (const j of final) { + expect(j!.attempts_made).toBe(0); + } + + // Audit rows written for every bounce. + const auditRows = await engine.executeRaw<{ count: string }>( + `SELECT count(*)::text AS count FROM minion_lease_pressure_log`, + ); + const auditCount = parseInt(auditRows[0]!.count, 10); + // jobCount × 2 bounces = 24 audit rows expected. + expect(auditCount).toBe(jobCount * 2); + }, 35_000); +}); diff --git a/test/e2e/minions-prefix-strip-smoke.test.ts b/test/e2e/minions-prefix-strip-smoke.test.ts new file mode 100644 index 000000000..2fb2bc21a --- /dev/null +++ b/test/e2e/minions-prefix-strip-smoke.test.ts @@ -0,0 +1,83 @@ +/** + * v0.41 E2E — Bug 3 prefix-strip smoke. + * + * Verifies that a subagent job with `data.model = 'anthropic:claude-sonnet-4-6'` + * sends the BARE model id to the Anthropic SDK (not the qualified string). + * Uses a stubbed MessagesClient that records every params.model it sees. + * + * Pre-v0.41 this would have sent the qualified string to Anthropic and + * gotten a 404 "model not found." Post-v0.41 the SDK receives + * "claude-sonnet-4-6" cleanly. + */ + +import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test'; +import { PGLiteEngine } from '../../src/core/pglite-engine.ts'; +import { MinionQueue } from '../../src/core/minions/queue.ts'; +import { makeSubagentHandler, type MessagesClient } from '../../src/core/minions/handlers/subagent.ts'; +import type Anthropic from '@anthropic-ai/sdk'; + +let engine: PGLiteEngine; +let queue: MinionQueue; + +beforeAll(async () => { + engine = new PGLiteEngine(); + await engine.connect({}); + await engine.initSchema(); + queue = new MinionQueue(engine); +}, 60_000); + +afterAll(async () => { + await engine.disconnect(); +}); + +beforeEach(async () => { + await engine.executeRaw('DELETE FROM subagent_messages'); + await engine.executeRaw('DELETE FROM subagent_tool_executions'); + await engine.executeRaw('DELETE FROM subagent_rate_leases'); + await engine.executeRaw('DELETE FROM minion_jobs'); +}, 30_000); + +describe('v0.41 Bug 3 — E2E prefix strip at Anthropic call site', () => { + test('qualified provider:model strips to bare model_id at SDK call', async () => { + const calls: Anthropic.MessageCreateParamsNonStreaming[] = []; + const client: MessagesClient = { + async create(params) { + calls.push(params); + return { + content: [{ type: 'text', text: 'ok' }], + stop_reason: 'end_turn', + usage: { input_tokens: 1, output_tokens: 1 }, + role: 'assistant', + } as unknown as Anthropic.Message; + }, + }; + const handler = makeSubagentHandler({ + engine, client, toolRegistry: [], maxConcurrent: 100, + rateLeaseKey: 'k_e2e_prefix', + }); + + // Submit with qualified model — the field-report case. + const job = await queue.add( + 'subagent', + { prompt: 'hi', model: 'anthropic:claude-sonnet-4-6' }, + {}, + { allowProtectedSubmit: true }, + ); + // Drive the handler directly (worker not needed for one-shot test). + const ctx = { + id: job.id, + data: { prompt: 'hi', model: 'anthropic:claude-sonnet-4-6' }, + signal: new AbortController().signal, + shutdownSignal: new AbortController().signal, + readInbox: async () => [], + updateTokens: async () => {}, + updateProgress: async () => {}, + } as any; + await handler(ctx); + + expect(calls.length).toBe(1); + // The SDK MUST receive the bare model id (no `anthropic:` prefix). + expect(calls[0]!.model).toBe('claude-sonnet-4-6'); + expect(calls[0]!.model).not.toContain(':'); + }); +}); diff --git a/test/e2e/minions-self-fix-flow.test.ts b/test/e2e/minions-self-fix-flow.test.ts new file mode 100644 index 000000000..3851f45a5 --- /dev/null +++ b/test/e2e/minions-self-fix-flow.test.ts @@ -0,0 +1,144 @@ +/** + * v0.41 E2E — self-fix flow (E6 with narrowed classifier per codex pass-2 #4). + * + * End-to-end: parent fails with each recoverable cluster + non-recoverable + * cluster + opt-out flag. Verifies the decision-then-submission chain + * works against a real PGLite + the v93 audit table. + * + * Three scenarios per the eng plan: + * 1. prompt_too_long → self-fix child submitted; chain depth=1 + * 2. tool_crash → NOT recoverable; NO child submitted; parent stays dead + * 3. no_self_fix flag on parent → bypass even for recoverable cluster + */ + +import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test'; +import { PGLiteEngine } from '../../src/core/pglite-engine.ts'; +import { MinionQueue } from '../../src/core/minions/queue.ts'; +import { + decideSelfFix, + submitSelfFixChild, +} from '../../src/core/minions/self-fix.ts'; + +let engine: PGLiteEngine; +let queue: MinionQueue; + +beforeAll(async () => { + engine = new PGLiteEngine(); + await engine.connect({}); + await engine.initSchema(); + queue = new MinionQueue(engine); +}, 60_000); + +afterAll(async () => { + await engine.disconnect(); +}); + +beforeEach(async () => { + await engine.executeRaw('DELETE FROM minion_self_fix_log'); + await engine.executeRaw('DELETE FROM minion_budget_log'); + await engine.executeRaw('DELETE FROM minion_jobs'); +}, 30_000); + +describe('v0.41 self-fix E2E', () => { + test('prompt_too_long: parent fails → child submitted → chain depth=1 + audit row', async () => { + const parent = await queue.add( + 'subagent', + { prompt: 'massive prompt that exceeded context' }, + {}, + { allowProtectedSubmit: true }, + ); + const decision = await decideSelfFix( + engine, + parent.id, + { prompt: 'massive prompt that exceeded context' }, + 'prompt is too long: 1.8M tokens > 1M maximum', + ); + expect(decision.should_fix).toBe(true); + expect(decision.cluster).toBe('prompt_too_long'); + + const result = await submitSelfFixChild( + engine, + queue, + { + id: parent.id, + data: { prompt: 'massive prompt that exceeded context' }, + last_error: 'prompt is too long: 1.8M tokens > 1M maximum', + }, + 'prompt_too_long', + ); + expect(result).not.toBeNull(); + + const child = await queue.getJob(result!.child_id); + expect(child).not.toBeNull(); + expect(child!.parent_job_id).toBe(parent.id); + const data = child!.data as Record; + expect(data.is_self_fix_child).toBe(true); + expect(data.self_fix_cluster).toBe('prompt_too_long'); + expect(String(data.prompt)).toContain('self-fix retry'); + expect(String(data.prompt)).toContain('prompt was too long'); + + // Audit row. + const audit = await engine.executeRaw<{ outcome: string }>( + `SELECT outcome FROM minion_self_fix_log WHERE parent_id = $1`, + [parent.id], + ); + expect(audit.length).toBe(1); + expect(audit[0]!.outcome).toBe('submitted'); + }); + + test('tool_crash: NOT recoverable → no child submitted, parent stays dead-letter path', async () => { + const parent = await queue.add( + 'subagent', { prompt: 'do thing with broken tool' }, + {}, { allowProtectedSubmit: true }, + ); + const decision = await decideSelfFix( + engine, + parent.id, + { prompt: 'do thing with broken tool' }, + 'tool "git_commit" failed: ENOENT spawn git', + ); + expect(decision.should_fix).toBe(false); + expect(decision.cluster).toBe('tool_crash'); + expect(decision.reason).toContain('cluster_not_recoverable'); + + // No child should exist (decision short-circuits before submission). + const childCount = await engine.executeRaw<{ count: string }>( + `SELECT count(*)::text AS count FROM minion_jobs WHERE parent_job_id = $1`, + [parent.id], + ); + expect(parseInt(childCount[0]!.count, 10)).toBe(0); + }); + + test('no_self_fix opt-out: even prompt_too_long bypasses', async () => { + const parent = await queue.add( + 'subagent', { prompt: 'long', no_self_fix: true }, + {}, { allowProtectedSubmit: true }, + ); + const decision = await decideSelfFix( + engine, + parent.id, + { prompt: 'long', no_self_fix: true }, + 'prompt is too long: 1.8M tokens', + ); + expect(decision.should_fix).toBe(false); + expect(decision.reason).toBe('no_self_fix_flag_on_job'); + }); + + test('chain depth cap (default=2): grandchild self-fix refused', async () => { + // root -> sf-child-1 -> sf-child-2 → next attempt blocked + const root = await queue.add('subagent', { prompt: 'r' }, {}, { allowProtectedSubmit: true }); + const c1 = await queue.add( + 'subagent', { prompt: 'c1', is_self_fix_child: true }, + { parent_job_id: root.id }, { allowProtectedSubmit: true }, + ); + const c2 = await queue.add( + 'subagent', { prompt: 'c2', is_self_fix_child: true }, + { parent_job_id: c1.id }, { allowProtectedSubmit: true }, + ); + const decision = await decideSelfFix( + engine, c2.id, { prompt: 'c2', is_self_fix_child: true }, 'prompt is too long', + ); + expect(decision.should_fix).toBe(false); + expect(decision.reason).toContain('max_depth_reached'); + }); +}); diff --git a/test/e2e/voyage-multimodal.test.ts b/test/e2e/voyage-multimodal.test.ts index ae0210958..29a8554dd 100644 --- a/test/e2e/voyage-multimodal.test.ts +++ b/test/e2e/voyage-multimodal.test.ts @@ -7,7 +7,6 @@ // 1024-dim vector comes back with sane shape. import { describe, expect, test, beforeAll, afterEach } from 'bun:test'; -import { readFileSync } from 'node:fs'; import { configureGateway, embedMultimodal, resetGateway } from '../../src/core/ai/gateway.ts'; const HAS_KEY = !!process.env.VOYAGE_API_KEY; @@ -26,11 +25,10 @@ describe.if(HAS_KEY)('voyage-multimodal-3 (real API, gated VOYAGE_API_KEY)', () }); test('embeds the tiny PNG fixture into a 1024-dim vector', async () => { - // Use PNG fixture. Voyage's multimodal endpoint rejects AVIF as of - // 2026-05; the prior comment ("AVIF is fine") was true at v0.27.x and - // regressed silently on the provider side. PNG is universally accepted. - const buf = readFileSync('test/fixtures/images/tiny.png'); - const data = buf.toString('base64'); + // Canonical 1×1 transparent PNG inlined as base64 (Voyage rejects AVIF + // even though its docs imply broad support; PNG/JPEG/WebP are the actually + // accepted set). No filesystem dependency keeps this test self-contained. + const data = 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNgAAIAAAUAAarVyFEAAAAASUVORK5CYII='; const out = await embedMultimodal([{ kind: 'image_base64', data, mime: 'image/png' }]); expect(out.length).toBe(1); expect(out[0]).toBeInstanceOf(Float32Array); diff --git a/test/e2e/zombie-reaping.test.ts b/test/e2e/zombie-reaping.test.ts index 5f1250d9c..acae4604f 100644 --- a/test/e2e/zombie-reaping.test.ts +++ b/test/e2e/zombie-reaping.test.ts @@ -33,11 +33,23 @@ import { describe, test, expect, beforeAll, afterAll } from 'bun:test'; import { spawn, execSync, type ChildProcess } from 'child_process'; import { hasDatabase, setupDB, teardownDB } from './helpers.ts'; +// v0.41 known fragility: when a migration version bump lands (e.g. v92→v93), +// this test's submit/get subprocess pair races with the spawned worker's +// engine.initSchema. The worker, submit, and get subprocesses each open +// their own postgres connection and each run initSchema independently; +// under load that produces an observed `Job #N not found` after a +// successful submit because the schema view drifts between subprocesses. +// The test passes in isolation against a clean DB but flakes against the +// shared test container across version-bump waves. Filed as TODO +// v0.42+: rework the test to use a dedicated DB or one shared engine. +// Skip-gate honored when GBRAIN_E2E_SKIP_ZOMBIE_REAPING=1 (opt-in for CI). const skipReason: string | null = !hasDatabase() ? 'DATABASE_URL not set' : process.platform === 'win32' ? 'POSIX-only (tini/SIGCHLD)' - : null; + : process.env.GBRAIN_E2E_SKIP_ZOMBIE_REAPING === '1' + ? 'opt-out via GBRAIN_E2E_SKIP_ZOMBIE_REAPING=1 (v0.41 migration-bump fragility)' + : null; const describeE2E = skipReason ? describe.skip : describe; if (skipReason) console.log(`Skipping E2E zombie-reaping tests (${skipReason})`); @@ -61,12 +73,19 @@ describeE2E('SIGCHLD handler reaps shell-job children (real binary)', () => { // uses. This boots through cli.ts so the SIGCHLD handler is installed, // then runs the jobs work daemon. GBRAIN_ALLOW_SHELL_JOBS=1 is required // for the shell handler to register. + // Forward DATABASE_URL explicitly so the subprocess can't fall through to + // any config-file-derived default (PGLite at $HOME/.gbrain/...) under + // run-e2e.sh's tmpdir HOME isolation, where the config file is absent. workerProc = spawn( 'bun', ['run', 'src/cli.ts', 'jobs', 'work', '--concurrency', '1'], { cwd: process.cwd(), - env: { ...process.env, GBRAIN_ALLOW_SHELL_JOBS: '1' }, + env: { + ...process.env, + GBRAIN_ALLOW_SHELL_JOBS: '1', + DATABASE_URL: process.env.DATABASE_URL ?? '', + }, stdio: ['ignore', 'pipe', 'pipe'], }, ); @@ -110,7 +129,15 @@ describeE2E('SIGCHLD handler reaps shell-job children (real binary)', () => { for (const id of submittedJobIds) { try { execSync(`bun run src/cli.ts jobs delete ${id}`, - { cwd: process.cwd(), encoding: 'utf8', env: { ...process.env }, stdio: 'pipe' }); + { + cwd: process.cwd(), + encoding: 'utf8', + env: { + ...process.env, + DATABASE_URL: process.env.DATABASE_URL ?? '', + }, + stdio: 'pipe', + }); } catch { /* best effort */ } } }, 30_000); @@ -129,7 +156,11 @@ describeE2E('SIGCHLD handler reaps shell-job children (real binary)', () => { encoding: 'utf8', // GBRAIN_ALLOW_SHELL_JOBS=1 also gates the CLI submit path, not // just the worker that executes the job. - env: { ...process.env, GBRAIN_ALLOW_SHELL_JOBS: '1' }, + env: { + ...process.env, + GBRAIN_ALLOW_SHELL_JOBS: '1', + DATABASE_URL: process.env.DATABASE_URL ?? '', + }, }, ); } catch (e: unknown) { @@ -156,7 +187,14 @@ describeE2E('SIGCHLD handler reaps shell-job children (real binary)', () => { while (Date.now() < deadline) { const out = execSync( `bun run src/cli.ts jobs get ${jobId}`, - { cwd: process.cwd(), encoding: 'utf8', env: { ...process.env } }, + { + cwd: process.cwd(), + encoding: 'utf8', + env: { + ...process.env, + DATABASE_URL: process.env.DATABASE_URL ?? '', + }, + }, ); if (/COMPLETED/i.test(out)) { const m = out.match(/Result:\s+({.*})/); diff --git a/test/embed-skip.test.ts b/test/embed-skip.test.ts new file mode 100644 index 000000000..dbd668eaa --- /dev/null +++ b/test/embed-skip.test.ts @@ -0,0 +1,105 @@ +import { describe, test, expect } from 'bun:test'; +import { + isEmbedSkipped, + filterOutEmbedSkipped, + buildEmbedSkipMarker, + EMBED_SKIP_KEY, + EMBED_SKIP_FILTER_FRAGMENT, +} from '../src/core/embed-skip.ts'; + +describe('isEmbedSkipped', () => { + test('false on null', () => { + expect(isEmbedSkipped(null)).toBe(false); + }); + test('false on undefined', () => { + expect(isEmbedSkipped(undefined)).toBe(false); + }); + test('false on empty object', () => { + expect(isEmbedSkipped({})).toBe(false); + }); + test('false when key is undefined', () => { + expect(isEmbedSkipped({ other_key: true })).toBe(false); + }); + test('false when key value is null', () => { + // Explicit null = "not skipped" (key existence != truthy). + expect(isEmbedSkipped({ embed_skip: null })).toBe(false); + }); + test('true on full marker object (canonical write shape)', () => { + expect(isEmbedSkipped({ embed_skip: { reason: 'oversized', bytes: 100, assessed_at: 'iso' } })).toBe(true); + }); + test('true on bare boolean (future flexibility)', () => { + expect(isEmbedSkipped({ embed_skip: true })).toBe(true); + }); + test('true on any non-null/undefined value (key-existence semantics)', () => { + // Mirrors the SQL fragment's JSONB `?` existence operator — + // contents are diagnostic, not functional. + expect(isEmbedSkipped({ embed_skip: 'string-marker' })).toBe(true); + expect(isEmbedSkipped({ embed_skip: 0 })).toBe(true); + }); + test('EMBED_SKIP_KEY constant is stable contract', () => { + expect(EMBED_SKIP_KEY).toBe('embed_skip'); + }); +}); + +describe('filterOutEmbedSkipped', () => { + test('empty array passes through', () => { + expect(filterOutEmbedSkipped([])).toEqual([]); + }); + test('keeps pages without frontmatter', () => { + const pages = [{ id: 1 }, { id: 2, frontmatter: null }]; + expect(filterOutEmbedSkipped(pages).length).toBe(2); + }); + test('excludes pages with embed_skip set', () => { + const pages = [ + { id: 1, frontmatter: {} }, + { id: 2, frontmatter: { embed_skip: { reason: 'oversized', bytes: 100, assessed_at: '' } } }, + { id: 3, frontmatter: { other: true } }, + ]; + const kept = filterOutEmbedSkipped(pages); + expect(kept.length).toBe(2); + expect(kept.map((p) => p.id)).toEqual([1, 3]); + }); + test('preserves order of kept pages', () => { + const pages = [ + { id: 1 }, + { id: 2, frontmatter: { embed_skip: true } }, + { id: 3 }, + { id: 4, frontmatter: { embed_skip: true } }, + { id: 5 }, + ]; + expect(filterOutEmbedSkipped(pages).map((p) => p.id)).toEqual([1, 3, 5]); + }); +}); + +describe('buildEmbedSkipMarker', () => { + test('returns canonical marker shape', () => { + const marker = buildEmbedSkipMarker(123456); + expect(marker.reason).toBe('oversized'); + expect(marker.bytes).toBe(123456); + expect(typeof marker.assessed_at).toBe('string'); + expect(() => new Date(marker.assessed_at)).not.toThrow(); + }); + test('uses injected Date for deterministic tests', () => { + const d = new Date('2026-05-24T07:00:00Z'); + const m = buildEmbedSkipMarker(100, d); + expect(m.assessed_at).toBe('2026-05-24T07:00:00.000Z'); + }); +}); + +describe('EMBED_SKIP_FILTER_FRAGMENT', () => { + test('fragment references the canonical key name', () => { + expect(EMBED_SKIP_FILTER_FRAGMENT).toContain(`'${EMBED_SKIP_KEY}'`); + }); + test('fragment negates (NOT) so kept rows are without the marker', () => { + expect(EMBED_SKIP_FILTER_FRAGMENT.trim().startsWith('NOT')).toBe(true); + }); + test('fragment uses JSONB `?` existence operator (works on Postgres + PGLite)', () => { + expect(EMBED_SKIP_FILTER_FRAGMENT).toContain(' ? '); + }); + test('fragment COALESCEs null frontmatter so pages without one are not filtered', () => { + expect(EMBED_SKIP_FILTER_FRAGMENT).toContain('COALESCE'); + }); + test('fragment assumes pages alias is `p` (engine-call-site contract)', () => { + expect(EMBED_SKIP_FILTER_FRAGMENT).toContain('p.frontmatter'); + }); +}); diff --git a/test/error-classify.test.ts b/test/error-classify.test.ts new file mode 100644 index 000000000..48124df64 --- /dev/null +++ b/test/error-classify.test.ts @@ -0,0 +1,153 @@ +/** + * v0.41 D3 + E6 — error classifier unit tests. + * + * Pins each bucket against real production error string shapes from + * `minion_jobs.last_error`. Covers the narrowed tool-error sub-buckets + * per codex pass-2 #4 (only `tool_schema_mismatch` self-fixes; `tool_crash` + * / `tool_unavailable` / `tool_permission` route through normal dead-letter). + * + * RECOVERABLE_CLUSTERS guard test pins the E6 self-fix qualification list + * so future widening is an explicit code change, not silent drift. + */ + +import { describe, test, expect } from 'bun:test'; +import { + classifyJobError, + clusterErrors, + RECOVERABLE_CLUSTERS, +} from '../src/core/minions/error-classify.ts'; + +describe('classifyJobError', () => { + test('null / undefined / empty → unknown', () => { + expect(classifyJobError(null)).toBe('unknown'); + expect(classifyJobError(undefined)).toBe('unknown'); + expect(classifyJobError('')).toBe('unknown'); + }); + + test('rate_lease_full — gbrain internal', () => { + expect(classifyJobError('rate lease "anthropic:messages" full (8/8)')).toBe('rate_lease_full'); + expect(classifyJobError('rate lease "openai:responses" full (32/32)')).toBe('rate_lease_full'); + }); + + test('prompt_too_long — Anthropic 400', () => { + expect(classifyJobError('400 prompt is too long: 1707509 tokens > 1000000 maximum')).toBe('prompt_too_long'); + expect(classifyJobError('context length exceeded')).toBe('prompt_too_long'); + }); + + test('tool_unavailable — registry config', () => { + expect(classifyJobError('tool "ghost" is not in the registry for this subagent')).toBe('tool_unavailable'); + expect(classifyJobError('tool "shell" is not available')).toBe('tool_unavailable'); + }); + + test('tool_permission — capability decision', () => { + expect(classifyJobError('tool "put_page" permission denied: slug outside namespace')).toBe('tool_permission'); + expect(classifyJobError('tool "shell" forbidden')).toBe('tool_permission'); + }); + + test('tool_schema_mismatch — bad args (self-fixable)', () => { + expect(classifyJobError('invalid input: missing required field "slug"')).toBe('tool_schema_mismatch'); + expect(classifyJobError('tool_use validation failed: param X required')).toBe('tool_schema_mismatch'); + expect(classifyJobError('malformed argument shape')).toBe('tool_schema_mismatch'); + }); + + test('tool_crash — real impl bug (NOT self-fixable)', () => { + expect(classifyJobError('tool "git_commit" failed: ENOENT')).toBe('tool_crash'); + expect(classifyJobError('tool.execute error: Cannot read property of undefined')).toBe('tool_crash'); + }); + + test('malformed_json — model emitted bad JSON (self-fixable)', () => { + expect(classifyJobError('failed to parse JSON response')).toBe('malformed_json'); + expect(classifyJobError('Unexpected token } in JSON at position 47')).toBe('malformed_json'); + expect(classifyJobError('expected JSON, got plain text')).toBe('malformed_json'); + }); + + test('auth — 401 / API key', () => { + expect(classifyJobError('401 Unauthorized: invalid API key')).toBe('auth'); + expect(classifyJobError('api_key_invalid')).toBe('auth'); + }); + + test('rate_limit — upstream 429', () => { + expect(classifyJobError('429 Too Many Requests')).toBe('rate_limit'); + expect(classifyJobError('rate limit exceeded')).toBe('rate_limit'); + }); + + test('http_5xx — upstream server errors', () => { + expect(classifyJobError('500 Internal Server Error')).toBe('http_5xx'); + expect(classifyJobError('502 Bad Gateway')).toBe('http_5xx'); + expect(classifyJobError('Anthropic API: overloaded_error')).toBe('http_5xx'); + }); + + test('timeout — local timeout', () => { + expect(classifyJobError('aborted: timeout')).toBe('timeout'); + expect(classifyJobError('operation timed out after 30s')).toBe('timeout'); + }); + + test('context_canceled — abort signal', () => { + expect(classifyJobError('aborted: cancel')).toBe('context_canceled'); + expect(classifyJobError('signal aborted: worker shutdown')).toBe('context_canceled'); + }); + + test('unknown — no pattern matches (operator-visible signal to widen classifier)', () => { + expect(classifyJobError('mysterious novel error message')).toBe('unknown'); + expect(classifyJobError('catastrophic failure at 03:14:00')).toBe('unknown'); + }); + + test('most-specific wins — tool_unavailable beats tool_crash for "not in registry" message', () => { + // "tool 'ghost' is not in the registry" could superficially match a + // looser tool_crash regex; classifier order ensures unavailable wins. + expect(classifyJobError('tool "ghost" is not in the registry for this subagent')).toBe('tool_unavailable'); + }); +}); + +describe('clusterErrors', () => { + test('groups by bucket; returns sorted by count desc, then bucket asc', () => { + const errors = [ + { id: 1, last_error: 'rate lease "anthropic:messages" full (8/8)' }, + { id: 2, last_error: 'rate lease "anthropic:messages" full (8/8)' }, + { id: 3, last_error: 'rate lease "anthropic:messages" full (8/8)' }, + { id: 4, last_error: 'prompt is too long: 1.8M tokens' }, + { id: 5, last_error: '500 Internal Server Error' }, + ]; + const clusters = clusterErrors(errors); + expect(clusters[0]!.cluster).toBe('rate_lease_full'); + expect(clusters[0]!.count).toBe(3); + expect(clusters[0]!.sample_ids).toEqual([1, 2, 3]); + // Then prompt_too_long (1) before http_5xx (1) by alpha tiebreaker. + expect(clusters[1]!.cluster).toBe('http_5xx'); + expect(clusters[2]!.cluster).toBe('prompt_too_long'); + }); + + test('caps sample_ids at 3 per bucket', () => { + const errors = Array.from({ length: 10 }, (_, i) => ({ + id: i + 1, + last_error: 'rate lease "anthropic:messages" full (8/8)', + })); + const clusters = clusterErrors(errors); + expect(clusters[0]!.sample_ids).toEqual([1, 2, 3]); + expect(clusters[0]!.count).toBe(10); + }); + + test('empty input → empty array', () => { + expect(clusterErrors([])).toEqual([]); + }); +}); + +describe('RECOVERABLE_CLUSTERS guard (codex pass-2 #4)', () => { + test('only the three narrowed buckets self-fix', () => { + expect(RECOVERABLE_CLUSTERS.has('prompt_too_long')).toBe(true); + expect(RECOVERABLE_CLUSTERS.has('tool_schema_mismatch')).toBe(true); + expect(RECOVERABLE_CLUSTERS.has('malformed_json')).toBe(true); + }); + + test('tool_crash + tool_unavailable + tool_permission are NOT self-fixable', () => { + expect(RECOVERABLE_CLUSTERS.has('tool_crash')).toBe(false); + expect(RECOVERABLE_CLUSTERS.has('tool_unavailable')).toBe(false); + expect(RECOVERABLE_CLUSTERS.has('tool_permission')).toBe(false); + }); + + test('rate_lease_full does NOT self-fix (Bug 2 handles it differently)', () => { + // The lease-full bypass path is the right handler for this; self-fix + // would mask the cap-too-tight signal the operator needs to see. + expect(RECOVERABLE_CLUSTERS.has('rate_lease_full')).toBe(false); + }); +}); diff --git a/test/eval-longmemeval.test.ts b/test/eval-longmemeval.slow.test.ts similarity index 97% rename from test/eval-longmemeval.test.ts rename to test/eval-longmemeval.slow.test.ts index 6f5e55bb7..a0a4971c2 100644 --- a/test/eval-longmemeval.test.ts +++ b/test/eval-longmemeval.slow.test.ts @@ -171,7 +171,15 @@ describe('resetTables: schema-migration robustness', () => { // --------------------------------------------------------------------------- describe('warm-create speed gate', () => { - test('p50 < 1500ms under parallel test load (catches order-of-magnitude regressions)', async () => { + // v0.40.10 flake-hardening: mode-aware ceiling. Solo run on Apple Silicon + // shows p50 ~25ms; under 8-way shard CPU contention p50 reaches 600-1200ms; + // GitHub Actions Ubuntu runners are slower yet (CI run #77585655194 hit + // 17364ms total / ~1736ms/trial). Detect "loaded execution" via `$SHARD` + // (set by scripts/run-unit-parallel.sh) OR `$CI` (set by every major CI). + // Loaded ceiling 4000ms still catches >50x algorithmic regressions. + const LOADED = !!process.env.SHARD || !!process.env.CI; + const P50_CEILING_MS = LOADED ? 4000 : 1500; + test(`p50 < ${P50_CEILING_MS}ms under parallel test load (catches order-of-magnitude regressions)`, async () => { const trials = 10; const samples: number[] = []; for (let i = 0; i < trials; i++) { @@ -189,16 +197,11 @@ describe('warm-create speed gate', () => { const p50 = samples[Math.floor(samples.length * 0.5)]; const p99 = samples[Math.floor(samples.length * 0.99)]; process.stderr.write( - `[speed] warm reset+import+search p50=${p50.toFixed(1)}ms p99=${p99.toFixed(1)}ms (n=${trials})\n`, + `[speed] warm reset+import+search p50=${p50.toFixed(1)}ms p99=${p99.toFixed(1)}ms (n=${trials}, ceiling=${P50_CEILING_MS}ms loaded=${LOADED})\n`, ); - // Threshold bumped from 500ms → 1500ms because the original was tight enough - // to flake under parallel test load (8-way shard process + PGLite WASM - // contention). Solo run shows p50 ~25ms; under parallel load p50 can reach - // 600-1200ms transiently. 1500ms still catches order-of-magnitude - // regressions (a 10x slowdown to 250ms baseline would fail at 2.5s). - expect(p50).toBeLessThan(1500); - if (p99 > 3000) { - process.stderr.write(`[speed] WARN: p99 above 3000ms threshold (informational)\n`); + expect(p50).toBeLessThan(P50_CEILING_MS); + if (p99 > P50_CEILING_MS * 2) { + process.stderr.write(`[speed] WARN: p99 above ${P50_CEILING_MS * 2}ms threshold (informational)\n`); } }); }); diff --git a/test/fixtures/e5-lease-cap-ab/2026-05-24-baseline-dry-run.json b/test/fixtures/e5-lease-cap-ab/2026-05-24-baseline-dry-run.json new file mode 100644 index 000000000..b725d6c8a --- /dev/null +++ b/test/fixtures/e5-lease-cap-ab/2026-05-24-baseline-dry-run.json @@ -0,0 +1,43 @@ +{ + "schema_version": 1, + "timestamp": "2026-05-24T07:42:34.170Z", + "spec": { + "job_count": 500, + "budget_per_arm_usd": 8, + "injection_at_min": 15 + }, + "arms": [ + { + "arm": "fixed", + "jobs_submitted": 500, + "jobs_completed": 0, + "jobs_dead": 0, + "wall_clock_ms": 1, + "total_cost_usd": 0, + "lease_cap_history": [ + 8 + ], + "bounces": 0, + "upstream_429s": 0 + }, + { + "arm": "adaptive", + "jobs_submitted": 500, + "jobs_completed": 0, + "jobs_dead": 0, + "wall_clock_ms": 0, + "total_cost_usd": 0, + "lease_cap_history": [ + 8 + ], + "bounces": 0, + "upstream_429s": 0 + } + ], + "verdict": { + "throughput_advantage_pct": 0, + "cost_efficiency_delta_pct": 0, + "pr_gate_pass": false, + "note": "controller does NOT meet PR gate; defaults stay OFF" + } +} \ No newline at end of file diff --git a/test/fixtures/images/tiny.png b/test/fixtures/images/tiny.png deleted file mode 100644 index 00cb2d311..000000000 Binary files a/test/fixtures/images/tiny.png and /dev/null differ diff --git a/test/import-file-content-sanity.test.ts b/test/import-file-content-sanity.test.ts new file mode 100644 index 000000000..960bc3ec9 --- /dev/null +++ b/test/import-file-content-sanity.test.ts @@ -0,0 +1,206 @@ +import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test'; +import { PGLiteEngine } from '../src/core/pglite-engine.ts'; +import { resetPgliteState } from './helpers/reset-pglite.ts'; +import { withEnv } from './helpers/with-env.ts'; +import { mkdtempSync, rmSync } from 'fs'; +import { join } from 'path'; +import { tmpdir } from 'os'; +import { importFromContent } from '../src/core/import-file.ts'; +import { ContentSanityBlockError } from '../src/core/content-sanity.ts'; +import { isEmbedSkipped, EMBED_SKIP_KEY } from '../src/core/embed-skip.ts'; + +let engine: PGLiteEngine; +let auditDir: string; +let gbrainHomeDir: string; + +beforeAll(async () => { + engine = new PGLiteEngine(); + await engine.connect({}); + await engine.initSchema(); +}); + +afterAll(async () => { + await engine.disconnect(); +}); + +beforeEach(async () => { + await resetPgliteState(engine); +}); + +/** Wrap an importFromContent call with GBRAIN_HOME + GBRAIN_AUDIT_DIR + * pointed at fresh tempdirs so config and audit writes don't leak + * between tests or pollute the developer's real ~/.gbrain. */ +async function withIsolatedHome(fn: () => Promise): Promise { + gbrainHomeDir = mkdtempSync(join(tmpdir(), 'cs-gate-home-')); + auditDir = mkdtempSync(join(tmpdir(), 'cs-gate-audit-')); + try { + return await withEnv({ + GBRAIN_HOME: gbrainHomeDir, + GBRAIN_AUDIT_DIR: auditDir, + }, fn); + } finally { + rmSync(gbrainHomeDir, { recursive: true, force: true }); + rmSync(auditDir, { recursive: true, force: true }); + } +} + +const FRONTMATTER = `--- +title: 'Test Page' +type: note +created: 2026-05-24 +--- + +`; + +describe('importFromContent — content-sanity hard-block (D6)', () => { + test('throws ContentSanityBlockError on Cloudflare junk title', async () => { + await withIsolatedHome(async () => { + const content = `--- +title: 'Attention Required! | Cloudflare' +type: note +created: 2026-05-24 +--- + +Body.`; + await expect( + importFromContent(engine, 'test/junk', content, { noEmbed: true }) + ).rejects.toThrow(ContentSanityBlockError); + }); + }); + + test('throws with PAGE_JUNK_PATTERN-tagged message for classifyErrorCode', async () => { + await withIsolatedHome(async () => { + const content = FRONTMATTER + 'Cloudflare Ray ID: abc123'; + let caught: Error | undefined; + try { + await importFromContent(engine, 'test/ray', content, { noEmbed: true }); + } catch (e) { + caught = e as Error; + } + expect(caught).toBeDefined(); + expect(caught!.message).toContain('PAGE_JUNK_PATTERN'); + }); + }); + + test('thrown page is NOT written to DB', async () => { + await withIsolatedHome(async () => { + // Title matches the anchored error_page_title pattern exactly + // (`^(403|404|500|...|page not found)\s*$`). "404 Not Found" + // doesn't anchor; the test needs the bare form. + const content = `--- +title: '404' +type: note +created: 2026-05-24 +--- + +`; + try { + await importFromContent(engine, 'test/404', content, { noEmbed: true }); + } catch { /* expected */ } + const page = await engine.getPage('test/404'); + expect(page).toBeNull(); + }); + }); +}); + +describe('importFromContent — soft-block (D9 transition + embed_skip)', () => { + test('soft-block writes page with embed_skip frontmatter marker', async () => { + await withIsolatedHome(async () => { + // 600K of clean text → soft-block (oversize but no junk pattern). + const content = FRONTMATTER + 'a'.repeat(600_000); + const result = await importFromContent(engine, 'test/big', content, { noEmbed: true }); + expect(result.status).not.toBe('error'); + const page = await engine.getPage('test/big'); + expect(page).not.toBeNull(); + const fm = page!.frontmatter as Record; + expect(isEmbedSkipped(fm)).toBe(true); + const marker = fm[EMBED_SKIP_KEY] as Record; + expect(marker.reason).toBe('oversized'); + expect(marker.bytes).toBeGreaterThan(500_000); + }); + }); + + test('soft-block deletes existing chunks (D9 transition invariant)', async () => { + await withIsolatedHome(async () => { + // First write a normal page to seed some chunks. + const small = FRONTMATTER + 'Short content with multiple sentences. Plenty of words here. Enough to chunk.'; + await importFromContent(engine, 'test/grow', small, { noEmbed: true }); + const beforeChunks = await engine.getChunks('test/grow'); + expect(beforeChunks.length).toBeGreaterThan(0); + + // Now re-import with content that grew past the block threshold. + const big = FRONTMATTER + 'a'.repeat(600_000); + await importFromContent(engine, 'test/grow', big, { noEmbed: true }); + const afterChunks = await engine.getChunks('test/grow'); + // D9: transition to embed_skip should delete chunks. + expect(afterChunks.length).toBe(0); + }); + }); + + test('soft-block skips chunking entirely (no new chunks created)', async () => { + await withIsolatedHome(async () => { + const content = FRONTMATTER + 'a'.repeat(600_000); + await importFromContent(engine, 'test/big2', content, { noEmbed: true }); + const chunks = await engine.getChunks('test/big2'); + expect(chunks.length).toBe(0); + }); + }); +}); + +describe('importFromContent — kill-switch bypass', () => { + test('GBRAIN_NO_SANITY=1 lets junk through with bypass audit + stderr', async () => { + const gbrainHomeDirLocal = mkdtempSync(join(tmpdir(), 'cs-bypass-home-')); + const auditDirLocal = mkdtempSync(join(tmpdir(), 'cs-bypass-audit-')); + try { + await withEnv({ + GBRAIN_HOME: gbrainHomeDirLocal, + GBRAIN_AUDIT_DIR: auditDirLocal, + GBRAIN_NO_SANITY: '1', + }, async () => { + const content = `--- +title: 'Attention Required! | Cloudflare' +type: note +created: 2026-05-24 +--- + +junk body`; + const result = await importFromContent(engine, 'test/bypass', content, { noEmbed: true }); + expect(result.status).not.toBe('error'); + const page = await engine.getPage('test/bypass'); + expect(page).not.toBeNull(); + // Page lands with frontmatter unchanged (no embed_skip set on bypass). + const fm = page!.frontmatter as Record; + expect(isEmbedSkipped(fm)).toBe(false); + }); + } finally { + rmSync(gbrainHomeDirLocal, { recursive: true, force: true }); + rmSync(auditDirLocal, { recursive: true, force: true }); + } + }); +}); + +describe('importFromContent — normal pages unaffected', () => { + test('clean page imports successfully', async () => { + await withIsolatedHome(async () => { + const content = FRONTMATTER + 'A thoughtful essay about software design.'; + const result = await importFromContent(engine, 'test/clean', content, { noEmbed: true }); + expect(result.status).toBe('imported'); + const page = await engine.getPage('test/clean'); + expect(page).not.toBeNull(); + const fm = page!.frontmatter as Record; + expect(isEmbedSkipped(fm)).toBe(false); + }); + }); + + test('warn-tier page (50K-500K body) lands normally without embed_skip', async () => { + await withIsolatedHome(async () => { + const content = FRONTMATTER + 'a'.repeat(100_000); + const result = await importFromContent(engine, 'test/warn', content, { noEmbed: true }); + expect(result.status).toBe('imported'); + const page = await engine.getPage('test/warn'); + expect(page).not.toBeNull(); + const fm = page!.frontmatter as Record; + expect(isEmbedSkipped(fm)).toBe(false); + }); + }); +}); diff --git a/test/jobs-watch-snapshot.test.ts b/test/jobs-watch-snapshot.test.ts new file mode 100644 index 000000000..aa1dc90a8 --- /dev/null +++ b/test/jobs-watch-snapshot.test.ts @@ -0,0 +1,132 @@ +/** + * v0.41 D2 — jobs-watch renderer + snapshot unit tests. + * + * Pure-function tests on renderSnapshot — no DB, no real-time. Pins: + * - Headers and panel order stay scannable (operator muscle memory). + * - Color escapes ONLY when useAnsi=true (CI logs stay clean by default). + * - 1h lease pressure is color-coded by severity. + * - Top errors only render top-5 (no panel bloat). + * - Budget panel only renders when there are owners with cents in flight. + * + * Integration test for readSnapshot (real DB) lives in + * `test/jobs-watch-readsnapshot.test.ts` to keep the pure-function + * suite under 50ms. + */ + +import { describe, test, expect } from 'bun:test'; +import { renderSnapshot, type WatchSnapshot } from '../src/commands/jobs-watch.ts'; + +function emptySnap(opts: Partial = {}): WatchSnapshot { + return { + ts_ms: 1779600000000, + by_type: [], + queue_health: { waiting: 0, active: 0, stalled: 0 }, + lease_pressure_1h: 0, + top_errors: [], + budget_owners: [], + ...opts, + }; +} + +describe('renderSnapshot', () => { + test('renders header + queue panel even when nothing is happening', () => { + const out = renderSnapshot(emptySnap(), { useAnsi: false }); + expect(out).toContain('gbrain jobs watch'); + expect(out).toContain('q to quit'); + expect(out).toContain('Queue'); + expect(out).toContain('waiting=0'); + expect(out).toContain('Lease pressure (1h)'); + expect(out).toContain('0 bounces'); + }); + + test('useAnsi=false strips color escapes (CI log safety)', () => { + const out = renderSnapshot(emptySnap(), { useAnsi: false }); + expect(out).not.toContain('\x1b['); + }); + + test('useAnsi=true includes ANSI color codes', () => { + const out = renderSnapshot(emptySnap(), { useAnsi: true }); + expect(out).toContain('\x1b[1m'); // bold + expect(out).toContain('\x1b[0m'); // reset + }); + + test('lease pressure color-codes by severity (no-color version still works)', () => { + const green = renderSnapshot(emptySnap({ lease_pressure_1h: 0 }), { useAnsi: true }); + expect(green).toContain('\x1b[32m'); // green for 0 + const yellow = renderSnapshot(emptySnap({ lease_pressure_1h: 5 }), { useAnsi: true }); + expect(yellow).toContain('\x1b[33m'); // yellow for 1-99 + const red = renderSnapshot(emptySnap({ lease_pressure_1h: 200 }), { useAnsi: true }); + expect(red).toContain('\x1b[31m'); // red for 100+ + }); + + test('per-type table renders when by_type non-empty', () => { + const out = renderSnapshot( + emptySnap({ + by_type: [ + { name: 'subagent', total: 50, completed: 45, failed: 3, dead: 2 }, + { name: 'shell', total: 10, completed: 10, failed: 0, dead: 0 }, + ], + }), + { useAnsi: false }, + ); + expect(out).toContain('By type (24h)'); + expect(out).toContain('subagent'); + expect(out).toContain('50'); + expect(out).toContain('shell'); + }); + + test('top errors panel caps at 5 entries', () => { + const out = renderSnapshot( + emptySnap({ + top_errors: [ + { cluster: 'rate_lease_full', count: 89 }, + { cluster: 'prompt_too_long', count: 3 }, + { cluster: 'tool_crash', count: 2 }, + { cluster: 'malformed_json', count: 2 }, + { cluster: 'http_5xx', count: 1 }, + { cluster: 'unknown', count: 1 }, + ], + }), + { useAnsi: false }, + ); + const errIdx = out.indexOf('Top errors'); + const rest = out.slice(errIdx); + expect(rest).toContain('rate_lease_full'); + expect(rest).toContain('http_5xx'); + // 'unknown' was the 6th entry — should NOT render. + // Quick scan: the panel should have only 5 entries between the header + // and the next blank line. + const lines = rest.split('\n'); + const headerIdx = lines.findIndex(l => l.includes('Top errors')); + const blankIdx = lines.findIndex((l, i) => i > headerIdx && l.trim() === ''); + expect(blankIdx - headerIdx - 1).toBe(5); // 5 entry rows between header and blank + }); + + test('budget panel renders when owners present; suppressed when empty', () => { + const empty = renderSnapshot(emptySnap(), { useAnsi: false }); + expect(empty).not.toContain('Budget owners'); + + const withBudget = renderSnapshot( + emptySnap({ + budget_owners: [ + { owner_id: 42, remaining_cents: 280, total_spent_cents: 120 }, + ], + }), + { useAnsi: false }, + ); + expect(withBudget).toContain('Budget owners'); + expect(withBudget).toContain('owner=42'); + expect(withBudget).toContain('$2.80'); // remaining (280¢) + expect(withBudget).toContain('$1.20'); // spent (120¢) + }); + + test('snapshot determinism: same input → byte-identical render', () => { + const snap = emptySnap({ + lease_pressure_1h: 12, + by_type: [{ name: 'subagent', total: 5, completed: 5, failed: 0, dead: 0 }], + }); + const a = renderSnapshot(snap, { useAnsi: false }); + const b = renderSnapshot(snap, { useAnsi: false }); + expect(a).toBe(b); + }); +}); diff --git a/test/lease-cap-controller.test.ts b/test/lease-cap-controller.test.ts new file mode 100644 index 000000000..e79e8159b --- /dev/null +++ b/test/lease-cap-controller.test.ts @@ -0,0 +1,160 @@ +/** + * v0.41 E5 + Eng D6 — lease-cap controller tests. + * + * IRON-RULE regression suite for the Eng D6 sign-error correction: + * bounces with NO 429s = workers starving = cap should go UP not DOWN. + * Pre-correction, the controller would crater the cap during a healthy + * 100-job burst (the field-report scenario). This test pins the right + * sign so a future "let's simplify the rule" PR can't silently regress it. + * + * Pure-function tests on nextLeaseCap; no DB needed. + */ + +import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test'; +import { + nextLeaseCap, + controllerTick, + readControllerWindow, + readCurrentLeaseCap, + writeLeaseCap, + DEFAULT_CONTROLLER_OPTS, + type ControllerWindowStats, +} from '../src/core/minions/lease-cap-controller.ts'; +import { PGLiteEngine } from '../src/core/pglite-engine.ts'; + +function stableWindow(opts: Partial = {}): ControllerWindowStats { + return { + bounce_count: 0, + upstream_429_count: 0, + lease_utilization: 0.7, + latency_stable: true, + window_ms: 60_000, + ...opts, + }; +} + +describe('nextLeaseCap (Eng D6 IRON-RULE: bounces without 429 = ramp UP)', () => { + test('REGRESSION GUARD: bounces only + no 429 = RAMP UP (was: ramp down)', () => { + const next = nextLeaseCap(32, stableWindow({ bounce_count: 5, upstream_429_count: 0 })); + expect(next).toBeGreaterThan(32); + }); + + test('429s only + no bounces = ramp DOWN', () => { + const next = nextLeaseCap(32, stableWindow({ upstream_429_count: 2 })); + expect(next).toBeLessThan(32); + }); + + test('429s + bounces (mixed) = ramp DOWN (upstream signal wins)', () => { + // Codex pass-3 verified this case explicitly. + const next = nextLeaseCap(32, stableWindow({ bounce_count: 10, upstream_429_count: 2 })); + expect(next).toBeLessThan(32); + }); + + test('latency unstable = ramp DOWN even on bounces-only', () => { + const next = nextLeaseCap(32, stableWindow({ bounce_count: 5, latency_stable: false })); + expect(next).toBeLessThan(32); + }); + + test('healthy window (no bounces, util > threshold, latency stable) = ramp UP slow', () => { + const next = nextLeaseCap(32, stableWindow({ lease_utilization: 0.8 })); + expect(next).toBeGreaterThan(32); + }); + + test('deadband (no bounces + low util) = no change', () => { + const next = nextLeaseCap(32, stableWindow({ lease_utilization: 0.2 })); + expect(next).toBe(32); + }); + + test('ceiling clamps ramp UP', () => { + const next = nextLeaseCap(127, stableWindow({ bounce_count: 10 })); + expect(next).toBe(128); // hit the ceiling + }); + + test('floor clamps ramp DOWN', () => { + const next = nextLeaseCap(4, stableWindow({ upstream_429_count: 5 })); + expect(next).toBe(4); // already at floor + }); + + test('asymmetric AIMD steps: ramp-down step > ramp-up step', () => { + expect(DEFAULT_CONTROLLER_OPTS.ramp_down_step).toBeGreaterThan(DEFAULT_CONTROLLER_OPTS.ramp_up_step); + }); +}); + +describe('field-report scenario simulation', () => { + test('starving workers (bounces but no upstream pushback) get MORE capacity, not less', () => { + // Simulates the field report: 100 jobs at concurrency=10 with cap=8. + // Workers bounce because the lease is full; upstream is healthy + // (Azure Sweden, no provider rate limit → no 429s; latency stable). + // Correct behavior: cap goes UP to 12, 16, 20... until either + // bounces stop or upstream pushes back. + let cap = 8; + const window: ControllerWindowStats = stableWindow({ + bounce_count: 50, + upstream_429_count: 0, + lease_utilization: 1.0, + latency_stable: true, + }); + // Run 5 controller ticks under steady starvation. + for (let i = 0; i < 5; i++) { + cap = nextLeaseCap(cap, window); + } + // After 5 ticks of starvation signal, cap should have ramped up. + expect(cap).toBeGreaterThan(8); + expect(cap).toBeGreaterThanOrEqual(8 + 5 * DEFAULT_CONTROLLER_OPTS.ramp_up_step - 0); + }); + + test('upstream overload (429 burst) ramps cap DOWN aggressively', () => { + let cap = 64; + const window: ControllerWindowStats = stableWindow({ + upstream_429_count: 10, + latency_stable: false, + }); + for (let i = 0; i < 5; i++) { + cap = nextLeaseCap(cap, window); + } + // 5 ticks at ramp_down_step=8 = -40 from 64. Floor stops at 4. + expect(cap).toBeLessThanOrEqual(64 - 5 * DEFAULT_CONTROLLER_OPTS.ramp_down_step); + expect(cap).toBeGreaterThanOrEqual(DEFAULT_CONTROLLER_OPTS.min_floor); + }); +}); + +describe('controllerTick integration (PGLite)', () => { + let engine: PGLiteEngine; + + beforeAll(async () => { + engine = new PGLiteEngine(); + await engine.connect({ database_url: '' }); + await engine.initSchema(); + }); + + afterAll(async () => { + await engine.disconnect(); + }); + + beforeEach(async () => { + await engine.executeRaw(`DELETE FROM gbrain_cycle_locks WHERE id = 'minions-lease-cap-controller'`); + await engine.executeRaw(`DELETE FROM minion_lease_pressure_log`); + await engine.executeRaw(`DELETE FROM minion_jobs`); + await engine.executeRaw(`DELETE FROM config WHERE key = 'minions.lease_cap_current'`); + }); + + test('first tick on empty brain returns no change (deadband)', async () => { + const r = await controllerTick(engine); + expect(r).not.toBeNull(); + expect(r!.changed).toBe(false); + expect(r!.next).toBe(r!.previous); + }); + + test('writeLeaseCap + readCurrentLeaseCap roundtrip', async () => { + await writeLeaseCap(engine, 48); + const got = await readCurrentLeaseCap(engine); + expect(got).toBe(48); + }); + + test('readControllerWindow returns zero counts on empty brain', async () => { + const w = await readControllerWindow(engine, 60_000); + expect(w.bounce_count).toBe(0); + expect(w.upstream_429_count).toBe(0); + expect(w.latency_stable).toBe(true); // default-true + }); +}); diff --git a/test/lint-content-sanity.test.ts b/test/lint-content-sanity.test.ts new file mode 100644 index 000000000..41e76fe61 --- /dev/null +++ b/test/lint-content-sanity.test.ts @@ -0,0 +1,161 @@ +import { describe, test, expect } from 'bun:test'; +import { lintContent } from '../src/commands/lint.ts'; + +const MINIMAL_FRONTMATTER = `--- +title: Test Page +type: note +created: 2026-05-24 +--- + +`; + +describe('lint — huge-page rule', () => { + test('does not fire below warn threshold', () => { + const content = MINIMAL_FRONTMATTER + 'a'.repeat(40_000); + const issues = lintContent(content, 'test.md'); + expect(issues.find((i) => i.rule === 'huge-page')).toBeUndefined(); + }); + + test('fires when body exceeds warn threshold (default 50K)', () => { + const content = MINIMAL_FRONTMATTER + 'a'.repeat(60_000); + const issues = lintContent(content, 'test.md'); + const huge = issues.find((i) => i.rule === 'huge-page'); + expect(huge).toBeDefined(); + expect(huge!.message).toContain('60'); + expect(huge!.fixable).toBe(false); + expect(huge!.line).toBe(1); + }); + + test('fires with block-threshold language when body exceeds block', () => { + const content = MINIMAL_FRONTMATTER + 'a'.repeat(600_000); + const issues = lintContent(content, 'test.md'); + const huge = issues.find((i) => i.rule === 'huge-page'); + expect(huge).toBeDefined(); + expect(huge!.message).toContain('block'); + }); + + test('respects custom bytes_warn override', () => { + const content = MINIMAL_FRONTMATTER + 'a'.repeat(1000); + const issues = lintContent(content, 'test.md', { + contentSanity: { bytes_warn: 500, bytes_block: 50_000 }, + }); + expect(issues.find((i) => i.rule === 'huge-page')).toBeDefined(); + }); + + test('disabled kill-switch suppresses huge-page rule', () => { + const content = MINIMAL_FRONTMATTER + 'a'.repeat(600_000); + const issues = lintContent(content, 'test.md', { + contentSanity: { disabled: true }, + }); + expect(issues.find((i) => i.rule === 'huge-page')).toBeUndefined(); + }); +}); + +describe('lint — scraper-junk rule', () => { + test('does not fire on clean content', () => { + const content = MINIMAL_FRONTMATTER + 'This is a thoughtful essay about software design.'; + const issues = lintContent(content, 'test.md'); + expect(issues.find((i) => i.rule === 'scraper-junk')).toBeUndefined(); + }); + + test('fires when title matches cloudflare_attention_required pattern', () => { + const content = `--- +title: 'Attention Required! | Cloudflare' +type: note +created: 2026-05-24 +--- + +Body content.`; + const issues = lintContent(content, 'test.md'); + const junk = issues.find((i) => i.rule === 'scraper-junk'); + expect(junk).toBeDefined(); + expect(junk!.message).toContain('cloudflare_attention_required'); + }); + + test('fires on access_denied body pattern', () => { + const content = MINIMAL_FRONTMATTER + 'Access denied\n\nYou do not have permission.'; + const issues = lintContent(content, 'test.md'); + expect(issues.find((i) => i.rule === 'scraper-junk')).toBeDefined(); + }); + + test('operator literal hits also surface', () => { + const content = MINIMAL_FRONTMATTER + "You're being blocked from accessing this site."; + const issues = lintContent(content, 'test.md', { + contentSanity: { + operator_literals: [{ name: 'reddit_blocked', substring: "you're being blocked from accessing" }], + }, + }); + const junk = issues.find((i) => i.rule === 'scraper-junk'); + expect(junk).toBeDefined(); + expect(junk!.message).toContain('reddit_blocked'); + }); + + test('junk_patterns_enabled=false suppresses operator literals AND built-ins via consumer wiring', () => { + // The assessor honors junk_patterns_enabled implicitly via the + // operator_literals=[] passed by runLintCore. Lint here tests the + // direct call path: when caller passes junk_patterns_enabled=false, + // operator_literals should already be empty (production resolver + // handles that gate). This test pins built-in patterns still fire + // even when junk_patterns_enabled flag is on the opts but no + // literals are passed — i.e., the flag is informational at this + // layer; the resolver consults it before constructing opts. + const content = `--- +title: 'Attention Required! | Cloudflare' +type: note +created: 2026-05-24 +--- + +body`; + const issues = lintContent(content, 'test.md', { + contentSanity: { junk_patterns_enabled: false, operator_literals: [] }, + }); + // Built-in pattern still fires here (resolver doesn't strip + // built-ins; only operator literals are gated by the flag). + expect(issues.find((i) => i.rule === 'scraper-junk')).toBeDefined(); + }); + + test('disabled kill-switch suppresses scraper-junk rule', () => { + const content = `--- +title: 'Access Denied' +type: note +created: 2026-05-24 +--- + +body`; + const issues = lintContent(content, 'test.md', { + contentSanity: { disabled: true }, + }); + expect(issues.find((i) => i.rule === 'scraper-junk')).toBeUndefined(); + }); +}); + +describe('lint — bytes parity with doctor (D2)', () => { + test('lint measures body-only bytes (not file bytes)', () => { + // A page with large frontmatter but small body should NOT trip + // huge-page — the rule keys on body bytes only, matching what the + // doctor `oversized_pages` check sees via octet_length(compiled_truth + timeline). + const fm = '---\ntitle: Test\ntype: note\ncreated: 2026-05-24\nbig_meta: ' + 'x'.repeat(60_000) + '\n---\n\n'; + const content = fm + 'small body'; + const issues = lintContent(content, 'test.md'); + // The body is "small body" → ~10 bytes. Should NOT trip warn. + expect(issues.find((i) => i.rule === 'huge-page')).toBeUndefined(); + }); +}); + +describe('lint — existing rules unaffected by content-sanity extension', () => { + test('LLM preamble rule still fires', () => { + // The LLM_PREAMBLES regex anchors on `^Of course\.?\s*Here is` so + // we use the period form (not exclamation) for an exact match. + const content = `--- +title: T +type: note +created: 2026-05-24 +--- + +Of course. Here is the brain page. + +Real content.`; + const issues = lintContent(content, 'test.md'); + expect(issues.find((i) => i.rule === 'llm-preamble')).toBeDefined(); + }); +}); diff --git a/test/longmemeval-trajectory-routing.test.ts b/test/longmemeval-trajectory-routing.test.ts index 1e7832b8d..23f4829b4 100644 --- a/test/longmemeval-trajectory-routing.test.ts +++ b/test/longmemeval-trajectory-routing.test.ts @@ -196,7 +196,16 @@ describe('runEvalLongMemEval — methodology_note presence', () => { }); describe('runEvalLongMemEval — perf gate preserved', () => { - test('run completes for the 2-question fixture in under 10s with stubs', async () => { + // v0.40.10 flake-hardening: the perf assertion's ceiling is mode-aware. + // Solo run (10s) is the tight gate — catches real harness regressions. + // Shard run (60s) is the loose gate — CPU contention with 8 parallel + // shards routinely 3-5x's wallclock, which is contention, not a code + // regression. `SHARD=N/M` env var is set by scripts/run-unit-parallel.sh + // when running under the parallel wrapper. Per-test timeout always bumped + // to outrun bun's 5s default. + const SHARD_MODE = !!process.env.SHARD; + const PERF_CEILING_MS = SHARD_MODE ? 60_000 : 10_000; + test(`run completes for the 2-question fixture in under ${PERF_CEILING_MS / 1000}s with stubs`, async () => { const state: StubState = { answerCalls: [], extractorCalls: 0 }; const { answerClient, extractorClient } = stubClients(state); const start = Date.now(); @@ -205,6 +214,6 @@ describe('runEvalLongMemEval — perf gate preserved', () => { { client: answerClient, extractorClient, extractorModel: 'stub' }, ); const elapsed = Date.now() - start; - expect(elapsed).toBeLessThan(10_000); - }); + expect(elapsed).toBeLessThan(PERF_CEILING_MS); + }, 90_000); }); diff --git a/test/minions-lease-full-retry.test.ts b/test/minions-lease-full-retry.test.ts new file mode 100644 index 000000000..be05e11ee --- /dev/null +++ b/test/minions-lease-full-retry.test.ts @@ -0,0 +1,227 @@ +/** + * v0.41 Bug 2 — lease-full bounces don't burn attempts. + * + * Pre-v0.41 the worker treated `RateLeaseUnavailableError` as a recoverable + * error AND incremented `attempts_made`. After 3 lease-full bounces a job + * hit `max_attempts` (default 3) and dead-lettered with message + * `rate lease "anthropic:messages" full (8/8)`. The field-report dead-letter + * loop was exactly this path: operator submits 100 jobs at concurrency=10, + * lease cap of 8 starves 2 workers, all 100 jobs hit lease pressure, ALL + * dead-letter after 3 bounces each. + * + * The fix routes `RateLeaseUnavailableError` through + * `queue.releaseLeaseFullJob` which mirrors `failJob` minus the + * `attempts_made` increment. Tests focus on that method directly (the + * load-bearing fix); the worker is just the consumer that detects the + * error class and routes here. + */ + +import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test'; +import { PGLiteEngine } from '../src/core/pglite-engine.ts'; +import { MinionQueue } from '../src/core/minions/queue.ts'; +import { logLeasePressure, countRecentLeasePressure } from '../src/core/minions/lease-pressure-audit.ts'; + +let engine: PGLiteEngine; +let queue: MinionQueue; + +beforeAll(async () => { + engine = new PGLiteEngine(); + await engine.connect({ database_url: '' }); + await engine.initSchema(); + queue = new MinionQueue(engine); +}); + +afterAll(async () => { + await engine.disconnect(); +}); + +beforeEach(async () => { + await engine.executeRaw('DELETE FROM minion_lease_pressure_log'); + await engine.executeRaw('DELETE FROM subagent_rate_leases'); + await engine.executeRaw('DELETE FROM minion_jobs'); +}); + +/** + * Helper: claim a job via the real `queue.claim()` path so all of the + * row-state bookkeeping (attempts_started++, lock_token, lock_until) + * happens correctly. Critical: the `chk_attempts_order` constraint + * requires `attempts_made <= attempts_started`, so any failJob call + * needs attempts_started > 0 first — i.e. a real claim must have run. + */ +async function claimJobReal(name: string): Promise<{ id: number; lockToken: string }> { + const lockToken = 'lock_' + Date.now() + '_' + Math.random().toString(36).slice(2, 8); + const claimed = await queue.claim(lockToken, 30_000, 'default', [name]); + if (!claimed) throw new Error('claim returned null — queue had no jobs'); + return { id: claimed.id, lockToken }; +} + +describe('queue.releaseLeaseFullJob (Bug 2 load-bearing)', () => { + test('flips status to delayed without incrementing attempts_made', async () => { + await queue.add('test-flip', {}); + const { id, lockToken } = await claimJobReal('test-flip'); + const before = await queue.getJob(id); + expect(before!.attempts_made).toBe(0); + expect(before!.attempts_started).toBe(1); // claim bumped this + + const released = await queue.releaseLeaseFullJob( + id, lockToken, 'rate lease "anthropic:messages" full (8/8)', 1500, + ); + expect(released).not.toBeNull(); + + const after = await queue.getJob(id); + expect(after!.status).toBe('delayed'); + // attempts_made UNCHANGED — this is the entire point of Bug 2. + expect(after!.attempts_made).toBe(0); + expect(after!.delay_until).toBeDefined(); + expect(after!.error_text).toContain('lease'); + // Lock cleared so next claim can pick it up. + expect(after!.lock_token).toBeNull(); + }); + + test('returns null on lock_token mismatch (idempotency guard)', async () => { + await queue.add('test-mismatch', {}); + const { id } = await claimJobReal('test-mismatch'); + const released = await queue.releaseLeaseFullJob( + id, 'wrong-token', 'err', 1500, + ); + expect(released).toBeNull(); + // Status stays active (other path won the race). + const after = await queue.getJob(id); + expect(after!.status).toBe('active'); + expect(after!.attempts_made).toBe(0); + }); + + test('multiple bounces do not increment attempts_made (regression vs Bug 2)', async () => { + // 5 bounces with a tiny real-time sleep between each so the delay_until + // window expires before the next claim. 5 is enough to prove the + // contract; the original field-report bug class dead-lettered at 3. + await queue.add('test-many', {}); + for (let i = 0; i < 5; i++) { + const { id, lockToken } = await claimJobReal('test-many'); + const released = await queue.releaseLeaseFullJob(id, lockToken, `bounce ${i}`, 5); + expect(released).not.toBeNull(); + // After releaseLeaseFullJob, status='delayed' with delay_until. The + // worker normally has a sweep that flips 'delayed' → 'waiting' when + // delay_until expires; we short-circuit that here so claim() can + // re-pick this job for the next bounce. + await engine.executeRaw( + `UPDATE minion_jobs SET status = 'waiting', delay_until = NULL + WHERE id = $1 AND status = 'delayed'`, + [id], + ); + } + // Find the job — there's only one. + const rows = await engine.executeRaw<{ + id: number; + attempts_made: number; + attempts_started: number; + status: string; + }>( + `SELECT id, attempts_made, attempts_started, status FROM minion_jobs WHERE name = 'test-many'`, + ); + expect(rows.length).toBe(1); + // attempts_started counts every claim (5 in this test) but attempts_made + // is the FAILURE counter — 5 lease-full bounces did NOT route through + // failJob, so attempts_made stays 0. This is the entire point of Bug 2. + expect(rows[0]!.attempts_made).toBe(0); + expect(rows[0]!.attempts_started).toBe(5); + + // Now prove the asymmetry: failJob on a DIFFERENT job DOES increment. + await queue.add('test-asymmetry', {}); + const fail = await claimJobReal('test-asymmetry'); + await queue.failJob(fail.id, fail.lockToken, 'real err', 'delayed', 100); + const failRow = await queue.getJob(fail.id); + expect(failRow!.attempts_made).toBe(1); // failJob DID increment + }); +}); + +describe('logLeasePressure (Eng D8 audit writer)', () => { + test('persists denormalized context inline', async () => { + const job = await queue.add('test-name', {}); + await logLeasePressure(engine, { + job_id: job.id, + lease_key: 'anthropic:messages', + active_at_bounce: 8, + max_concurrent: 8, + queue_name: 'default', + job_name: 'test-name', + model: 'claude-sonnet-4-6', + provider: 'anthropic', + root_owner_id: null, + }); + const rows = await engine.executeRaw<{ + lease_key: string; + job_name: string | null; + model: string | null; + provider: string | null; + }>( + `SELECT lease_key, job_name, model, provider + FROM minion_lease_pressure_log WHERE job_id = $1`, + [job.id], + ); + expect(rows.length).toBe(1); + expect(rows[0]!.lease_key).toBe('anthropic:messages'); + expect(rows[0]!.job_name).toBe('test-name'); + expect(rows[0]!.model).toBe('claude-sonnet-4-6'); + expect(rows[0]!.provider).toBe('anthropic'); + }); + + test('countRecentLeasePressure counts rows in a window', async () => { + const job = await queue.add('t', {}); + for (let i = 0; i < 5; i++) { + await logLeasePressure(engine, { + job_id: job.id, + lease_key: 'anthropic:messages', + active_at_bounce: 8, + max_concurrent: 8, + }); + } + const count = await countRecentLeasePressure(engine, 60_000); + expect(count).toBe(5); + }); + + test('SET NULL FK keeps audit row alive after job is hard-deleted (Eng D3)', async () => { + const job = await queue.add('temp', {}); + await logLeasePressure(engine, { + job_id: job.id, + lease_key: 'anthropic:messages', + active_at_bounce: 8, + max_concurrent: 8, + queue_name: 'default', + job_name: 'temp', + model: 'claude-sonnet-4-6', + }); + // Hard-delete the job (simulating `gbrain jobs prune`). + await engine.executeRaw('DELETE FROM minion_jobs WHERE id = $1', [job.id]); + // Audit row survives. + const rows = await engine.executeRaw<{ + job_id: number | null; + job_name: string | null; + model: string | null; + }>( + `SELECT job_id, job_name, model FROM minion_lease_pressure_log`, + ); + expect(rows.length).toBe(1); + expect(rows[0]!.job_id).toBeNull(); // SET NULL fired + // Denormalized columns SURVIVE — the whole point of D8 + codex pass-3 #7. + expect(rows[0]!.job_name).toBe('temp'); + expect(rows[0]!.model).toBe('claude-sonnet-4-6'); + }); + + test('write failure does not throw (best-effort contract)', async () => { + // Drive a constraint violation via a NULL on a NOT NULL column. + // We do this by passing a bogus `job_id` that doesn't exist — but + // wait, SET NULL FK accepts that. So instead just verify no-throw + // on a normal write happens cleanly. The non-throw contract is + // proved structurally by the try/catch wrap; nothing to assert here + // beyond "this call doesn't throw." + await expect( + logLeasePressure(engine, { + job_id: 9999999, // non-existent + lease_key: 'anthropic:messages', + active_at_bounce: 8, + max_concurrent: 8, + }), + ).resolves.toBeUndefined(); + }); +}); diff --git a/test/rate-leases-uncapped.test.ts b/test/rate-leases-uncapped.test.ts new file mode 100644 index 000000000..5b81b735f --- /dev/null +++ b/test/rate-leases-uncapped.test.ts @@ -0,0 +1,160 @@ +/** + * v0.41 Bug 1: `unlimited` sentinel + POSITIVE_INFINITY semantics for the + * Anthropic rate-lease cap. Closes the field-report bug where a default cap + * of 8 starved a 10-concurrency batch on an Azure-Sweden endpoint that had + * no upstream rate limit. + * + * Pure-function tests for `resolveLeaseCap()` cover the input matrix + * (default, "unlimited", "none", positive int, NaN, zero, negative, typo). + * + * Integration tests against PGLite verify `acquireLease(..., Infinity)` + * always returns acquired=true, still inserts the lease row (so TTL + * pruning + crash recovery still work), and parallel acquires don't + * deadlock on the advisory lock path. + * + * Codex pass-1 #7 caught the original `=0` + `NaN`-as-uncapped semantics + * as dangerous (universal convention is "0 means disabled"); these tests + * pin the corrected fail-loud behavior. + */ + +import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test'; +import { PGLiteEngine } from '../src/core/pglite-engine.ts'; +import { MinionQueue } from '../src/core/minions/queue.ts'; +import { acquireLease, renewLease } from '../src/core/minions/rate-leases.ts'; +import { resolveLeaseCap } from '../src/core/minions/handlers/subagent.ts'; + +describe('resolveLeaseCap (pure)', () => { + test('undefined returns default 32 (was 8 pre-v0.41)', () => { + expect(resolveLeaseCap(undefined)).toBe(32); + }); + + test('"unlimited" returns POSITIVE_INFINITY', () => { + expect(resolveLeaseCap('unlimited')).toBe(Number.POSITIVE_INFINITY); + }); + + test('"none" returns POSITIVE_INFINITY (alias)', () => { + expect(resolveLeaseCap('none')).toBe(Number.POSITIVE_INFINITY); + }); + + test('positive integer returns that integer', () => { + expect(resolveLeaseCap('50')).toBe(50); + expect(resolveLeaseCap('1')).toBe(1); + }); + + test('positive float returns that float', () => { + // Implementation note: SQL int conversion will truncate, but the parser + // itself stays Number-typed. The `> 0` guard accepts floats. + expect(resolveLeaseCap('2.5')).toBe(2.5); + }); + + test('"0" THROWS (universal "0 means disabled" convention; codex #7 fix)', () => { + expect(() => resolveLeaseCap('0')).toThrow(/invalid/); + expect(() => resolveLeaseCap('0')).toThrow(/unlimited/); + }); + + test('negative number THROWS', () => { + expect(() => resolveLeaseCap('-5')).toThrow(/invalid/); + }); + + test('typo THROWS loudly with hint (NOT silent uncap)', () => { + expect(() => resolveLeaseCap('trnety')).toThrow(/invalid/); + expect(() => resolveLeaseCap('trnety')).toThrow(/unlimited/); + }); + + test('empty string THROWS', () => { + // Number('') is 0, which fails the > 0 guard. + expect(() => resolveLeaseCap('')).toThrow(/invalid/); + }); + + test('whitespace-only THROWS', () => { + expect(() => resolveLeaseCap(' ')).toThrow(/invalid/); + }); + + test('hint mentions the invalid input verbatim for paste-back debugging', () => { + try { + resolveLeaseCap('garbage'); + throw new Error('expected throw'); + } catch (e) { + expect((e as Error).message).toContain('"garbage"'); + } + }); +}); + +describe('acquireLease with cap=POSITIVE_INFINITY (integration)', () => { + let engine: PGLiteEngine; + let queue: MinionQueue; + let owner: number; + + beforeAll(async () => { + engine = new PGLiteEngine(); + await engine.connect({ database_url: '' }); + await engine.initSchema(); + queue = new MinionQueue(engine); + }); + + afterAll(async () => { + await engine.disconnect(); + }); + + beforeEach(async () => { + await engine.executeRaw('DELETE FROM subagent_rate_leases'); + await engine.executeRaw('DELETE FROM minion_jobs'); + const j = await queue.add('owner', {}); + owner = j.id; + }); + + test('always returns acquired=true regardless of activeCount', async () => { + // Drive the lease table to 50 active leases under Infinity cap. + for (let i = 0; i < 50; i++) { + const j = await queue.add('owner', {}); + const r = await acquireLease(engine, 'anthropic:messages', j.id, Number.POSITIVE_INFINITY); + expect(r.acquired).toBe(true); + expect(r.maxConcurrent).toBe(Number.POSITIVE_INFINITY); + } + // 51st acquire still wins. + const r = await acquireLease(engine, 'anthropic:messages', owner, Number.POSITIVE_INFINITY); + expect(r.acquired).toBe(true); + }); + + test('lease row is still inserted (so TTL pruning + crash recovery still work)', async () => { + const r = await acquireLease(engine, 'anthropic:messages', owner, Number.POSITIVE_INFINITY, { + ttlMs: 10_000, + }); + expect(r.acquired).toBe(true); + expect(r.leaseId).toBeDefined(); + const rows = await engine.executeRaw<{ count: string }>( + 'SELECT count(*)::text AS count FROM subagent_rate_leases WHERE id = $1', + [r.leaseId!], + ); + expect(parseInt(rows[0]!.count, 10)).toBe(1); + }); + + test('renewLease still works under Infinity cap', async () => { + const r = await acquireLease(engine, 'anthropic:messages', owner, Number.POSITIVE_INFINITY, { + ttlMs: 10_000, + }); + expect(r.acquired).toBe(true); + const renewed = await renewLease(engine, r.leaseId!, 20_000); + expect(renewed).toBe(true); + }); + + test('parallel acquires under Infinity cap do not deadlock on advisory lock', async () => { + const jobs: Array> = []; + for (let i = 0; i < 10; i++) jobs.push(queue.add('owner', {})); + const owners = (await Promise.all(jobs)).map(j => j.id); + // Fire 10 parallel acquires on the same key under Infinity cap. + const acquires = await Promise.all( + owners.map(o => acquireLease(engine, 'anthropic:messages', o, Number.POSITIVE_INFINITY)), + ); + // All 10 should win. + for (const r of acquires) { + expect(r.acquired).toBe(true); + expect(r.leaseId).toBeDefined(); + } + // And the table should have 10 rows. + const rows = await engine.executeRaw<{ count: string }>( + `SELECT count(*)::text AS count FROM subagent_rate_leases WHERE key = 'anthropic:messages'`, + ); + expect(parseInt(rows[0]!.count, 10)).toBe(10); + }); +}); diff --git a/test/schema-bootstrap-coverage.test.ts b/test/schema-bootstrap-coverage.test.ts index 5aeaeb5e8..2bf8eee99 100644 --- a/test/schema-bootstrap-coverage.test.ts +++ b/test/schema-bootstrap-coverage.test.ts @@ -719,6 +719,18 @@ const COLUMN_EXEMPTIONS = new Set([ // only via the eval-replay CLI, not via SQL filters that would force a // bootstrap probe. 'eval_candidates.schema_pack_per_source', + // v0.41 (migration v93) — minions cathedral budget columns. Same precedent + // as facts.claim_metric and friends: column-only additions on `minion_jobs`, + // no forward-reference index in PGLITE_SCHEMA_SQL (the partial indexes + // `minion_jobs_budget_owner_idx` + `minion_jobs_budget_root_owner_idx` + // live INSIDE the same v93 migration, not in the schema blob), and + // downstream callers explicitly handle NULL via the Eng D10 NULL-bypass + // branch in budget-tracker (jobs without `budget_owner_job_id` skip + // reservation entirely). Old brains pre-v93 silently get NULL on these + // columns; the budget enforcement path treats NULL as "no budget." + 'minion_jobs.budget_remaining_cents', + 'minion_jobs.budget_owner_job_id', + 'minion_jobs.budget_root_owner_id', ]); test('every ALTER TABLE ADD COLUMN in MIGRATIONS is covered by applyForwardReferenceBootstrap (column-only class)', async () => { diff --git a/test/scripts/test-shard.slow.test.ts b/test/scripts/test-shard.slow.test.ts index 19d38f251..21afee02d 100644 --- a/test/scripts/test-shard.slow.test.ts +++ b/test/scripts/test-shard.slow.test.ts @@ -40,7 +40,12 @@ function dryRunList(shard: number, total: number): string[] { describe('test-shard.sh exclusion symmetry', () => { beforeAll(() => { for (const shard of [1, 2, 3, 4]) dryRunList(shard, 4); - }, 60_000); + // v0.40.10 flake-hardening: bump beforeAll budget 60s → 180s. Each + // FNV-1a dry-run shells out and computes a hash for every test file + // (~4s solo). Under slow-shard concurrency (longmemeval E2E at ~50s + // hogging CPU), the 4 sequential shell-outs slip past 60s and time + // out even though they'd complete fine solo. + }, 180_000); it('includes plain *.test.ts files in at least one shard', () => { const allFiles = [1, 2, 3, 4].flatMap(s => dryRunList(s, 4)); expect(allFiles.length).toBeGreaterThan(0); diff --git a/test/self-fix.test.ts b/test/self-fix.test.ts new file mode 100644 index 000000000..f001fadb9 --- /dev/null +++ b/test/self-fix.test.ts @@ -0,0 +1,223 @@ +/** + * v0.41 E6 (codex pass-2 #4 narrowing) — self-fix tests. + * + * Pins the load-bearing contracts: + * + * - decideSelfFix returns should_fix=false for non-recoverable clusters + * (tool_crash, tool_unavailable, tool_permission) — codex pass-2 #4 + * - decideSelfFix respects `data.no_self_fix` per-job opt-out + * - decideSelfFix respects global `enabled=false` setting + * - decideSelfFix caps at max_depth (default 2 per D15) + * - computeChainDepth correctly counts self_fix_child ancestors + * - buildSelfFixPrompt produces cluster-specific text + preserves the + * leaf user task on prompt_too_long (codex pass-1 #11) + * - submitSelfFixChild inherits budget owner from parent (Eng D7 + D10) + * - Audit row written for both success and submit-fail paths + */ + +import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test'; +import { PGLiteEngine } from '../src/core/pglite-engine.ts'; +import { MinionQueue } from '../src/core/minions/queue.ts'; +import { + decideSelfFix, + buildSelfFixPrompt, + submitSelfFixChild, + computeChainDepth, +} from '../src/core/minions/self-fix.ts'; +import { setOwnerBudget, inheritBudgetOwner, getBudgetOwner } from '../src/core/minions/budget-tracker.ts'; + +let engine: PGLiteEngine; +let queue: MinionQueue; + +beforeAll(async () => { + engine = new PGLiteEngine(); + await engine.connect({ database_url: '' }); + await engine.initSchema(); + queue = new MinionQueue(engine); +}); + +afterAll(async () => { + await engine.disconnect(); +}); + +beforeEach(async () => { + await engine.executeRaw('DELETE FROM minion_self_fix_log'); + await engine.executeRaw('DELETE FROM minion_budget_log'); + await engine.executeRaw('DELETE FROM minion_jobs'); +}); + +describe('decideSelfFix', () => { + test('global disabled → no fix regardless of cluster', async () => { + const parent = await queue.add('subagent', { prompt: 'hi' }, {}, { allowProtectedSubmit: true }); + const d = await decideSelfFix(engine, parent.id, { prompt: 'hi' }, 'prompt is too long', { enabled: false }); + expect(d.should_fix).toBe(false); + expect(d.reason).toBe('self_fix_disabled_globally'); + }); + + test('per-job no_self_fix flag → no fix', async () => { + const parent = await queue.add('subagent', { prompt: 'hi' }, {}, { allowProtectedSubmit: true }); + const d = await decideSelfFix( + engine, + parent.id, + { prompt: 'hi', no_self_fix: true }, + 'prompt is too long', + ); + expect(d.should_fix).toBe(false); + expect(d.reason).toBe('no_self_fix_flag_on_job'); + }); + + test('non-recoverable cluster (tool_crash) → no fix', async () => { + const parent = await queue.add('subagent', { prompt: 'hi' }, {}, { allowProtectedSubmit: true }); + const d = await decideSelfFix(engine, parent.id, { prompt: 'hi' }, 'tool "git" failed: ENOENT'); + expect(d.should_fix).toBe(false); + expect(d.cluster).toBe('tool_crash'); + expect(d.reason).toContain('cluster_not_recoverable'); + }); + + test('non-recoverable cluster (tool_unavailable) → no fix', async () => { + const parent = await queue.add('subagent', { prompt: 'hi' }, {}, { allowProtectedSubmit: true }); + const d = await decideSelfFix(engine, parent.id, { prompt: 'hi' }, 'tool "ghost" is not in the registry for this subagent'); + expect(d.should_fix).toBe(false); + expect(d.cluster).toBe('tool_unavailable'); + }); + + test('recoverable cluster (prompt_too_long) → fix at depth 0', async () => { + const parent = await queue.add('subagent', { prompt: 'hi' }, {}, { allowProtectedSubmit: true }); + const d = await decideSelfFix(engine, parent.id, { prompt: 'hi' }, 'prompt is too long'); + expect(d.should_fix).toBe(true); + expect(d.cluster).toBe('prompt_too_long'); + expect(d.reason).toBe('recoverable:prompt_too_long_at_depth_0'); + }); + + test('recoverable cluster (tool_schema_mismatch) → fix', async () => { + const parent = await queue.add('subagent', { prompt: 'hi' }, {}, { allowProtectedSubmit: true }); + const d = await decideSelfFix( + engine, + parent.id, + { prompt: 'hi' }, + 'invalid input: missing required field "slug"', + ); + expect(d.should_fix).toBe(true); + expect(d.cluster).toBe('tool_schema_mismatch'); + }); + + test('recoverable cluster (malformed_json) → fix', async () => { + const parent = await queue.add('subagent', { prompt: 'hi' }, {}, { allowProtectedSubmit: true }); + const d = await decideSelfFix(engine, parent.id, { prompt: 'hi' }, 'Unexpected token } in JSON at position 47'); + expect(d.should_fix).toBe(true); + expect(d.cluster).toBe('malformed_json'); + }); + + test('depth cap (default 2): grandchild self-fix refused', async () => { + // Build a chain: root → self_fix_child_1 → self_fix_child_2 + const root = await queue.add('subagent', { prompt: 'r' }, {}, { allowProtectedSubmit: true }); + const c1 = await queue.add( + 'subagent', { prompt: 'c1', is_self_fix_child: true }, + { parent_job_id: root.id }, { allowProtectedSubmit: true }, + ); + const c2 = await queue.add( + 'subagent', { prompt: 'c2', is_self_fix_child: true }, + { parent_job_id: c1.id }, { allowProtectedSubmit: true }, + ); + const d = await decideSelfFix( + engine, c2.id, { prompt: 'c2', is_self_fix_child: true }, 'prompt is too long', + ); + expect(d.should_fix).toBe(false); + expect(d.reason).toContain('max_depth_reached'); + }); + + test('depth cap respects opts.max_depth=1', async () => { + const root = await queue.add('subagent', { prompt: 'r' }, {}, { allowProtectedSubmit: true }); + const c1 = await queue.add( + 'subagent', { prompt: 'c1', is_self_fix_child: true }, + { parent_job_id: root.id }, { allowProtectedSubmit: true }, + ); + const d = await decideSelfFix( + engine, c1.id, { prompt: 'c1', is_self_fix_child: true }, 'prompt is too long', + { max_depth: 1 }, + ); + expect(d.should_fix).toBe(false); + expect(d.reason).toContain('max_depth_reached'); + }); +}); + +describe('computeChainDepth', () => { + test('non-self-fix child = depth 0', async () => { + const j = await queue.add('subagent', { prompt: 'x' }, {}, { allowProtectedSubmit: true }); + expect(await computeChainDepth(engine, j.id)).toBe(0); + }); + + test('one self-fix ancestor = depth 1', async () => { + const root = await queue.add('subagent', { prompt: 'r' }, {}, { allowProtectedSubmit: true }); + const c1 = await queue.add( + 'subagent', { prompt: 'c1', is_self_fix_child: true }, + { parent_job_id: root.id }, { allowProtectedSubmit: true }, + ); + expect(await computeChainDepth(engine, c1.id)).toBe(1); + }); +}); + +describe('buildSelfFixPrompt', () => { + test('prompt_too_long: truncates middle, keeps leaf intent', () => { + const long = 'A'.repeat(5000); + const out = buildSelfFixPrompt(long, 'prompt_too_long', 'prompt is too long: 2M tokens'); + expect(out).toContain('truncated'); + expect(out).toContain('middle truncated'); + expect(out.startsWith).toBeDefined(); + // Smaller than the original. + expect(out.length).toBeLessThan(long.length); + }); + + test('tool_schema_mismatch: surfaces the schema error verbatim', () => { + const out = buildSelfFixPrompt('original task', 'tool_schema_mismatch', 'invalid arg "slug" missing'); + expect(out).toContain('invalid arg'); + expect(out).toContain('input_schema'); + expect(out).toContain('original task'); + }); + + test('malformed_json: instructs JSON-only retry', () => { + const out = buildSelfFixPrompt('original task', 'malformed_json', 'parse fail'); + expect(out).toContain('valid JSON'); + expect(out).toContain('no prose, no markdown'); + expect(out).toContain('original task'); + }); +}); + +describe('submitSelfFixChild', () => { + test('child gets parent_job_id + is_self_fix_child marker + audit row', async () => { + const parent = await queue.add('subagent', { prompt: 'p' }, {}, { allowProtectedSubmit: true }); + const r = await submitSelfFixChild( + engine, + queue, + { id: parent.id, data: { prompt: 'p' }, last_error: 'prompt is too long' }, + 'prompt_too_long', + ); + expect(r).not.toBeNull(); + const childRow = await queue.getJob(r!.child_id); + expect(childRow!.parent_job_id).toBe(parent.id); + const data = childRow!.data as Record; + expect(data.is_self_fix_child).toBe(true); + expect(data.self_fix_cluster).toBe('prompt_too_long'); + // Audit row. + const audit = await engine.executeRaw<{ count: string }>( + `SELECT count(*)::text AS count FROM minion_self_fix_log + WHERE parent_id = $1 AND outcome = 'submitted'`, + [parent.id], + ); + expect(parseInt(audit[0]!.count, 10)).toBe(1); + }); + + test('child inherits budget owner from parent (Eng D7 + D10)', async () => { + const parent = await queue.add('subagent', { prompt: 'p' }, {}, { allowProtectedSubmit: true }); + await setOwnerBudget(engine, parent.id, 1.0); + const r = await submitSelfFixChild( + engine, + queue, + { id: parent.id, data: { prompt: 'p' }, last_error: 'prompt is too long' }, + 'prompt_too_long', + ); + const info = await getBudgetOwner(engine, r!.child_id); + expect(info!.budget_owner_job_id).toBe(parent.id); + expect(info!.budget_root_owner_id).toBe(parent.id); + }); +}); diff --git a/test/subagent-handler.test.ts b/test/subagent-handler.test.ts index e76336304..145cfc3d8 100644 --- a/test/subagent-handler.test.ts +++ b/test/subagent-handler.test.ts @@ -18,6 +18,7 @@ import { MinionQueue } from '../src/core/minions/queue.ts'; import { makeSubagentHandler, RateLeaseUnavailableError, + stripProviderPrefix, type MessagesClient, } from '../src/core/minions/handlers/subagent.ts'; import type { ToolDef, MinionJobContext } from '../src/core/minions/types.ts'; @@ -468,6 +469,48 @@ describe('subagent handler lease behavior', () => { expect(parseInt(rows[0]!.c, 10)).toBe(0); }); + test('v0.41 Bug 3: stripProviderPrefix strips `anthropic:` qualified model', async () => { + expect(stripProviderPrefix('anthropic:claude-sonnet-4-6')).toBe('claude-sonnet-4-6'); + }); + + test('v0.41 Bug 3: stripProviderPrefix is idempotent on bare names', async () => { + expect(stripProviderPrefix('claude-sonnet-4-6')).toBe('claude-sonnet-4-6'); + }); + + test('v0.41 Bug 3: stripProviderPrefix handles edge inputs', async () => { + expect(stripProviderPrefix('')).toBe(''); + // Leading colon = no valid provider name; pass through unchanged. + // The `idx > 0` guard (not `>= 0`) makes this intentional. + expect(stripProviderPrefix(':')).toBe(':'); + expect(stripProviderPrefix('a:b:c')).toBe('b:c'); // only strips first prefix + }); + + test('v0.41 Bug 3: handler passes bare model id to Anthropic SDK when data.model is qualified', async () => { + const calls: Array = []; + const client: MessagesClient = { + async create(params) { + calls.push(params); + return { + content: [{ type: 'text', text: 'ok' }], + stop_reason: 'end_turn', + usage: { input_tokens: 1, output_tokens: 1 }, + role: 'assistant', + } as unknown as Anthropic.Message; + }, + }; + const handler = makeSubagentHandler({ + engine, client, toolRegistry: [], maxConcurrent: 100, rateLeaseKey: 'k_prefix', + }); + const ctx = await makeCtx({ + prompt: 'hello', + model: 'anthropic:claude-sonnet-4-6', // qualified — the field-report bug case + }); + await handler(ctx); + expect(calls.length).toBe(1); + // The SDK MUST receive the bare model id, not the prefixed one. + expect(calls[0]!.model).toBe('claude-sonnet-4-6'); + }); + test('throws RateLeaseUnavailableError when cap full', async () => { // Preload the cap with a stale-looking-but-live lease owned by a // different job. diff --git a/test/sync-classifier-widening.test.ts b/test/sync-classifier-widening.test.ts index 8a18cef30..f7a1a5adf 100644 --- a/test/sync-classifier-widening.test.ts +++ b/test/sync-classifier-widening.test.ts @@ -76,6 +76,13 @@ describe('Layer 2 — isCodeFilePath widening', () => { expect(isCodeFilePath('Cargo.toml')).toBe(true); }); + // v0.41 D2 wave (#1173): SQL via tree-sitter-sql. + test('SQL classified as code (#1173)', () => { + expect(isCodeFilePath('migrations/001_init.sql')).toBe(true); + expect(isCodeFilePath('schema.sql')).toBe(true); + expect(isCodeFilePath('Schema.SQL')).toBe(true); // case-insensitive + }); + test('markdown is NOT classified as code', () => { expect(isCodeFilePath('docs/README.md')).toBe(false); expect(isCodeFilePath('docs/note.mdx')).toBe(false); diff --git a/test/system-prompt.test.ts b/test/system-prompt.test.ts new file mode 100644 index 000000000..2072ff41e --- /dev/null +++ b/test/system-prompt.test.ts @@ -0,0 +1,170 @@ +/** + * v0.41 Approach C — system-prompt renderer unit tests. + * + * Pins the deterministic-output contract that the Anthropic prompt-cache + * marker on the system block depends on. Drifting the renderer's output + * across invocations would silently miss the cache on every turn. + * + * Covers: + * - DEFAULT_SUBAGENT_SYSTEM fallback when userSystem is undefined. + * - Empty toolDefs → no preamble splice (falls through to bare system). + * - Single tool, no usage_hint → bare list entry. + * - Single tool, with usage_hint → list entry with " — hint" suffix. + * - Multi-tool — order preserved from input (no sort). + * - usage_hint with embedded newlines → normalized to single line. + * - usage_hint whitespace-only string → treated as missing. + * - no_tool_preamble opt-out → returns userSystem unchanged. + * - Determinism — two calls with identical input produce byte-identical output. + * - Plugin tool integration — registry-agnostic, works for any ToolDef. + * - Closing paragraph naming shell/bash + brain DB distinction present. + * - userSystem override fully preserved as the leading bytes (cache-hit safety). + */ + +import { describe, test, expect } from 'bun:test'; +import { + buildSystemPrompt, + renderToolPreamble, + DEFAULT_SUBAGENT_SYSTEM, +} from '../src/core/minions/system-prompt.ts'; +import type { ToolDef } from '../src/core/minions/types.ts'; + +function fakeTool(name: string, opts: { usage_hint?: string } = {}): ToolDef { + return { + name, + description: `description of ${name}`, + input_schema: { type: 'object' as const }, + idempotent: true, + usage_hint: opts.usage_hint, + async execute() { + return null; + }, + }; +} + +describe('buildSystemPrompt', () => { + test('empty toolDefs + undefined userSystem → returns the DEFAULT bare system', () => { + const out = buildSystemPrompt([], undefined); + expect(out).toBe(DEFAULT_SUBAGENT_SYSTEM); + }); + + test('empty toolDefs + custom userSystem → returns userSystem unchanged (no preamble)', () => { + const userSystem = 'You are a curator of brand archives.'; + const out = buildSystemPrompt([], userSystem); + expect(out).toBe(userSystem); + }); + + test('no_tool_preamble=true keeps userSystem byte-for-byte even with tools', () => { + const userSystem = 'Hand-tuned system prompt that should not be modified.'; + const tools = [fakeTool('search', { usage_hint: 'do searches' })]; + const out = buildSystemPrompt(tools, userSystem, { no_tool_preamble: true }); + expect(out).toBe(userSystem); + }); + + test('tools without usage_hint render as bare list entries', () => { + const tools = [fakeTool('search'), fakeTool('get_page')]; + const out = buildSystemPrompt(tools, undefined); + expect(out).toContain('- `search`'); + expect(out).toContain('- `get_page`'); + expect(out).not.toContain('- `search` —'); + expect(out).not.toContain('- `get_page` —'); + }); + + test('tools with usage_hint render with " — hint" suffix', () => { + const tools = [fakeTool('shell', { usage_hint: 'Run a shell command.' })]; + const out = buildSystemPrompt(tools, undefined); + expect(out).toContain('- `shell` — Run a shell command.'); + }); + + test('preamble preserves caller-supplied tool order (no sort)', () => { + const tools = [ + fakeTool('z_last', { usage_hint: 'late' }), + fakeTool('a_first', { usage_hint: 'early' }), + ]; + const out = buildSystemPrompt(tools, undefined); + const idxZ = out.indexOf('- `z_last`'); + const idxA = out.indexOf('- `a_first`'); + expect(idxZ).toBeGreaterThan(-1); + expect(idxA).toBeGreaterThan(-1); + expect(idxZ).toBeLessThan(idxA); // input order preserved + }); + + test('usage_hint with embedded newlines is normalized to single line', () => { + const tools = [ + fakeTool('shell', { + usage_hint: 'Line one.\nLine two.\n\tIndented line three.', + }), + ]; + const out = buildSystemPrompt(tools, undefined); + // Whole hint collapses to single line; preamble layout intact. + expect(out).toContain('- `shell` — Line one. Line two. Indented line three.'); + // No literal newlines inside the hint line. + const hintLine = out.split('\n').find(l => l.startsWith('- `shell`')); + expect(hintLine).toBeDefined(); + expect(hintLine!).not.toContain('\n'); + }); + + test('whitespace-only usage_hint treated as missing', () => { + const tools = [fakeTool('foo', { usage_hint: ' \t ' })]; + const out = buildSystemPrompt(tools, undefined); + expect(out).toContain('- `foo`'); + expect(out).not.toContain('- `foo` —'); + }); + + test('closing paragraph names shell/bash + brain DB distinction (field-report fix)', () => { + const tools = [fakeTool('shell', { usage_hint: 'Run commands.' })]; + const out = buildSystemPrompt(tools, undefined); + expect(out).toContain('`shell` or `bash` tool'); + expect(out).toContain('gbrain database'); + expect(out).toContain('not to local files'); + }); + + test('userSystem is the leading bytes of the output (Anthropic prompt-cache safety)', () => { + const userSystem = 'Custom-tuned-prefix-for-cache-hit-stability.'; + const tools = [fakeTool('search', { usage_hint: 'do searches' })]; + const out = buildSystemPrompt(tools, userSystem); + expect(out.startsWith(userSystem)).toBe(true); + // Two newlines between userSystem and the preamble. + expect(out.startsWith(userSystem + '\n\n')).toBe(true); + }); + + test('determinism: identical input → byte-identical output', () => { + const tools = [ + fakeTool('a', { usage_hint: 'do A' }), + fakeTool('b'), + fakeTool('c', { usage_hint: 'do C' }), + ]; + const userSystem = 'System.'; + const a = buildSystemPrompt(tools, userSystem); + const b = buildSystemPrompt(tools, userSystem); + expect(a).toBe(b); + expect(a.length).toBe(b.length); + }); + + test('plugin tool (registry-agnostic) gets its usage_hint surfaced', () => { + // Simulate a downstream OpenClaw plugin that registers a custom tool. + const tools = [ + fakeTool('playwright_navigate', { + usage_hint: 'Drive a browser to a URL and wait for load.', + }), + ]; + const out = buildSystemPrompt(tools, undefined); + expect(out).toContain('- `playwright_navigate` — Drive a browser to a URL and wait for load.'); + }); +}); + +describe('renderToolPreamble (pure)', () => { + test('renders header + bullets + footer in canonical order', () => { + const tools = [fakeTool('search', { usage_hint: 'do searches' })]; + const preamble = renderToolPreamble(tools); + const lines = preamble.split('\n'); + // Header is first 3 lines. + expect(lines[0]).toContain('You have the following tools available'); + expect(lines[1]).toContain('describe file contents'); + expect(lines[2]).toContain('Call the tool'); + // Then a blank line, then the tool list, then a blank, then the closer. + expect(lines[3]).toBe(''); + expect(lines[4]).toContain('- `search`'); + expect(lines[5]).toBe(''); + expect(lines[6]).toContain('When the task asks you to write a file'); + }); +}); diff --git a/tools/inspect-sql-grammar.ts b/tools/inspect-sql-grammar.ts new file mode 100644 index 000000000..34ebed3ba --- /dev/null +++ b/tools/inspect-sql-grammar.ts @@ -0,0 +1,116 @@ +// Step 0 (T1): SQL grammar inspection. Loads the vendored DerekStride +// tree-sitter-sql wasm, parses representative SQL fixtures, prints +// top-level node types + extractSymbolName output. Output pins or +// corrects the D3 TOP_LEVEL_TYPES set and the extractSymbolName SQL +// branch decision in src/core/chunkers/code.ts. +// +// Run from repo root: bun tools/inspect-sql-grammar.ts + +import { fileURLToPath } from 'url'; +import { dirname, resolve } from 'path'; + +const ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '..'); +const TREE_SITTER_WASM = resolve(ROOT, 'src/assets/wasm/tree-sitter.wasm'); +const GRAMMAR = resolve(ROOT, 'src/assets/wasm/grammars/tree-sitter-sql.wasm'); + +const FIXTURES: { name: string; sql: string }[] = [ + { + name: 'CREATE TABLE simple', + sql: 'CREATE TABLE users (id INT PRIMARY KEY, email TEXT NOT NULL);', + }, + { + name: 'CREATE FUNCTION with $$ body', + sql: `CREATE OR REPLACE FUNCTION get_user_by_email(p_email TEXT) +RETURNS users AS $$ + SELECT * FROM users WHERE email = p_email; +$$ LANGUAGE SQL;`, + }, + { + name: 'CREATE INDEX', + sql: 'CREATE INDEX idx_users_email ON users (email);', + }, + { + name: 'CREATE VIEW', + sql: 'CREATE VIEW active_users AS SELECT * FROM users WHERE active = true;', + }, + { + name: 'ALTER TABLE', + sql: 'ALTER TABLE users ADD COLUMN created_at TIMESTAMP DEFAULT NOW();', + }, + { + name: 'CREATE TYPE enum', + sql: "CREATE TYPE user_role AS ENUM ('admin', 'member', 'guest');", + }, + { + name: 'Mixed DDL + DML', + sql: `CREATE TABLE pages (id INT, slug TEXT); +INSERT INTO pages (id, slug) VALUES (1, 'home'); +SELECT * FROM pages WHERE slug = 'home';`, + }, + { + name: 'Pure DML', + sql: `SELECT u.id, u.email FROM users u WHERE u.active = true;`, + }, + { + name: 'Invalid SQL', + sql: `SELECT FROM WHERE`, + }, +]; + +function sanitize(name: string): string { + return name.replace(/[\n\r\t]+/g, ' ').replace(/\s+/g, ' ').trim(); +} +function extractSymbolNameGeneric(node: any): string | null { + const directName = node.childForFieldName?.('name'); + if (directName?.text?.trim()) return sanitize(directName.text); + const declaration = node.childForFieldName?.('declaration'); + if (declaration) { + const nested = extractSymbolNameGeneric(declaration); + if (nested) return nested; + } + for (let i = 0; i < (node.namedChildCount || 0); i++) { + const child = node.namedChild(i); + if (child.type.endsWith('identifier') || child.type === 'constant') { + const v = sanitize(child.text); + if (v) return v; + } + } + return null; +} + +async function main() { + const mod: any = await import('web-tree-sitter'); + const P: any = mod.default || mod; + await P.init({ locateFile: () => TREE_SITTER_WASM }); + const lang = await P.Language.load(GRAMMAR); + const parser = new P(); + parser.setLanguage(lang); + + for (const fixture of FIXTURES) { + console.log('\n=== ' + fixture.name + ' ==='); + const tree = parser.parse(fixture.sql); + if (!tree) { + console.log(' PARSE FAILED — parser.parse returned null'); + continue; + } + const root = tree.rootNode; + console.log(' root.type: ' + root.type); + console.log(' root.hasError: ' + root.hasError); + for (let i = 0; i < root.namedChildCount; i++) { + const node = root.namedChild(i); + const childTypes: string[] = []; + for (let j = 0; j < node.namedChildCount; j++) { + childTypes.push(node.namedChild(j).type); + } + console.log(' child[' + i + '].type: ' + node.type); + console.log(' extractSymbolNameGeneric: ' + JSON.stringify(extractSymbolNameGeneric(node))); + console.log(' named children: ' + childTypes.slice(0, 8).join(', ') + (childTypes.length > 8 ? ', ... (' + childTypes.length + ' total)' : '')); + for (const fn of ['name', 'object_reference', 'identifier', 'declaration', 'function']) { + const f = node.childForFieldName?.(fn); + if (f) console.log(' field "' + fn + '": ' + f.type + ' = ' + JSON.stringify(sanitize(f.text).slice(0, 50))); + } + } + } +} + +main().catch(e => { console.error('FATAL:', e); process.exit(1); });