mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-27 21:19:18 +00:00
* feat(eval): cycle-default — single source of truth for TTY/non-TTY cycle count (#1784) * fix(jobs): decouple 'jobs watch' format (--json) from loop (--follow); non-TTY prints one human snapshot (#1784) * fix(reindex): human cost-refusal unless --json; spend guardrail unchanged (#1784) * fix(eval): annotate non-interactive cycle/budget defaults; runner core TTY-agnostic (#1784) * chore: bump version and changelog (v0.42.15.0) Decouple CLI primary output from process.stdout.isTTY (#1784): human by default, JSON only with --json; jobs watch non-TTY one-shots; eval banners annotate non-interactive defaults; reindex-code refusal is human unless --json. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: CLAUDE.md Key Files notes for v0.42.15.0 isTTY-output decoupling (#1784) Annotate jobs.ts (jobs watch format/loop split + resolveWatchMode + the new cycle-default.ts), reindex-code.ts (human cost-refusal), and eval-cross-modal.ts (cycle/budget banner annotations). Regenerate llms-full.txt. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Garry Tan
co-authored by
Claude Opus 4.8
Claude Opus 4.8
parent
1036f8f752
commit
488f89e0dc
@@ -2,6 +2,42 @@
|
||||
|
||||
All notable changes to GBrain will be documented in this file.
|
||||
|
||||
## [0.42.15.0] - 2026-06-02
|
||||
|
||||
**Commands print real data when you run them from a subagent, a pipe, or cron, not just when you have a terminal.** A handful of gbrain commands quietly changed what they printed based on whether a terminal was attached. Run them from a coding agent, a `| cat` pipe, or a cron job and you'd get JSON when you wanted human text, or nothing useful at all. The read commands (`get`, `list`, `search`, `query`) were already fine; this fixes the ones that weren't.
|
||||
|
||||
The clearest case was `gbrain jobs watch`. In a terminal you got the live dashboard; piped or from a subagent you got an endless stream of JSON with no way to a human view. Now the rule is simple and the same for every command: **human output by default, JSON only when you pass `--json`.** A terminal still controls cosmetics (the live cursor-managed dashboard, colors), never the data.
|
||||
|
||||
```
|
||||
gbrain jobs watch # terminal: live dashboard. piped: one human snapshot, then exits.
|
||||
gbrain jobs watch --json # one JSON snapshot (machine-readable)
|
||||
gbrain jobs watch --follow # stream continuously (human, or JSONL with --json)
|
||||
```
|
||||
|
||||
`jobs watch` split into two independent knobs: `--json` picks the format, `--follow` picks the cadence. Non-interactive runs print one snapshot and exit (clean for capture), so a subagent that just wants the current queue state gets it in one shot instead of a loop it has to kill.
|
||||
|
||||
### What else changed
|
||||
|
||||
- **`gbrain reindex --code` refusal is now readable.** When it declines to spend money re-embedding without `--yes` in a non-interactive shell, it now prints a plain-English refusal instead of a JSON error blob (JSON only with `--json`). The safety behavior is unchanged: it still refuses and exits 2 rather than spending unconfirmed.
|
||||
- **The eval commands stop hiding why they did less work.** `gbrain eval cross-modal` and `gbrain eval takes-quality` run fewer cycles non-interactively (a deliberate cost guard). They already printed the cycle count; now the line says *why* it's low and how to change it: `cycles: 1 (non-interactive default; --cycles N for more)`. Same for the `$1` budget default on `gbrain eval suspected-contradictions` (`--budget-usd N to raise`).
|
||||
|
||||
Nothing here changes a TTY/interactive session: the live `jobs watch` dashboard, prompts, and your normal terminal output are all identical.
|
||||
|
||||
## To take advantage of v0.42.15.0
|
||||
|
||||
`gbrain upgrade` is all you need. There's no migration and no schema change.
|
||||
|
||||
1. **Update:**
|
||||
```bash
|
||||
gbrain upgrade
|
||||
```
|
||||
2. **Verify the fix:** run a command non-interactively and confirm you get real output.
|
||||
```bash
|
||||
gbrain jobs watch </dev/null | cat # one human snapshot, exits 0
|
||||
gbrain jobs watch --json </dev/null | cat # one JSON line
|
||||
```
|
||||
3. **If you scripted `gbrain jobs watch` for a JSON stream non-interactively,** add `--json --follow` to keep the old streaming behavior. (For scripting, `gbrain jobs stats --json` / `gbrain jobs list --json` remain the cleaner surfaces.)
|
||||
4. **If anything looks wrong,** file an issue at https://github.com/garrytan/gbrain/issues with the command you ran and what you saw.
|
||||
## [0.42.14.0] - 2026-06-02
|
||||
|
||||
**Two zero-config gaps closed: code-* queries now tell you whether the graph is built, and `gbrain init` tells you up front when your embedding key is missing.**
|
||||
|
||||
@@ -1,5 +1,26 @@
|
||||
# TODOS
|
||||
|
||||
## v0.42.15.0 isTTY-output follow-ups (v0.42+)
|
||||
|
||||
Filed from the v0.42.15.0 wave (#1784, decouple primary output from
|
||||
`process.stdout.isTTY`). Both are the same axis-conflation class the wave fixed
|
||||
but were deliberately scoped OUT — neither is a #1784 regression.
|
||||
|
||||
- [ ] **P2 — `sync.ts:2491` emits a JSON cost-refusal even without `--json`.** The
|
||||
`gbrain sync --all` cost gate has the byte-identical pattern that
|
||||
`reindex-code.ts:457` had before #1784: non-TTY or `--json` → JSON envelope +
|
||||
exit 2, conflating "refuse to spend" with "machine-readable output." The
|
||||
refusal should be human text unless `--json` is explicit. Out of scope for
|
||||
#1784 because the sync cost-gate is documented as intentional in CLAUDE.md and
|
||||
deserves its own deliberate change. Fix: mirror the extracted
|
||||
`buildCostRefusal({json, ...})` helper (`reindex-code.ts`). The guardrail
|
||||
(exit 2, no spend) stays; only the FORMAT splits on `--json`.
|
||||
- [ ] **P3 — `gbrain jobs --help` has no subcommand list.** jobs.ts dispatches
|
||||
on a bare subcommand string with no HELP const, so `watch` (and every other
|
||||
jobs subcommand) is undocumented in `--help`. The new `watch` `--json` /
|
||||
`--follow` flags are documented only in the file JSDoc. Add a HELP table to the
|
||||
`jobs` command listing every subcommand + its flags.
|
||||
|
||||
## v0.42.12.0 self-upgrade follow-ups (v0.43+)
|
||||
|
||||
Filed from the self-upgrading-gbrain wave. All deliberately scoped OUT (D7a/D7b
|
||||
|
||||
@@ -27,7 +27,7 @@ per-release `**vX.Y.Z:**` narration — CI enforces this
|
||||
- `src/core/embedding-pricing.ts` — `EMBEDDING_PRICING` map keyed `provider:model` for the post-upgrade reindex cost estimate. Sibling to `anthropic-pricing.ts`. Entries: OpenAI text-embedding-3-large ($0.13/1M), 3-small ($0.02/1M), ada-002 ($0.10/1M), Voyage 3-large ($0.18/1M), 3 ($0.06/1M). `lookupEmbeddingPrice(modelString)` returns a tagged union (`known` with price + `unknown` with provider name); `estimateCostFromChars(charCount, pricePerMTok)` uses 3.5 chars/token. Unknown providers degrade to "estimate unavailable" instead of fabricating numbers.
|
||||
- `src/core/post-upgrade-reembed.ts` — Pure functions backing the `gbrain upgrade` chunker-bump cost prompt. `computeReembedEstimate(engine, model)` queries real SQL (`COUNT(*)` + `COALESCE(SUM(LENGTH(compiled_truth)) + SUM(LENGTH(timeline)), 0)`) on `pages WHERE chunker_version < MARKDOWN_CHUNKER_VERSION`. `formatReembedPrompt(est, graceSeconds)` is the stderr-line formatter. `runPostUpgradeReembedPrompt(engine, model, opts)` orchestrates the 10-second Ctrl-C window; TTY-only wait (non-TTY auto-proceeds for CI / cron); `GBRAIN_NO_REEMBED=1` bails with a doctor-warning marker; `GBRAIN_REEMBED_GRACE_SECONDS=0` skips the wait.
|
||||
- `src/commands/reindex.ts` — `gbrain reindex --markdown [--limit N] [--dry-run] [--json] [--no-embed] [--repo PATH]`. Walks `pages WHERE page_kind = 'markdown' AND chunker_version < MARKDOWN_CHUNKER_VERSION` in 100-row batches ordered by id. Rows with non-null `source_path` re-import via `importFromFile`; rows without fall back to `importFromContent`. Both paths pass `forceRechunk: true` to bypass `importFromContent`'s `content_hash` short-circuit — without it the chunker version bump never reaches pages whose source content hasn't changed, AND the stripFactsFence privacy strip never applies to pre-strip chunks. Idempotent — partial-completion re-runs pick up via id-ordered batches. Wired into `src/commands/upgrade.ts:runPostUpgrade` after `apply-migrations`. The DB-only fallback (no source file on disk) does NOT pass body-only `compiled_truth` to `importFromContent` (that path re-parses with EMPTY frontmatter and OVERWRITES the page's real frontmatter/title/timeline); it `getPage`+`getTags`, reconstructs FULL markdown via `serializeMarkdown(frontmatter, compiled_truth, timeline, {type, title, tags})`, and re-imports THAT so re-chunking a DB-only page preserves everything while bumping `chunker_version`. Pinned by `test/reindex-preserve-tags.test.ts`.
|
||||
- `src/commands/reindex-code.ts` — `gbrain reindex --code [--source ID] [--dry-run] [--yes] [--json] [--force] [--no-embed]`. Walks `pages WHERE type = 'code'` in 100-row batches, replays through `importCodeFile` for chunk + embed + content_hash folding. Idempotent unless `--force` bypasses the content_hash early-return. Cost-preview model field reads `getEmbeddingModelName()` from the gateway so preview reflects what the gateway will actually embed with. An informational stderr nudge inside `runReindexCode` (so dry-run + execute both surface it): when the configured embedding model isn't code-tuned (allowlist `{'voyage-code-3'}`, case-insensitive bare match), prints a recommendation to switch to `voyage:voyage-code-3`; suppress with `GBRAIN_NO_CODE_MODEL_NUDGE=1`, `--no-embed`, or `--json`. Pure `shouldNudgeCodeModel(bareName)` returns a tagged `NudgeDecision` union (takes the bare model name, emits qualified `voyage:voyage-code-3` for the paste-ready `gbrain config set` line). Pinned by `test/ai/voyage-code-3-recipe.test.ts`, `test/reindex-code-nudge.serial.test.ts`, `test/reindex-code-model-source.serial.test.ts` (IRON-RULE regression for the cost-preview fix).
|
||||
- `src/commands/reindex-code.ts` — `gbrain reindex --code [--source ID] [--dry-run] [--yes] [--json] [--force] [--no-embed]`. Walks `pages WHERE type = 'code'` in 100-row batches, replays through `importCodeFile` for chunk + embed + content_hash folding. Idempotent unless `--force` bypasses the content_hash early-return. Cost-preview model field reads `getEmbeddingModelName()` from the gateway so preview reflects what the gateway will actually embed with. An informational stderr nudge inside `runReindexCode` (so dry-run + execute both surface it): when the configured embedding model isn't code-tuned (allowlist `{'voyage-code-3'}`, case-insensitive bare match), prints a recommendation to switch to `voyage:voyage-code-3`; suppress with `GBRAIN_NO_CODE_MODEL_NUDGE=1`, `--no-embed`, or `--json`. Pure `shouldNudgeCodeModel(bareName)` returns a tagged `NudgeDecision` union (takes the bare model name, emits qualified `voyage:voyage-code-3` for the paste-ready `gbrain config set` line). When `--yes` is absent and the caller is non-TTY or passed `--json`, the cost gate refuses (exit 2, no spend) via the pure exported `buildCostRefusal({json, previewMsg, preview, costUsd, model}): {stdout?, stderr?}` — JSON envelope only when `--json` is explicit, otherwise a human refusal on stderr (the spend guardrail is independent of the output format). Pinned by `test/ai/voyage-code-3-recipe.test.ts`, `test/reindex-code-nudge.serial.test.ts`, `test/reindex-code-model-source.serial.test.ts` (IRON-RULE regression for the cost-preview fix), `test/reindex-cost-refusal.test.ts`.
|
||||
- `src/commands/sync.ts:resolveSlugByPathOrSourcePath` — Resolves a slug by `pages.source_path` first (returns the stored slug for frontmatter-fallback pages whose path doesn't derive a slug), then falls back to `resolveSlugForPath(path)`. Threaded into all 4 delete/rename call sites (`performSync`'s un-syncable cleanup at ~:531, deletes at ~:603, rename oldSlug at ~:622). Without this, emoji-only / Thai / Arabic filenames whose slug came from frontmatter would orphan on delete/rename (the delete path would compute the wrong path-derived slug). Best-effort query — pre-migration brains fall through to the legacy path.
|
||||
- `src/core/utils.ts` — Shared SQL utilities extracted from postgres-engine.ts. Exports `parseEmbedding(value)` (throws on unknown input, used by migration + ingest paths where data integrity matters) and `tryParseEmbedding(value)` (returns `null` + warns once per process, used by search/rescore paths where availability matters more than strictness). `isUndefinedColumnError(err)` predicate — pattern-matches Postgres SQLSTATE 42703 / "column ... does not exist" with engine-driver shape variation tolerated; replaces bare `catch {}` blocks in `oauth-provider.ts` so genuine errors (lock timeout, network blip, permission denied) propagate while column-missing falls through to the legacy fallback. `validateSourceId(id)` throws on anything outside `^[a-z0-9_-]+$`, used by the per-source disk-layout fix in patterns.ts/synthesize.ts before any `join(brainDir, '.sources', source_id, slug+'.md')` so source_id can't traverse out of brainDir. `rowToPage` populates the required `Page.source_id` from the SELECT projection (`scripts/check-source-id-projection.sh` enforces every projection feeding `rowToPage` includes the column).
|
||||
- `src/core/db.ts` — Connection management, schema initialization. `resolveSessionTimeouts()` returns `statement_timeout` + `idle_in_transaction_session_timeout` (defaults 5min each, env-overridable via `GBRAIN_STATEMENT_TIMEOUT`/`GBRAIN_IDLE_TX_TIMEOUT`/`GBRAIN_CLIENT_CHECK_INTERVAL`). Both `connect()` (module singleton) and `PostgresEngine.connect()` (worker pool) consume the result via postgres.js's `connection` option, sending GUCs as startup parameters that survive PgBouncer transaction mode (`setSessionDefaults` kept as a back-compat no-op shim).
|
||||
@@ -64,7 +64,8 @@ per-release `**vX.Y.Z:**` narration — CI enforces this
|
||||
- `docs/architecture/RETRIEVAL.md` + `docs/architecture/RETRIEVAL_MAXPOOL_INCIDENT.md` — retrieval-pipeline architecture reference + the named-thing-miss incident write-up (root cause, the five-layer fix, the eval that pins it).
|
||||
- `src/core/types.ts` extension + `src/core/operations.ts:search` + `src/core/import-file.ts` + `src/cli.ts` + `src/core/search/telemetry.ts` — the wiring layer for the retrieval cathedral. `SearchResult` gains `evidence`, `create_safety`, `title_match_boost`, `alias_hit` (all optional; evidence/create_safety reference the union types in `evidence.ts`). The `search` MCP op uses a cheap-hybrid path by default and accepts a per-call `mode` (conservative|balanced|tokenmax) honored ONLY for trusted/local callers (`resolvePerCallMode(ctx, ...)` — remote callers use the configured mode so a remote provider can't force tokenmax spend); every search path stamps evidence fail-soft. `importFromContent` projects frontmatter `aliases:` into `page_aliases` via `normalizeAliasList` + `engine.setPageAliases` so new + changed pages register aliases at ingest. `src/cli.ts` adds the `gbrain search diagnose` dispatch (lazy import) and reconciles the `search` CLI path with the cheap-hybrid op. `src/core/search/telemetry.ts` extends the rollup with the rank-1 base_score drift signal (sum/count + 3 coarse buckets, aggregate not per-query), surfaced via `gbrain search stats`, backed by migration v111's `search_telemetry` columns. Tests: `test/cli-search-dispatch.test.ts`, `test/search/per-call-mode.test.ts`, `test/search/telemetry-rank1.test.ts`, `test/search/title-boost-stage.test.ts`, `test/search/alias-hop.test.ts`, `test/search/evidence.test.ts`, `test/search/searchvector-maxpool.test.ts`, `test/search/pre-migration-failopen.test.ts`.
|
||||
- `src/commands/eval.ts` — `gbrain eval` command: single-run table + A/B config comparison. Sub-subcommand dispatch on `args[0]` routes `gbrain eval export` + `gbrain eval prune` + `gbrain eval replay` into session-capture handlers; bare `gbrain eval --qrels …` fall-through preserves the legacy IR-metrics flow. `gbrain eval cross-modal` is in the dispatch (the user-facing path is the cli.ts no-DB branch — `src/commands/eval.ts:cross-modal` only fires when callers re-enter with an existing engine).
|
||||
- `src/commands/eval-cross-modal.ts` — multi-model quality gate. Three different-provider frontier models score the OUTPUT against the TASK on a 5-dim list. Verdict `pass` (exit 0) / `fail` (exit 1) / `inconclusive` (exit 2; <2/3 model successes). Reuses `src/core/ai/gateway.ts:chat()` so config/auth/aliasing comes from the gateway recipe registry — no parallel provider stack. Self-configures the gateway (`configureGateway(loadConfig() + process.env)`) since the cli.ts dispatch bypasses `connectEngine()`. Default cycles 3 in TTY, 1 in non-TTY (partial cost guardrail). Receipts land at `gbrainPath('eval-receipts')/<slug>-<sha8-of-output>.json`. `--batch <jsonl> [--limit N] [--concurrent N] [--max-usd FLOAT] [--yes]` fans out cross-modal scoring across a LongMemEval-shape JSONL; mutually exclusive with `--task` (fail-fast usage error if both set); filters `kind: "by_type_summary"` rows; pre-flight cost estimate refuses if `> --max-usd` without `--yes` (default cap 5.00 USD). Semaphore-bounded fan-out via inline `runWithLimit<T>(items, limit, fn)` (exported for unit tests): max N questions in-flight × 3 model slots = ceiling of 3N parallel API calls (default `--concurrent 3` → 9). Per-question receipts land in a per-batch tempdir and are deleted at end of run; the summary receipt inlines per-question verdicts as JSON, not file paths. Exit precedence (batch-level policy, NOT inherited from aggregate.ts): ERROR > FAIL > INCONCLUSIVE > PASS. DI seam: `runEvalCrossModal(args, opts?: {runEval?: typeof runEval})` mirrors `runEvalLongMemEval(args, {client?})`; tests pass `opts.runEval` to bypass real LLM calls AND the gateway availability check. Pinned by `test/eval-cross-modal-batch.test.ts`.
|
||||
- `src/commands/eval-cross-modal.ts` — multi-model quality gate. Three different-provider frontier models score the OUTPUT against the TASK on a 5-dim list. Verdict `pass` (exit 0) / `fail` (exit 1) / `inconclusive` (exit 2; <2/3 model successes). Reuses `src/core/ai/gateway.ts:chat()` so config/auth/aliasing comes from the gateway recipe registry — no parallel provider stack. Self-configures the gateway (`configureGateway(loadConfig() + process.env)`) since the cli.ts dispatch bypasses `connectEngine()`. Default cycles 3 in TTY, 1 in non-TTY (partial cost guardrail) via the shared `resolveCycleDefault(explicit, isTty)` in `src/core/eval/cycle-default.ts`; the cost-estimate banner appends `cycleDefaultSuffix(...)` (`for 1 cycle(s) (non-interactive default; --cycles N for more)`) when the value is the silent non-TTY fallback, so the 1-vs-3 difference isn't hidden. Receipts land at `gbrainPath('eval-receipts')/<slug>-<sha8-of-output>.json`. `--batch <jsonl> [--limit N] [--concurrent N] [--max-usd FLOAT] [--yes]` fans out cross-modal scoring across a LongMemEval-shape JSONL; mutually exclusive with `--task` (fail-fast usage error if both set); filters `kind: "by_type_summary"` rows; pre-flight cost estimate refuses if `> --max-usd` without `--yes` (default cap 5.00 USD). Semaphore-bounded fan-out via inline `runWithLimit<T>(items, limit, fn)` (exported for unit tests): max N questions in-flight × 3 model slots = ceiling of 3N parallel API calls (default `--concurrent 3` → 9). Per-question receipts land in a per-batch tempdir and are deleted at end of run; the summary receipt inlines per-question verdicts as JSON, not file paths. Exit precedence (batch-level policy, NOT inherited from aggregate.ts): ERROR > FAIL > INCONCLUSIVE > PASS. DI seam: `runEvalCrossModal(args, opts?: {runEval?: typeof runEval})` mirrors `runEvalLongMemEval(args, {client?})`; tests pass `opts.runEval` to bypass real LLM calls AND the gateway availability check. Pinned by `test/eval-cross-modal-batch.test.ts`.
|
||||
- `src/core/eval/cycle-default.ts` — single source of truth for the eval cycle-count default. Exports `DEFAULT_CYCLES_TTY = 3`, `DEFAULT_CYCLES_NONTTY = 1`, `resolveCycleDefault(explicit, isTty): {cycles, usedNonTtyDefault}`, and `cycleDefaultSuffix(r)` (returns ` (non-interactive default; --cycles N for more)` only when the non-TTY default was applied, else `''`). Consumed by `eval-cross-modal.ts`, `eval-takes-quality.ts` (run + regress), and `takes-quality-eval/runner.ts` (core uses only the constant — library stays TTY-agnostic; the CLI owns the TTY=3 upgrade + banner annotation). `eval-suspected-contradictions.ts` applies the same transparency to its `$5`/`$1` budget default via a `budgetUsdExplicit` flag (the budget is overwritten in-place so explicitness can't be inferred post-hoc). Not shared with `resolveWorkersWithClamp` (different domain, no engine, no dedup). Pinned by `test/eval/cycle-default.test.ts`, `test/eval-suspected-contradictions-budget-default.test.ts`.
|
||||
- `src/core/cross-modal-eval/json-repair.ts` — `parseModelJSON(raw)` named export with a 4-strategy fallback chain (direct parse → fence-strip → trailing-comma + single-quote + embedded-newline repair → regex nuclear option). Adversarial input throws rather than fabricating scores — the aggregator treats a throw as "this model contributed nothing this cycle" so the gate stays correct at >=2/3 successes.
|
||||
- `src/core/cross-modal-eval/aggregate.ts` — pure verdict logic. Pass criterion: `(successes >= 2) AND (every dim mean >= 7) AND (every dim min across models >= 5)`. Inconclusive when <2/3 models returned parseable scores (regression guard for the v1 `Object.values({}).every(...) === true` empty-array PASS bug).
|
||||
- `src/core/cross-modal-eval/runner.ts` — orchestrator. Each cycle runs `Promise.allSettled([gwChat(slotA), gwChat(slotB), gwChat(slotC)])` (bare allSettled, no rate-leases for the CLI path). Stops early on PASS or INCONCLUSIVE; runs up to 3 cycles. Default slots: `openai:gpt-4o` / `anthropic:claude-opus-4-7` / `google:gemini-1.5-pro`. `estimateCost()` exports a small per-model pricing table (drifts; refresh alongside model-family bumps).
|
||||
@@ -230,7 +231,7 @@ per-release `**vX.Y.Z:**` narration — CI enforces this
|
||||
- `src/core/minions/attachments.ts` — Attachment validation (path traversal, null byte, oversize, base64, duplicate detection).
|
||||
- `src/commands/agent.ts` — `gbrain agent run <prompt> [flags]` CLI. Submits `subagent` (or N children + 1 aggregator) under `{allowProtectedSubmit: true}`. Single-entry `--fanout-manifest` short-circuits. Children get `on_child_fail: 'continue'` + `max_stalled: 3`. `--follow` is the default on TTY; streams logs + polls `waitForCompletion` in parallel. Ctrl-C detaches, does not cancel.
|
||||
- `src/commands/agent-logs.ts` — `gbrain agent logs <job> [--follow] [--since]`. Merges JSONL heartbeat audit + `subagent_messages` into a chronological timeline. `parseSince` accepts ISO-8601 or relative (`5m`, `1h`, `2d`). Transcript tail renders only for terminal jobs.
|
||||
- `src/commands/jobs.ts` — `gbrain jobs` CLI subcommands + `gbrain jobs work` daemon. `case 'work'` wraps `worker.start()` in try/finally and owns engine lifecycle — calls `engine.disconnect()` on shutdown with loud error logging (the worker must not disconnect an engine it doesn't own; pool slots free immediately on shutdown rather than waiting for TCP keepalive). `jobs submit` surfaces the full `MinionJobInput` retry/backoff/timeout/idempotency surface as flags: `--max-stalled`, `--backoff-type fixed|exponential`, `--backoff-delay`, `--backoff-jitter`, `--timeout-ms`, `--idempotency-key`. `jobs smoke --sigkill-rescue` is the SIGKILL-rescue regression guard. `registerBuiltinHandlers` always registers `subagent` + `subagent_aggregator` (no env flag — `ANTHROPIC_API_KEY` is the cost gate, trust is via `PROTECTED_JOB_NAMES`) and loads `GBRAIN_PLUGIN_PATH` plugins at startup with a loud per-plugin line; `shell` handler still gated by `GBRAIN_ALLOW_SHELL_JOBS=1` (RCE surface). The `autopilot-cycle` handler forwards `job.data.phases` to `runCycle`, validated against `ALL_PHASES` from `src/core/cycle.ts` (invalid names filtered; empty/missing falls back to the default cycle). The `sync` handler resolves `sourceId` at entry from `sources.local_path` (mirrors `cycle.ts:480`) so multi-source brains read the per-source `last_commit` anchor; concurrency routes through `autoConcurrency()` in `src/core/sync-concurrency.ts` (PGLite stays serial); `noEmbed` default is `true`. `gbrain jobs supervisor status` at `jobs.ts:803-826` consumes `summarizeCrashes()` from `src/core/minions/handlers/supervisor-audit.ts` for parity with `gbrain doctor`: JSON adds `crashes_by_cause: {runtime_error, oom_or_external_kill, unknown, legacy}` + `clean_exits_24h`; human output gains per-cause + clean-exits lines. Pinned by 4 source-grep wiring assertions in `test/doctor.test.ts` requiring `crashes_by_cause` + `clean_exits_24h=` in both `doctor.ts` and `jobs.ts`.
|
||||
- `src/commands/jobs.ts` — `gbrain jobs` CLI subcommands + `gbrain jobs work` daemon. `case 'work'` wraps `worker.start()` in try/finally and owns engine lifecycle — calls `engine.disconnect()` on shutdown with loud error logging (the worker must not disconnect an engine it doesn't own; pool slots free immediately on shutdown rather than waiting for TCP keepalive). `jobs submit` surfaces the full `MinionJobInput` retry/backoff/timeout/idempotency surface as flags: `--max-stalled`, `--backoff-type fixed|exponential`, `--backoff-delay`, `--backoff-jitter`, `--timeout-ms`, `--idempotency-key`. `jobs smoke --sigkill-rescue` is the SIGKILL-rescue regression guard. `registerBuiltinHandlers` always registers `subagent` + `subagent_aggregator` (no env flag — `ANTHROPIC_API_KEY` is the cost gate, trust is via `PROTECTED_JOB_NAMES`) and loads `GBRAIN_PLUGIN_PATH` plugins at startup with a loud per-plugin line; `shell` handler still gated by `GBRAIN_ALLOW_SHELL_JOBS=1` (RCE surface). The `autopilot-cycle` handler forwards `job.data.phases` to `runCycle`, validated against `ALL_PHASES` from `src/core/cycle.ts` (invalid names filtered; empty/missing falls back to the default cycle). The `sync` handler resolves `sourceId` at entry from `sources.local_path` (mirrors `cycle.ts:480`) so multi-source brains read the per-source `last_commit` anchor; concurrency routes through `autoConcurrency()` in `src/core/sync-concurrency.ts` (PGLite stays serial); `noEmbed` default is `true`. `gbrain jobs supervisor status` at `jobs.ts:803-826` consumes `summarizeCrashes()` from `src/core/minions/handlers/supervisor-audit.ts` for parity with `gbrain doctor`: JSON adds `crashes_by_cause: {runtime_error, oom_or_external_kill, unknown, legacy}` + `clean_exits_24h`; human output gains per-cause + clean-exits lines. Pinned by 4 source-grep wiring assertions in `test/doctor.test.ts` requiring `crashes_by_cause` + `clean_exits_24h=` in both `doctor.ts` and `jobs.ts`. `gbrain jobs watch` decouples its two output axes: `--json` picks FORMAT (human default, never gated on isTTY), `--follow` picks LOOP (default `isTTY && !json`). Non-TTY with no flags prints ONE human snapshot then exits (clean for subagent/pipe/cron); `--follow` opts into a continuous stream (human plain per tick, or JSONL with `--json`); a TTY with no flags keeps the live ANSI dashboard. Resolution is the pure `resolveWatchMode(opts, isTTY): {json, follow, useAnsiDashboard}` in `src/commands/jobs-watch.ts`; the dispatch wires `--follow`. Pinned by `test/jobs-watch-mode.test.ts` (format×loop matrix incl. the TTY+`--json`-one-shot case) + `test/e2e/non-tty-output.serial.test.ts` (the `cmd </dev/null` non-empty-stdout contract).
|
||||
- `src/commands/features.ts` — `gbrain features --json --auto-fix`: usage scan + feature adoption salesman.
|
||||
- `src/commands/autopilot.ts` — `gbrain autopilot --install`: self-maintaining brain daemon (sync+extract+embed). Consumes `detectTini()` from `src/core/minions/spawn-helpers.ts`, resolved once at startup. Composes a `ChildWorkerSupervisor` instance for spawn-and-respawn (no inline `crashCount`/`startWorker`/`child.on('exit')`); `--max-rss 2048` and `maxCrashes: 5` preserved. `onMaxCrashesExceeded` routes through autopilot's own `shutdown('max_crashes')` so the autopilot lockfile gets cleaned up. `shutdown()` drains via `childSupervisor.killChild('SIGTERM')` + `awaitChildExit(35_000)`. Pinned by `test/autopilot-supervisor-wiring.test.ts` (6 static-shape guards: composes ChildWorkerSupervisor not legacy names, `--max-rss 2048` in argv, `maxCrashes: 5` literal, shutdown-via-callback, no workerProc reference).
|
||||
- `src/mcp/server.ts` — MCP stdio server (generated from operations). Tool-call handler delegates to `dispatchToolCall` from `src/mcp/dispatch.ts` so stdio + HTTP transports share one validation, context-build, and error-format path. Stdin `'end'` / `'close'` shutdown hooks are skipped when `process.env.MCP_STDIO === '1'` — gateway-piped stdio MCP wrappers (OpenClaw's `bundle-mcp`) pipe the handshake then close their stdin half, which would otherwise kill the server before the first tool call; signal handlers (SIGTERM/SIGINT/SIGHUP) + the parent-process watchdog still cover legitimate disconnects. `src/commands/serve.ts` exposes `ServeOptions.mcpStdio?: boolean` as a test seam so the guard is exercisable without process.env mutation. Pinned by `test/serve-stdio-lifecycle.test.ts`.
|
||||
|
||||
+1
-1
@@ -143,5 +143,5 @@
|
||||
"bun": ">=1.3.10"
|
||||
},
|
||||
"license": "MIT",
|
||||
"version": "0.42.14.0"
|
||||
"version": "0.42.15.0"
|
||||
}
|
||||
|
||||
@@ -24,6 +24,7 @@ import { createHash } from 'crypto';
|
||||
import { gbrainPath, loadConfig } from '../core/config.ts';
|
||||
import { configureGateway, isAvailable } from '../core/ai/gateway.ts';
|
||||
import { runWithLimit } from '../core/worker-pool.ts';
|
||||
import { resolveCycleDefault, cycleDefaultSuffix } from '../core/eval/cycle-default.ts';
|
||||
import {
|
||||
DEFAULT_DIMENSIONS,
|
||||
DEFAULT_SLOTS,
|
||||
@@ -342,7 +343,10 @@ export async function runEvalCrossModal(args: string[], opts: RunCrossModalOpts
|
||||
}
|
||||
|
||||
const slug = parsed.slug ?? inferSlugFromOutputPath(parsed.output);
|
||||
const cycles = parsed.cycles ?? (isTTY() ? 3 : 1);
|
||||
// #1784: resolve the cycle default once; annotate the cost banner below when
|
||||
// it's the silent non-TTY fallback so the 1-vs-3 difference isn't a surprise.
|
||||
const cycleDef = resolveCycleDefault(parsed.cycles, isTTY());
|
||||
const cycles = cycleDef.cycles;
|
||||
const dimensions = parsed.dimensions ?? DEFAULT_DIMENSIONS;
|
||||
const receiptDir = parsed.receiptDir ?? gbrainPath('eval-receipts');
|
||||
const maxTokens = parsed.maxTokens ?? 4000;
|
||||
@@ -372,7 +376,7 @@ export async function runEvalCrossModal(args: string[], opts: RunCrossModalOpts
|
||||
const cost = estimateCost(slots, cycles, maxTokens);
|
||||
process.stderr.write(
|
||||
`[eval cross-modal] estimated cost: ~$${cost.perCycleUSD.toFixed(2)}/cycle, ` +
|
||||
`~$${cost.perRunMaxUSD.toFixed(2)} max for ${cycles} cycle(s).\n`,
|
||||
`~$${cost.perRunMaxUSD.toFixed(2)} max for ${cycles} cycle(s)${cycleDefaultSuffix(cycleDef)}.\n`,
|
||||
);
|
||||
for (const note of cost.notes) {
|
||||
process.stderr.write(`[eval cross-modal] note: ${note}\n`);
|
||||
|
||||
@@ -62,6 +62,12 @@ interface ParsedFlags {
|
||||
judge?: string;
|
||||
limit?: number;
|
||||
budgetUsd: number;
|
||||
/**
|
||||
* #1784: true when --budget-usd was passed explicitly. The TTY-derived
|
||||
* default ($5 TTY / $1 non-TTY) is overwritten in-place, so explicitness
|
||||
* can't be inferred post-hoc — track it here to annotate the banner.
|
||||
*/
|
||||
budgetUsdExplicit: boolean;
|
||||
output?: string;
|
||||
maxPairChars: number;
|
||||
sampling: 'deterministic' | 'score-first';
|
||||
@@ -78,7 +84,7 @@ interface ParsedFlags {
|
||||
help: boolean;
|
||||
}
|
||||
|
||||
function parseFlags(args: string[]): ParsedFlags {
|
||||
export function parseFlags(args: string[]): ParsedFlags {
|
||||
// Sub-subcommand: first positional that doesn't start with --
|
||||
let sub: 'run' | 'trend' | 'review' = 'run';
|
||||
const rest: string[] = [];
|
||||
@@ -99,6 +105,7 @@ function parseFlags(args: string[]): ParsedFlags {
|
||||
// judge intentionally undefined here — resolved in runRun via resolveModel
|
||||
// so config keys + tier defaults govern. CLI --judge flag wins when set.
|
||||
budgetUsd: isTty ? 5 : 1,
|
||||
budgetUsdExplicit: false,
|
||||
maxPairChars: 1500,
|
||||
sampling: 'deterministic',
|
||||
noCache: false,
|
||||
@@ -122,7 +129,7 @@ function parseFlags(args: string[]): ParsedFlags {
|
||||
else if (arg === '--top-k') f.topK = Number.parseInt(next(), 10);
|
||||
else if (arg === '--judge') f.judge = next();
|
||||
else if (arg === '--limit') f.limit = Number.parseInt(next(), 10);
|
||||
else if (arg === '--budget-usd') f.budgetUsd = Number.parseFloat(next());
|
||||
else if (arg === '--budget-usd') { f.budgetUsd = Number.parseFloat(next()); f.budgetUsdExplicit = true; }
|
||||
else if (arg === '--output') f.output = next();
|
||||
else if (arg === '--max-pair-chars') f.maxPairChars = Number.parseInt(next(), 10);
|
||||
else if (arg === '--sampling') {
|
||||
@@ -264,8 +271,13 @@ async function runRun(engine: BrainEngine, f: ParsedFlags): Promise<void> {
|
||||
fallback: 'anthropic:claude-haiku-4-5',
|
||||
});
|
||||
|
||||
// #1784: annotate the budget when it's the silent non-TTY default ($1) so the
|
||||
// 5-vs-1 difference isn't a surprise to pipe / cron / subagent callers.
|
||||
const budgetSuffix = (process.stdout.isTTY !== true && !f.budgetUsdExplicit)
|
||||
? ' (non-interactive default; --budget-usd N to raise)'
|
||||
: '';
|
||||
console.error(
|
||||
`Contradiction probe: ${queries.length} queries, top-${f.topK}, judge=${judgeModel}, budget=$${f.budgetUsd.toFixed(2)}.`,
|
||||
`Contradiction probe: ${queries.length} queries, top-${f.topK}, judge=${judgeModel}, budget=$${f.budgetUsd.toFixed(2)}${budgetSuffix}.`,
|
||||
);
|
||||
|
||||
// v0.34 / Lane C: cost-estimate prompt — TTY-only Ctrl-C window before
|
||||
|
||||
@@ -23,6 +23,7 @@ import type { BrainEngine } from '../core/engine.ts';
|
||||
import { configureGateway } from '../core/ai/gateway.ts';
|
||||
import { loadConfig } from '../core/config.ts';
|
||||
import { runEval, DEFAULT_MODEL_PANEL } from '../core/takes-quality-eval/runner.ts';
|
||||
import { resolveCycleDefault, cycleDefaultSuffix } from '../core/eval/cycle-default.ts';
|
||||
import { writeReceipt } from '../core/takes-quality-eval/receipt-write.ts';
|
||||
import { loadReceiptFromDisk } from '../core/takes-quality-eval/replay.ts';
|
||||
import { compareReceipts } from '../core/takes-quality-eval/regress.ts';
|
||||
@@ -138,7 +139,11 @@ export async function runEvalTakesQuality(engine: BrainEngine, args: string[]):
|
||||
|
||||
if (subcmd === 'run') {
|
||||
const limit = parseIntFlag(argv, '--limit', 100);
|
||||
const cycles = parseIntFlag(argv, '--cycles', process.stdout.isTTY ? 3 : 1);
|
||||
// #1784: keep parseIntFlag for value validation; resolveCycleDefault drives
|
||||
// the banner annotation when the value is the silent non-TTY fallback.
|
||||
const cycleDef = resolveCycleDefault(undefined, process.stdout.isTTY === true);
|
||||
const cycles = parseIntFlag(argv, '--cycles', cycleDef.cycles);
|
||||
const cyclesSuffix = getFlag(argv, '--cycles') === undefined ? cycleDefaultSuffix(cycleDef) : '';
|
||||
const budgetStr = getFlag(argv, '--budget-usd');
|
||||
const budgetUsd = budgetStr === undefined ? null : Number(budgetStr);
|
||||
if (budgetStr !== undefined && !Number.isFinite(budgetUsd)) {
|
||||
@@ -153,7 +158,7 @@ export async function runEvalTakesQuality(engine: BrainEngine, args: string[]):
|
||||
if (!json) {
|
||||
process.stderr.write(
|
||||
`[eval takes-quality] sampling ${limit} take(s) from ${source}; ` +
|
||||
`panel: ${models.join(', ')}; cycles: ${cycles}` +
|
||||
`panel: ${models.join(', ')}; cycles: ${cycles}${cyclesSuffix}` +
|
||||
(budgetUsd === null ? '' : `; budget: $${budgetUsd.toFixed(2)}`) +
|
||||
'\n',
|
||||
);
|
||||
@@ -208,11 +213,14 @@ export async function runEvalTakesQuality(engine: BrainEngine, args: string[]):
|
||||
process.exit(2);
|
||||
}
|
||||
const limit = parseIntFlag(argv, '--limit', 100);
|
||||
const cycles = parseIntFlag(argv, '--cycles', process.stdout.isTTY ? 3 : 1);
|
||||
// #1784: same annotation treatment as the run subcommand.
|
||||
const cycleDef = resolveCycleDefault(undefined, process.stdout.isTTY === true);
|
||||
const cycles = parseIntFlag(argv, '--cycles', cycleDef.cycles);
|
||||
const cyclesSuffix = getFlag(argv, '--cycles') === undefined ? cycleDefaultSuffix(cycleDef) : '';
|
||||
|
||||
const prior = loadReceiptFromDisk(againstPath);
|
||||
if (!json) {
|
||||
process.stderr.write(`[eval takes-quality regress] running fresh eval to compare against ${againstPath}\n`);
|
||||
process.stderr.write(`[eval takes-quality regress] running fresh eval (cycles: ${cycles}${cyclesSuffix}) to compare against ${againstPath}\n`);
|
||||
}
|
||||
const result = await runEval(engine, {
|
||||
limit,
|
||||
|
||||
+66
-19
@@ -9,15 +9,27 @@
|
||||
* 60fps; 1s keeps the SQL load nominal even when multiple watch sessions
|
||||
* point at the same brain).
|
||||
*
|
||||
* Rendering: manual ANSI cursor management (no TUI dep). Clears the
|
||||
* screen on first render, then redraws from the top each tick using
|
||||
* cursor-home + erase-down. On non-TTY (cron / wrapped redirect),
|
||||
* falls through to one snapshot line per tick in `--progress-json`
|
||||
* shape so wrappers can parse.
|
||||
* Two independent axes (v0.42.11.0, #1784 — decoupled from `isTTY`):
|
||||
* - FORMAT (what data prints): human by default, JSON only when `--json` is
|
||||
* passed. NEVER gated on isTTY.
|
||||
* - LOOP (cadence): `--follow` streams continuously; default is `isTTY` —
|
||||
* continuous live dashboard in a terminal, ONE snapshot then exit when
|
||||
* non-TTY (pipe / cron / subagent). Identical data either way, so defaulting
|
||||
* the loop from isTTY is a cosmetic UX call, not a data gate.
|
||||
*
|
||||
* Quit: Ctrl-C (SIGINT), 'q', or stdin close — the watcher restores the
|
||||
* cursor + clears its own region on shutdown so the terminal isn't left
|
||||
* with a half-rendered dashboard.
|
||||
* Resulting matrix:
|
||||
* TTY, no flags → live ANSI dashboard (cursor-managed, loops)
|
||||
* non-TTY, no flags → ONE human plain-text snapshot, exit
|
||||
* any + --json → JSON snapshot (one-shot, or JSONL stream w/ --follow)
|
||||
* any + --follow → continuous (human plain per tick, or JSONL w/ --json)
|
||||
*
|
||||
* Rendering: manual ANSI cursor management (no TUI dep) for the live dashboard
|
||||
* only. Clears the screen on first render, then redraws from the top each tick
|
||||
* using cursor-home + erase-down.
|
||||
*
|
||||
* Quit: in the live dashboard, Ctrl-C (SIGINT) or 'q' restores the cursor +
|
||||
* clears its region. Non-TTY one-shots (nothing to quit); a non-TTY `--follow`
|
||||
* stream runs until the process is killed.
|
||||
*
|
||||
* No SSE consumer in v0.41 — local polling against the brain engine is
|
||||
* the foundation. SSE wiring through `serve-http.ts` is filed as a
|
||||
@@ -188,32 +200,63 @@ export async function readSnapshot(engine: BrainEngine): Promise<WatchSnapshot>
|
||||
export interface WatchOptions {
|
||||
/** Refresh interval. Default 1000ms. */
|
||||
refreshMs?: number;
|
||||
/** Stream JSON snapshots to stdout (non-TTY mode). */
|
||||
/** FORMAT axis: emit JSON instead of human text. Default human. Explicit only. */
|
||||
json?: boolean;
|
||||
/**
|
||||
* LOOP axis: stream continuously. Default = `process.stdout.isTTY` — live
|
||||
* dashboard in a terminal, one snapshot then exit when non-TTY. Pass `true`
|
||||
* to force a continuous stream even off-TTY (cron tail / log pipe).
|
||||
*/
|
||||
follow?: boolean;
|
||||
}
|
||||
|
||||
export interface WatchMode {
|
||||
/** FORMAT: emit JSON instead of human text. */
|
||||
json: boolean;
|
||||
/** LOOP: continuous stream vs one-shot. */
|
||||
follow: boolean;
|
||||
/** Live cursor-managed colored dashboard (TTY + human + looping only). */
|
||||
useAnsiDashboard: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* Main entrypoint for `gbrain jobs watch`. Runs until SIGINT or 'q'
|
||||
* keypress (on TTY). Non-TTY mode loops with --progress-json output.
|
||||
* Pure resolver for the format × loop matrix (extracted for unit-testing the
|
||||
* exact TTY-gating contract this command fixes, #1784). The data printed never
|
||||
* depends on isTTY; only the loop cadence + ANSI cursor management do.
|
||||
*
|
||||
* follow default = `isTTY && !json`: a terminal human view is the live
|
||||
* dashboard (loops), but `--json` (any) and non-TTY both one-shot unless the
|
||||
* caller passes `--follow` explicitly. Matches the file-header matrix.
|
||||
*/
|
||||
export function resolveWatchMode(opts: WatchOptions, isTTY: boolean): WatchMode {
|
||||
const json = opts.json === true; // FORMAT: explicit only — never from isTTY.
|
||||
const follow = opts.follow ?? (isTTY && !json);
|
||||
const useAnsiDashboard = isTTY && !json && follow;
|
||||
return { json, follow, useAnsiDashboard };
|
||||
}
|
||||
|
||||
/**
|
||||
* Main entrypoint for `gbrain jobs watch`. See the file header for the
|
||||
* format (`--json`) × loop (`--follow`) matrix. The data printed never depends
|
||||
* on isTTY; only the loop cadence and the ANSI cursor management do.
|
||||
*/
|
||||
export async function runWatch(engine: BrainEngine, opts: WatchOptions = {}): Promise<void> {
|
||||
const refreshMs = opts.refreshMs ?? 1000;
|
||||
const isTTY = process.stdout.isTTY === true;
|
||||
const json = opts.json || !isTTY;
|
||||
const { json, follow, useAnsiDashboard } = resolveWatchMode(opts, process.stdout.isTTY === true);
|
||||
|
||||
let stopped = false;
|
||||
const stop = () => {
|
||||
stopped = true;
|
||||
};
|
||||
|
||||
if (isTTY && !json) {
|
||||
if (useAnsiDashboard) {
|
||||
process.stdout.write(ANSI.cursorHide + ANSI.clear + ANSI.cursorHome);
|
||||
process.on('SIGINT', () => {
|
||||
process.stdout.write(ANSI.cursorShow + ANSI.clear + ANSI.cursorHome);
|
||||
stop();
|
||||
process.exit(0);
|
||||
});
|
||||
// Read stdin for 'q' keypress.
|
||||
// Read stdin for 'q' keypress (terminal-only affordance).
|
||||
if (process.stdin.isTTY && process.stdin.setRawMode) {
|
||||
process.stdin.setRawMode(true);
|
||||
process.stdin.resume();
|
||||
@@ -227,15 +270,19 @@ export async function runWatch(engine: BrainEngine, opts: WatchOptions = {}): Pr
|
||||
}
|
||||
}
|
||||
|
||||
while (!stopped) {
|
||||
do {
|
||||
const snap = await readSnapshot(engine);
|
||||
if (json) {
|
||||
process.stdout.write(JSON.stringify({ event: 'jobs.watch.snapshot', ...snap }) + '\n');
|
||||
} else {
|
||||
// TTY: clear + cursor-home + render.
|
||||
} else if (useAnsiDashboard) {
|
||||
// Live dashboard: clear + cursor-home + colored render.
|
||||
process.stdout.write(ANSI.cursorHome + ANSI.eraseDown);
|
||||
process.stdout.write(renderSnapshot(snap, { useAnsi: true }));
|
||||
} else {
|
||||
// Non-TTY (or --follow without a terminal): plain human snapshot, no ANSI.
|
||||
process.stdout.write(renderSnapshot(snap, { useAnsi: false }) + '\n');
|
||||
}
|
||||
if (!follow) break; // one-shot: render once, exit.
|
||||
await new Promise(r => setTimeout(r, refreshMs));
|
||||
}
|
||||
} while (!stopped);
|
||||
}
|
||||
|
||||
@@ -1098,14 +1098,17 @@ HANDLER TYPES (built in)
|
||||
}
|
||||
|
||||
case 'watch': {
|
||||
// v0.41 D2 — live TTY dashboard (or JSON snapshots on non-TTY).
|
||||
// v0.41 D2 — live dashboard; v0.42.11.0 (#1784) decoupled output from TTY.
|
||||
// Flags: --json (FORMAT, human default), --follow (LOOP, default=isTTY so
|
||||
// non-TTY one-shots), --refresh-ms=N. Non-TTY no-flag → one human snapshot.
|
||||
try { await queue.ensureSchema(); }
|
||||
catch (e) { console.error(e instanceof Error ? e.message : String(e)); process.exit(1); }
|
||||
const { runWatch } = await import('./jobs-watch.ts');
|
||||
const refreshArg = args.find(a => a.startsWith('--refresh-ms='));
|
||||
const refreshMs = refreshArg ? parseInt(refreshArg.split('=')[1] ?? '1000', 10) : 1000;
|
||||
const json = hasFlag(args, '--json');
|
||||
await runWatch(engine, { refreshMs, json });
|
||||
const follow = hasFlag(args, '--follow') ? true : undefined; // undefined → default to isTTY
|
||||
await runWatch(engine, { refreshMs, json, follow });
|
||||
break;
|
||||
}
|
||||
|
||||
|
||||
@@ -376,6 +376,45 @@ export async function runReindexCode(
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* v0.42.11.0 (#1784) — what to print when the cost gate refuses to spend
|
||||
* non-interactively without `--yes`. The REFUSAL (exit 2, no spend) is the
|
||||
* guardrail and is correct; the FORMAT is a separate axis. Pre-#1784 this path
|
||||
* always emitted a JSON envelope even without `--json`, violating the repo's
|
||||
* "human by default" convention. Now: JSON only when `--json` is explicit;
|
||||
* otherwise a human refusal on stderr. Pure + exported so it's unit-testable
|
||||
* without a brain or a real cost preview.
|
||||
*/
|
||||
export interface CostRefusal {
|
||||
stdout?: string;
|
||||
stderr?: string;
|
||||
}
|
||||
export function buildCostRefusal(opts: {
|
||||
json: boolean;
|
||||
previewMsg: string;
|
||||
preview: unknown;
|
||||
costUsd: number;
|
||||
model: string;
|
||||
}): CostRefusal {
|
||||
if (opts.json) {
|
||||
const envelope = serializeError(errorFor({
|
||||
class: 'ConfirmationRequired',
|
||||
code: 'cost_preview_requires_yes',
|
||||
message: opts.previewMsg,
|
||||
hint: 'Pass --yes to proceed, or --dry-run to see the preview and exit 0.',
|
||||
}));
|
||||
return {
|
||||
stdout: JSON.stringify({ error: envelope, preview: opts.preview, costUsd: opts.costUsd, model: opts.model }),
|
||||
};
|
||||
}
|
||||
return {
|
||||
stderr:
|
||||
`${opts.previewMsg}\n` +
|
||||
'Refusing to re-embed non-interactively without confirmation. ' +
|
||||
'Pass --yes to proceed, or --dry-run for the preview (exit 0).',
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* CLI entrypoint. Parses argv, wires cost-preview gate + JSON/TTY branching,
|
||||
* delegates to runReindexCode. Exit codes: 0 on success/dry-run, 2 on
|
||||
@@ -456,13 +495,11 @@ export async function runReindexCodeCli(engine: BrainEngine, args: string[]): Pr
|
||||
if (!yes) {
|
||||
const isTTY = Boolean(process.stdout.isTTY) && Boolean(process.stdin.isTTY);
|
||||
if (!isTTY || json) {
|
||||
const envelope = serializeError(errorFor({
|
||||
class: 'ConfirmationRequired',
|
||||
code: 'cost_preview_requires_yes',
|
||||
message: previewMsg,
|
||||
hint: 'Pass --yes to proceed, or --dry-run to see the preview and exit 0.',
|
||||
}));
|
||||
console.log(JSON.stringify({ error: envelope, preview, costUsd, model: getEmbeddingModelName() }));
|
||||
// Guardrail unchanged: refuse + exit 2, no spend. Only the FORMAT splits
|
||||
// on --json now (human refusal on stderr otherwise) — #1784.
|
||||
const refusal = buildCostRefusal({ json, previewMsg, preview, costUsd, model: getEmbeddingModelName() });
|
||||
if (refusal.stdout) console.log(refusal.stdout);
|
||||
if (refusal.stderr) console.error(refusal.stderr);
|
||||
process.exit(2);
|
||||
}
|
||||
console.log(previewMsg);
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
/**
|
||||
* v0.42.11.0 (#1784) — single source of truth for the eval cycle-count default.
|
||||
*
|
||||
* Several eval commands (`eval cross-modal`, `eval takes-quality run/regress`)
|
||||
* and the takes-quality runner core resolved their cycle default as
|
||||
* `process.stdout.isTTY ? 3 : 1`. The non-TTY value of 1 is a deliberate
|
||||
* cost-conservative default (each cycle calls frontier models), but the split
|
||||
* was SILENT — a subagent / pipe / cron run got 1 with nothing explaining why,
|
||||
* which is the surprise issue #1784 names.
|
||||
*
|
||||
* The fix is NOT a new stderr notice line — those commands already print the
|
||||
* resolved cycle count in their existing banner. Instead, callers ANNOTATE that
|
||||
* existing banner via `cycleDefaultSuffix` when the value came from the non-TTY
|
||||
* default, so the operator sees `cycles: 1 (non-interactive default; --cycles N
|
||||
* for more)` instead of a bare `cycles: 1`.
|
||||
*
|
||||
* The runner CORE (`takes-quality-eval/runner.ts`) consumes only
|
||||
* `DEFAULT_CYCLES_NONTTY` — library code stays TTY-agnostic; the CLI layer owns
|
||||
* the TTY=3 upgrade + the banner annotation.
|
||||
*
|
||||
* Deliberately NOT shared with `resolveWorkersWithClamp` (sync-concurrency.ts):
|
||||
* different domain, no engine, no per-process dedup. Sharing would be premature.
|
||||
*/
|
||||
|
||||
/** Interactive (TTY) default: deeper eval, more model calls. */
|
||||
export const DEFAULT_CYCLES_TTY = 3;
|
||||
|
||||
/** Non-interactive (pipe / cron / subagent) default: cost-conservative. */
|
||||
export const DEFAULT_CYCLES_NONTTY = 1;
|
||||
|
||||
export interface CycleResolution {
|
||||
/** The effective cycle count. */
|
||||
cycles: number;
|
||||
/**
|
||||
* True ONLY when no explicit value was given AND we are non-TTY — i.e. the
|
||||
* caller fell through to `DEFAULT_CYCLES_NONTTY`. This is the case worth
|
||||
* annotating in the banner so the 1-vs-3 difference isn't silent.
|
||||
*/
|
||||
usedNonTtyDefault: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the cycle count from an explicit value + TTY-ness.
|
||||
*
|
||||
* explicit set → {explicit, false} (caller asked; no annotation)
|
||||
* undefined+TTY → {3, false} (interactive default; visible live)
|
||||
* undefined+!TTY→ {1, true} (cost-safe default; ANNOTATE)
|
||||
*/
|
||||
export function resolveCycleDefault(
|
||||
explicit: number | undefined,
|
||||
isTty: boolean,
|
||||
): CycleResolution {
|
||||
if (explicit !== undefined) return { cycles: explicit, usedNonTtyDefault: false };
|
||||
if (isTty) return { cycles: DEFAULT_CYCLES_TTY, usedNonTtyDefault: false };
|
||||
return { cycles: DEFAULT_CYCLES_NONTTY, usedNonTtyDefault: true };
|
||||
}
|
||||
|
||||
/**
|
||||
* Banner suffix to append to an EXISTING stderr line that already prints the
|
||||
* cycle count. Empty string unless the non-TTY default was applied, so the
|
||||
* common (TTY or explicit) cases get no extra text.
|
||||
*/
|
||||
export function cycleDefaultSuffix(r: CycleResolution): string {
|
||||
return r.usedNonTtyDefault ? ' (non-interactive default; --cycles N for more)' : '';
|
||||
}
|
||||
@@ -31,6 +31,7 @@ import {
|
||||
} from './receipt-name.ts';
|
||||
import type { TakesQualityReceipt } from './receipt.ts';
|
||||
import { estimateCost, getPricing, PricingNotFoundError } from './pricing.ts';
|
||||
import { DEFAULT_CYCLES_NONTTY } from '../eval/cycle-default.ts';
|
||||
|
||||
export const DEFAULT_MODEL_PANEL = [
|
||||
'openai:gpt-4o',
|
||||
@@ -145,7 +146,10 @@ async function callOneModel(
|
||||
|
||||
export async function runEval(engine: BrainEngine, opts: RunOpts = {}): Promise<RunResult> {
|
||||
const limit = opts.limit ?? 100;
|
||||
const cycles = opts.cycles ?? (process.stdout.isTTY ? 3 : 1);
|
||||
// Library core stays TTY-agnostic (#1784): default to the cost-conservative
|
||||
// value. The CLI layer (eval-takes-quality.ts) owns the TTY=3 upgrade + the
|
||||
// banner annotation; it always passes an explicit `cycles` down.
|
||||
const cycles = opts.cycles ?? DEFAULT_CYCLES_NONTTY;
|
||||
const models = opts.models ?? DEFAULT_MODEL_PANEL;
|
||||
const budgetUsd = opts.budgetUsd ?? null;
|
||||
const source = opts.source ?? 'db';
|
||||
|
||||
@@ -0,0 +1,61 @@
|
||||
/**
|
||||
* v0.42.11.0 (#1784) — non-TTY output contract E2E (the `cmd </dev/null` smoke
|
||||
* test the issue asked for).
|
||||
*
|
||||
* `jobs watch` is the command whose PRIMARY output was gated on isTTY: non-TTY
|
||||
* callers (subagent / pipe / cron) got JSON-only with no way to a human view,
|
||||
* and the loop ran forever. The fix decouples FORMAT (--json) from LOOP
|
||||
* (--follow, default=isTTY), so a non-TTY run prints ONE human snapshot and
|
||||
* exits.
|
||||
*
|
||||
* This drives the REAL binary in a non-TTY child (stdin ignored, stdout piped →
|
||||
* `process.stdout.isTTY` is false), so it exercises the exact path subagents and
|
||||
* pipes hit. Hermetic PGLite brain in a temp HOME; no Postgres, no API keys.
|
||||
*
|
||||
* Serial because it spawns subprocesses against a temp brain and PGLite is a
|
||||
* single-writer engine.
|
||||
*/
|
||||
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
|
||||
import { execFileSync } from 'child_process';
|
||||
import { mkdtempSync, rmSync } from 'fs';
|
||||
import { tmpdir } from 'os';
|
||||
import { join } from 'path';
|
||||
|
||||
describe('non-TTY output contract: jobs watch (#1784)', () => {
|
||||
let home: string;
|
||||
let env: NodeJS.ProcessEnv;
|
||||
|
||||
beforeAll(() => {
|
||||
home = mkdtempSync(join(tmpdir(), 'gbrain-nontty-e2e-'));
|
||||
env = { ...process.env, GBRAIN_HOME: home };
|
||||
// Fresh local PGLite brain so `jobs watch` has an engine to read.
|
||||
execFileSync('bun', ['run', 'src/cli.ts', 'init', '--pglite', '--no-embedding', '--non-interactive'], {
|
||||
cwd: process.cwd(), env, stdio: 'ignore',
|
||||
});
|
||||
}, 60_000);
|
||||
|
||||
afterAll(() => {
|
||||
try { rmSync(home, { recursive: true, force: true }); } catch { /* best-effort */ }
|
||||
});
|
||||
|
||||
test('non-TTY, no flags → ONE human snapshot, non-empty, not JSON, exit 0', () => {
|
||||
// stdin 'ignore' + stdout 'pipe' makes the child non-TTY on both ends.
|
||||
const out = execFileSync('bun', ['run', 'src/cli.ts', 'jobs', 'watch'], {
|
||||
cwd: process.cwd(), env, stdio: ['ignore', 'pipe', 'ignore'], encoding: 'utf-8', timeout: 30_000,
|
||||
});
|
||||
expect(out.length).toBeGreaterThan(0); // the bug was (no output)
|
||||
expect(out).toContain('gbrain jobs watch'); // human renderer header
|
||||
expect(out).toContain('Queue'); // human panel
|
||||
expect(out.trimStart().startsWith('{')).toBe(false); // NOT JSON by default
|
||||
}, 35_000);
|
||||
|
||||
test('non-TTY, --json → ONE JSON snapshot, parses, exit 0', () => {
|
||||
const out = execFileSync('bun', ['run', 'src/cli.ts', 'jobs', 'watch', '--json'], {
|
||||
cwd: process.cwd(), env, stdio: ['ignore', 'pipe', 'ignore'], encoding: 'utf-8', timeout: 30_000,
|
||||
});
|
||||
const firstLine = out.split('\n').find(l => l.trim().length > 0) ?? '';
|
||||
const parsed = JSON.parse(firstLine);
|
||||
expect(parsed.event).toBe('jobs.watch.snapshot');
|
||||
expect(parsed.queue_health).toBeDefined();
|
||||
}, 35_000);
|
||||
});
|
||||
@@ -0,0 +1,37 @@
|
||||
/**
|
||||
* v0.42.11.0 (#1784) — suspected-contradictions budget-default detection.
|
||||
*
|
||||
* The TTY-derived budget default ($5 TTY / $1 non-TTY) is overwritten in-place,
|
||||
* so the banner needs an explicit `budgetUsdExplicit` flag to know whether to
|
||||
* annotate the value as the silent non-interactive default. These run non-TTY
|
||||
* (bun test pipes stdout), so the default resolves to $1.
|
||||
*/
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { parseFlags } from '../src/commands/eval-suspected-contradictions.ts';
|
||||
|
||||
describe('parseFlags budgetUsdExplicit', () => {
|
||||
test('no --budget-usd → non-TTY default $1, not flagged explicit', () => {
|
||||
const f = parseFlags([]);
|
||||
expect(f.budgetUsd).toBe(1); // bun test is non-TTY
|
||||
expect(f.budgetUsdExplicit).toBe(false);
|
||||
});
|
||||
|
||||
test('--budget-usd 3 → value 3, flagged explicit', () => {
|
||||
const f = parseFlags(['--budget-usd', '3']);
|
||||
expect(f.budgetUsd).toBe(3);
|
||||
expect(f.budgetUsdExplicit).toBe(true);
|
||||
});
|
||||
|
||||
test('explicit value equal to the default still counts as explicit', () => {
|
||||
const f = parseFlags(['--budget-usd', '1']);
|
||||
expect(f.budgetUsd).toBe(1);
|
||||
expect(f.budgetUsdExplicit).toBe(true);
|
||||
});
|
||||
|
||||
test('subcommand positional does not break flag parsing', () => {
|
||||
const f = parseFlags(['run', '--budget-usd', '2']);
|
||||
expect(f.sub).toBe('run');
|
||||
expect(f.budgetUsd).toBe(2);
|
||||
expect(f.budgetUsdExplicit).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,60 @@
|
||||
/**
|
||||
* v0.42.11.0 (#1784) — cycle-default resolver unit tests.
|
||||
*
|
||||
* Pins the 4 branches of resolveCycleDefault + the banner-suffix contract.
|
||||
* Pure functions, no DB, no env mutation.
|
||||
*/
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import {
|
||||
resolveCycleDefault,
|
||||
cycleDefaultSuffix,
|
||||
DEFAULT_CYCLES_TTY,
|
||||
DEFAULT_CYCLES_NONTTY,
|
||||
} from '../../src/core/eval/cycle-default.ts';
|
||||
|
||||
describe('resolveCycleDefault', () => {
|
||||
test('explicit value wins (TTY) — no annotation', () => {
|
||||
const r = resolveCycleDefault(5, true);
|
||||
expect(r).toEqual({ cycles: 5, usedNonTtyDefault: false });
|
||||
});
|
||||
|
||||
test('explicit value wins (non-TTY) — no annotation', () => {
|
||||
const r = resolveCycleDefault(5, false);
|
||||
expect(r).toEqual({ cycles: 5, usedNonTtyDefault: false });
|
||||
});
|
||||
|
||||
test('TTY default → 3, no annotation', () => {
|
||||
const r = resolveCycleDefault(undefined, true);
|
||||
expect(r).toEqual({ cycles: DEFAULT_CYCLES_TTY, usedNonTtyDefault: false });
|
||||
expect(r.cycles).toBe(3);
|
||||
});
|
||||
|
||||
test('non-TTY default → 1, ANNOTATED (the silent-degradation case)', () => {
|
||||
const r = resolveCycleDefault(undefined, false);
|
||||
expect(r).toEqual({ cycles: DEFAULT_CYCLES_NONTTY, usedNonTtyDefault: true });
|
||||
expect(r.cycles).toBe(1);
|
||||
});
|
||||
|
||||
test('explicit 1 in non-TTY is NOT flagged as the default (user asked for it)', () => {
|
||||
const r = resolveCycleDefault(1, false);
|
||||
expect(r.usedNonTtyDefault).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('cycleDefaultSuffix', () => {
|
||||
test('empty unless the non-TTY default was applied', () => {
|
||||
expect(cycleDefaultSuffix({ cycles: 3, usedNonTtyDefault: false })).toBe('');
|
||||
expect(cycleDefaultSuffix({ cycles: 5, usedNonTtyDefault: false })).toBe('');
|
||||
});
|
||||
|
||||
test('names the --cycles override when annotated', () => {
|
||||
const s = cycleDefaultSuffix({ cycles: 1, usedNonTtyDefault: true });
|
||||
expect(s).toContain('non-interactive default');
|
||||
expect(s).toContain('--cycles');
|
||||
});
|
||||
|
||||
test('round-trips with resolveCycleDefault non-TTY path', () => {
|
||||
expect(cycleDefaultSuffix(resolveCycleDefault(undefined, false))).not.toBe('');
|
||||
expect(cycleDefaultSuffix(resolveCycleDefault(undefined, true))).toBe('');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,44 @@
|
||||
/**
|
||||
* v0.42.15.0 (#1784) — jobs-watch format × loop matrix.
|
||||
*
|
||||
* Pins resolveWatchMode so the TTY-gating contract can't regress. The bug this
|
||||
* guards: TTY + --json used to loop forever (follow defaulted to isTTY); the
|
||||
* documented matrix says --json one-shots unless --follow. Caught by codex in
|
||||
* the pre-landing review; the e2e missed it (non-TTY only).
|
||||
*/
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { resolveWatchMode } from '../src/commands/jobs-watch.ts';
|
||||
|
||||
describe('resolveWatchMode — format × loop matrix', () => {
|
||||
test('TTY, no flags → live dashboard (json off, follow on, ansi on)', () => {
|
||||
expect(resolveWatchMode({}, true)).toEqual({ json: false, follow: true, useAnsiDashboard: true });
|
||||
});
|
||||
|
||||
test('non-TTY, no flags → one human snapshot (the bug fix: not JSON, one-shot)', () => {
|
||||
expect(resolveWatchMode({}, false)).toEqual({ json: false, follow: false, useAnsiDashboard: false });
|
||||
});
|
||||
|
||||
test('TTY + --json → ONE JSON snapshot, NOT a forever loop (codex finding)', () => {
|
||||
expect(resolveWatchMode({ json: true }, true)).toEqual({ json: true, follow: false, useAnsiDashboard: false });
|
||||
});
|
||||
|
||||
test('non-TTY + --json → one JSON snapshot', () => {
|
||||
expect(resolveWatchMode({ json: true }, false)).toEqual({ json: true, follow: false, useAnsiDashboard: false });
|
||||
});
|
||||
|
||||
test('TTY + --follow → live dashboard loops', () => {
|
||||
expect(resolveWatchMode({ follow: true }, true)).toEqual({ json: false, follow: true, useAnsiDashboard: true });
|
||||
});
|
||||
|
||||
test('non-TTY + --follow → plain human stream (loops, no ansi)', () => {
|
||||
expect(resolveWatchMode({ follow: true }, false)).toEqual({ json: false, follow: true, useAnsiDashboard: false });
|
||||
});
|
||||
|
||||
test('--json --follow → JSONL stream, never the ansi dashboard', () => {
|
||||
expect(resolveWatchMode({ json: true, follow: true }, true)).toEqual({ json: true, follow: true, useAnsiDashboard: false });
|
||||
});
|
||||
|
||||
test('explicit --follow false on a TTY → one-shot (override wins)', () => {
|
||||
expect(resolveWatchMode({ follow: false }, true)).toEqual({ json: false, follow: false, useAnsiDashboard: false });
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,52 @@
|
||||
/**
|
||||
* v0.42.11.0 (#1784) — reindex-code cost-refusal format unit tests.
|
||||
*
|
||||
* The cost gate still REFUSES to spend non-interactively without --yes (exit 2,
|
||||
* tested at the CLI layer). This pins the separate FORMAT axis: JSON only when
|
||||
* --json is explicit; otherwise a human refusal on stderr. Pure function — no
|
||||
* brain, no real cost preview.
|
||||
*/
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { buildCostRefusal } from '../src/commands/reindex-code.ts';
|
||||
|
||||
const PREVIEW_MSG = 'reindex-code: 42 code page(s), ~12,345 tokens, est. $0.12 on voyage:voyage-code-3.';
|
||||
|
||||
describe('buildCostRefusal', () => {
|
||||
test('--json → machine-readable envelope on stdout, nothing on stderr', () => {
|
||||
const r = buildCostRefusal({
|
||||
json: true,
|
||||
previewMsg: PREVIEW_MSG,
|
||||
preview: { totalPages: 42, totalTokens: 12345 },
|
||||
costUsd: 0.12,
|
||||
model: 'voyage:voyage-code-3',
|
||||
});
|
||||
expect(r.stderr).toBeUndefined();
|
||||
expect(typeof r.stdout).toBe('string');
|
||||
const parsed = JSON.parse(r.stdout!);
|
||||
expect(parsed.error).toBeDefined();
|
||||
// The structured refusal must still carry the confirmation code + context.
|
||||
expect(JSON.stringify(parsed.error)).toContain('cost_preview_requires_yes');
|
||||
expect(parsed.costUsd).toBe(0.12);
|
||||
expect(parsed.model).toBe('voyage:voyage-code-3');
|
||||
expect(parsed.preview).toEqual({ totalPages: 42, totalTokens: 12345 });
|
||||
});
|
||||
|
||||
test('no --json → human refusal on stderr, nothing on stdout (not JSON)', () => {
|
||||
const r = buildCostRefusal({
|
||||
json: false,
|
||||
previewMsg: PREVIEW_MSG,
|
||||
preview: {},
|
||||
costUsd: 0.12,
|
||||
model: 'voyage:voyage-code-3',
|
||||
});
|
||||
expect(r.stdout).toBeUndefined();
|
||||
expect(typeof r.stderr).toBe('string');
|
||||
// Human text: includes the preview, the refusal reason, and both escape hatches.
|
||||
expect(r.stderr).toContain(PREVIEW_MSG);
|
||||
expect(r.stderr).toContain('Refusing to re-embed');
|
||||
expect(r.stderr).toContain('--yes');
|
||||
expect(r.stderr).toContain('--dry-run');
|
||||
// Must NOT be a JSON envelope.
|
||||
expect(r.stderr!.trimStart().startsWith('{')).toBe(false);
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user