mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-27 22:15:33 +00:00
* fix(minions): honest attempt accounting + cooperative abort-honoring + per-handler timeouts (#1737) - Wall-clock and stall dead-letter paths now increment attempts_made (terminal, no retry — wall-clock fires at 2x cumulative timeout; retrying non-idempotent embed/subagent work would duplicate side effects). Surface stalled_counter in jobs get so 'started 3 / stalled 2 / attempts 0' reads true instead of looking like broken accounting. - Thread AbortSignal through embed-backfill/autopilot-cycle -> runPhaseEmbed -> runEmbedCore -> embedAll(Stale)/embedPage, checking it on BOTH --stale and --all paths and between embed batches. A timed-out embed phase now bails within a batch, so the cycle finally releases gbrain_cycle_locks instead of running the full 10-15 min after the job was killed (the daily cycle-wedge). New shared src/core/abort-check.ts (isAborted/throwIfAborted/anySignal). - Per-handler default wall-clock budget (handler-timeouts.ts) stamped at submit for long handlers without an explicit timeout_ms. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(minions): queue-scoped DB supervisor singleton + canonical pidfile + doctor max-rss check (#1849) - Acquire a queue-scoped DB lock (tryAcquireDbLock, keyed on the raw DB identity + queue) on supervisor.start(): a second supervisor on the same (db, queue) fails fast with exit 2 regardless of $HOME/--pid-file. Refresh on a dedicated timer; on refresh failure past the threshold, fail SAFE (exit non-zero) before the TTL could lapse and let a second supervisor take over. Release on shutdown. - Canonical default pidfile keyed on brain id (currentBrainId, config-only, no DB connect) so two brains under one HOME no longer share supervisor.pid. - doctor: new supervisor_singleton check surfaces the effective --max-rss (from the started audit event) and warns when the lock holder's (host,pid) differs from the local pidfile — comparing host+pid, not bare pid. Registered in doctor-categories. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(agent-voice): topic-aware persona context via server-resolved topicId (#1851) Summon Mars/Venus into a specific conversation topic so they boot already knowing the recent thread. A per-topic call link carries only topicId (+ optional display topicName); the server resolves the recent-conversation context from the brain (topics/<topicId>.md) — topic CONTENT is never accepted over the wire (that would be prompt injection + a leak into URLs/referrers/logs). topicId is a strict slug with a path-traversal guard. New '# Topic Context' prompt slot injected after the persona body so identity-first ordering still wins; persona identity unchanged. No topicId -> generic behavior. Contract doc + persona skill docs updated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: file #1737 slot-reservation fair-scheduling follow-up TODO (F7) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: bump version and changelog (v0.42.29.0) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: sync KEY_FILES.md for v0.42.29.0 minions + abort wave (#1737, #1849) Fold the #1737/#1849 behavior into the existing per-file entries and add the two new core files, keeping the doc at current-state truth: - queue.ts: honest attempt accounting on wall-clock + stall dead-letter paths; defaultTimeoutMsFor stamping at submit. - supervisor.ts: queue-scoped DB singleton lock (supervisorLockId, classifySupervisorSingleton, LOCK_LOST, refresh-fail-safe, brain-id pidfile, max_rss_mb audit). - worker-registry.ts: currentDbIdentity(). - New entries: src/core/abort-check.ts, src/core/minions/handler-timeouts.ts. - New doctor.ts extension: supervisor_singleton check. - cycle.ts / embed.ts extensions: AbortSignal threading note. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
78 lines
3.3 KiB
TypeScript
78 lines
3.3 KiB
TypeScript
/**
|
|
* Test fixture: spawns a MinionSupervisor with options parsed from env vars.
|
|
*
|
|
* Used by test/supervisor.test.ts integration tests. Separate file because
|
|
* the supervisor calls `process.exit()` at the end of its lifecycle — tests
|
|
* spawn this runner as a subprocess to observe exit codes and audit events
|
|
* without killing the test runner itself.
|
|
*
|
|
* Env vars (all optional, sensible defaults for tests):
|
|
* SUP_CLI_PATH — worker binary path (default: /bin/sh exit-1 script)
|
|
* SUP_PID_FILE — PID file path (REQUIRED; each test uses a unique one)
|
|
* SUP_MAX_CRASHES — max consecutive crashes (default: 3)
|
|
* SUP_BACKOFF_FLOOR_MS — test-only short backoff (default: 1)
|
|
* SUP_HEALTH_INTERVAL_MS — how often healthCheck fires (default: 999_999 off)
|
|
* SUP_ALLOW_SHELL_JOBS — "1" to set allowShellJobs:true, else false
|
|
* SUP_QUEUE — queue name (default: 'default')
|
|
* SUP_AUDIT_DIR — GBRAIN_AUDIT_DIR override (default: tmpdir/supervisor-test)
|
|
*/
|
|
|
|
import { MinionSupervisor } from '../../src/core/minions/supervisor.ts';
|
|
import { writeSupervisorEvent } from '../../src/core/minions/handlers/supervisor-audit.ts';
|
|
import type { BrainEngine } from '../../src/core/engine.ts';
|
|
|
|
// Mock engine: healthCheck() calls engine.executeRaw; return empty rows so
|
|
// the query path exercises without needing Postgres.
|
|
//
|
|
// #1849: start() now acquires the queue-scoped DB singleton lock via
|
|
// tryAcquireDbLock, which uses the postgres `sql` tagged-template escape hatch.
|
|
// The stub returns a single row from every call so acquire succeeds (length 1
|
|
// → acquired) and refresh/release are no-ops. Each spawned runner is a fresh
|
|
// process, so there's no cross-test lock state to clean up.
|
|
const sqlStub = (..._args: unknown[]) => Promise.resolve([{ id: 'supervisor-lock' }]);
|
|
const mockEngine: Partial<BrainEngine> = {
|
|
kind: 'postgres' as const,
|
|
executeRaw: async () => [],
|
|
sql: sqlStub,
|
|
} as unknown as BrainEngine;
|
|
|
|
const pidFile = process.env.SUP_PID_FILE;
|
|
if (!pidFile) {
|
|
console.error('SUP_PID_FILE env var is required');
|
|
process.exit(99);
|
|
}
|
|
|
|
const cliPath = process.env.SUP_CLI_PATH ?? '/bin/sh';
|
|
const maxCrashes = parseInt(process.env.SUP_MAX_CRASHES ?? '3', 10);
|
|
const backoffFloor = parseInt(process.env.SUP_BACKOFF_FLOOR_MS ?? '1', 10);
|
|
const healthInterval = parseInt(process.env.SUP_HEALTH_INTERVAL_MS ?? '999999', 10);
|
|
const allowShellJobs = process.env.SUP_ALLOW_SHELL_JOBS === '1';
|
|
const queueName = process.env.SUP_QUEUE ?? 'default';
|
|
// SUP_MAX_RSS: when set, pin an explicit watchdog cap (tests the passthrough
|
|
// path). When unset, MinionSupervisor auto-sizes cgroup-aware (issue #1678).
|
|
const maxRssExplicit = process.env.SUP_MAX_RSS !== undefined
|
|
? parseInt(process.env.SUP_MAX_RSS, 10)
|
|
: undefined;
|
|
|
|
if (process.env.SUP_AUDIT_DIR) {
|
|
process.env.GBRAIN_AUDIT_DIR = process.env.SUP_AUDIT_DIR;
|
|
}
|
|
|
|
const supervisorPid = process.pid;
|
|
|
|
const supervisor = new MinionSupervisor(mockEngine as BrainEngine, {
|
|
concurrency: 1,
|
|
queue: queueName,
|
|
pidFile,
|
|
maxCrashes,
|
|
healthInterval,
|
|
cliPath,
|
|
allowShellJobs,
|
|
json: true,
|
|
_backoffFloorMs: backoffFloor,
|
|
...(maxRssExplicit !== undefined ? { maxRssMb: maxRssExplicit } : {}),
|
|
onEvent: (emission) => writeSupervisorEvent(emission, supervisorPid),
|
|
});
|
|
|
|
await supervisor.start();
|