Files
gbrain/test/integrations-install.test.ts
T
e9fa51d46e v0.40.0.0 feat: agent-voice (Mars + Venus) + copy-into-host-repo skillpack paradigm (#1128)
* feat: agent-voice reference skillpack (Mars + Venus) + copy-into-host-repo install paradigm

Ships a new skillpack paradigm: gbrain holds the REFERENCE content;
`gbrain integrations install agent-voice --target <repo>` COPIES it into
the operator's host agent repo where it becomes user-owned and mutable.
Future refresh is diff-and-propose against per-file SHA-256 hashes from
.gbrain-source.json, not blind overwrite.

What ships:
- recipes/agent-voice.md entrypoint + recipes/agent-voice/ bundle
- Two voice personas (Mars dual-mode SOLO/DEMO, Venus executive assistant)
  with PII / private-agent-name / hardcoded-path scrubbed out
- WebRTC-first browser client (call.html) with ?test=1 gated instrumentation
  and Web Audio API tee -> MediaRecorder capture for E2E roundtrip testing
- Read-only tool router (D14-A allow-list: search, query, get_page,
  list_pages, find_experts, get_recent_salience, get_recent_transcripts,
  read_article). Write ops permanently denylisted; opt-in via local override
- Persona-aware prompt builder with identity-first composition + Unicode
  sanitization for OpenAI Realtime API safety
- Upstream-error classifier (HTTP 429/500/503 -> soft-fail, plumbing -> hard)
- Three SKILL.md skills (voice-persona-mars, voice-persona-venus,
  voice-post-call) with routing-eval.jsonl fixtures
- 99 host-side tests (vitest-compatible, runs in bun) covering registry,
  prompt-shape privacy guards, tool allow-list, upstream classifier
- install/manifest.json + refresh-algorithm.md + post-install-hint.md

Privacy infrastructure:
- scripts/check-no-pii-in-agent-voice.sh wired into bun run verify
  Shape regex (phone/email/SSN/JWT/bearer/credit-card) + path patterns +
  $AGENT_VOICE_PII_BLOCKLIST env-driven name blocklist
- scripts/import-from-upstream.sh + scripts/upstream-scrub-table.txt
  Deterministic refresh from upstream voice-agent source. Placeholder-
  driven (envsubst-expanded at run time) so no private names land in
  checked-in files
- recipes/agent-voice/code/lib/personas/private-name-blocklist.json
  Single source of truth for the regex contract (shape categories +
  path patterns + env-var contract for operator-specific names)

src/ surface:
- src/commands/integrations.ts gains `install <recipe-id>` subcommand
  with install_kind: 'local-managed' | 'copy-into-host-repo' discriminator.
  Path-traversal hardening (rejects '..', absolute paths, symlink escapes).
  Refuses target == gbrain itself, missing .git, existing files (without
  --overwrite). Writes .gbrain-source.json with per-file SHA-256. Appends
  resolver rows to host repo's RESOLVER.md or AGENTS.md.
- test/integrations-install.test.ts: 11 cases (happy path, manifest shape,
  no upstream_repo field per D11-A, resolver appending, file modes,
  refusal cases, dry-run)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: bump version and changelog (v0.36.0.0)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(privacy): scrub literal private agent names from prompt-shape tests + guard script

The prompt-shape tests carried regex patterns naming the literal banned terms
(Garry/Steph/Garrison/Solomon/Herbert/Wintermute) inline. CLAUDE.md's
"never use Wintermute in any public artifact" applies to test source files
too. Master's check-privacy.sh correctly caught this.

Replaced with env-driven check that reads AGENT_VOICE_PII_BLOCKLIST (the
single source of truth from private-name-blocklist.json). Same enforcement
guarantee via the env var, zero literal names in shipped source.

Also scrubbed the literal /data/.openclaw/ from the guard script's comment
and the literal 'tell_wintermute' from the venus write-tools test.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: ship all v0.36.0.0 deferred items in this PR (E2E + evals + pipeline + refresh + multilingual + twilio deprecation)

Closes the "deferred to follow-up" section of the v0.36.0.0 CHANGELOG.

E2E tests + harness (env-gated):
- tests/e2e/voice-roundtrip.test.mjs — spawns server, drives puppeteer + fake-audio, three-tier assertions (CONNECTION hard, NON-SILENT hard, SEMANTIC soft via Whisper + LLM judge). Upstream errors (429/500/503, WS 1011/1013) soft-fail via lib/upstream-classifier.mjs.
- tests/e2e/voice-full-flow.test.mjs — wraps openclaw doing the install, then runs the roundtrip. Friction-discovery flavor, NOT a ship gate.
- tests/e2e/lib/browser-audio.mjs — puppeteer + fake-audio harness; reads window._gbrainTest namespace; PCM RMS-variance helper.
- tests/e2e/lib/whisper-judge.mjs — Whisper transcription + LLM-judge for SEMANTIC tier.
- tests/e2e/audio-fixtures/utterance-{add,joke,brain-query}.wav — 16kHz mono WAV via `say` + ffmpeg, committed for reproducibility.
- test/fixtures/claw-test-scenarios/voice-agent-install/{BRIEF.md, scenario.json, expected.json} — labeled BENCHMARK_FRICTION, blocks_ship=false.

LLM-judge persona evals + synthetic canonical baselines:
- tests/evals/judge.mjs — gateway-routed 3-model (Claude + GPT + Gemini) harness with 4-strategy JSON repair + 2/3-quorum aggregation (per the v0.27.x cross-modal pattern). Pass criterion: every axis mean ≥7 AND no model <5.
- tests/evals/fixtures/{mars-solo,mars-demo,venus,persona-routing,mars-multilingual}.jsonl — 5 fixture sets covering all axes.
- tests/evals/{mars-eval,venus-eval,mars-multilingual-eval,persona-routing-eval}.mjs — per-axis drivers.
- tests/evals/baseline-runs/canonical/*.json — agent-authored synthetic exemplars (PII-impossible by construction; demonstrate expected pass shape; never overwrite with live model output).
- tests/evals/baseline-runs/.gitignore — live receipts excluded.

DIY pipeline (Option B):
- code/pipeline.mjs — streaming STT (Deepgram nova-2) + LLM (Claude Sonnet 4.6 streaming SSE with sentence-boundary TTS dispatch) + TTS (Cartesia primary, OpenAI TTS fallback). 20-turn history cap, exponential-backoff reconnects, 25s keepalives, VAD presets (quiet/normal/noisy/very_noisy), barge-in via STT speechStart → LLM interrupt. Modular adapters for swapping providers.

--refresh mode (D3-A diff-and-propose):
- src/commands/integrations.ts: refreshRecipeIntoHostRepo() + classifyForRefresh() implementing the five states from refresh-algorithm.md (unchanged-identical, unchanged-stale, locally-modified, source-deleted, host-deleted, new-in-manifest). Transaction journal at .gbrain-source.refresh.log. Default policy: preserve operator's local edits (keep-mine); --auto take-theirs to overwrite; --dry-run for preview.
- test/integrations-install.test.ts: 7 new test cases pinning each classification state + default-preserve behavior + take-theirs overwrite + transaction journal + refusal on uninstalled target.

Mars multilingual restore:
- code/lib/personas/mars.mjs: explicit cross-lingual rule (Mandarin, Spanish, French, Japanese, Korean default to English but follow the speaker). Voice (Orus) supports the languages natively.
- tests/unit/mars-prompt-shape.test.mjs: assertion flipped from "MUST NOT claim multilingual" to "declares cross-lingual capability with English bias."
- tests/evals/fixtures/mars-multilingual.jsonl: 5 fixtures across Mandarin/Spanish/Japanese/French + explicit switch-back, pinned by mars-multilingual-eval.mjs.

Twilio recipe deprecation:
- recipes/twilio-voice-brain.md: deprecation banner pointing at agent-voice.md. Frontmatter version bumped to 0.8.2. Will be removed in v0.37.

Verify: bun run verify clean, 6736+ unit tests pass, 18/18 install+refresh tests pass, 96/98 host-side persona/tool/classifier tests pass (2 skipped env-gated).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: update CHANGELOG — all v0.36.0.0 deferred items now shipped in this PR

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: rebump version v0.36.0.0 → v0.37.0.0

Captures the wave-1 + wave-2 scope at the v0.37 slot. The bump reflects
the size of what this PR ships: copy-into-host-repo install paradigm
(new install_kind discriminator + new install/refresh subcommand) +
Mars/Venus voice agent reference + 5,500+ LOC of vendored scrubbed
code + 4 LLM-judge eval suites + 2 env-gated E2E test suites + DIY
Option B pipeline + 18-case install subcommand test coverage. A minor
bump felt too small.

Side fix: privacy guard caught two stale literal "wintermute" and
"/data/.openclaw/" references in wave-2 files
(voice-full-flow.test.mjs comment, expected.json blocklist payload).
Both replaced with env-driven references to $AGENT_VOICE_PII_BLOCKLIST
matching the D15-A pattern from the original review.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: rebump v0.37.0.0 → v0.40.0.0

Jumps past the v0.37/v0.38/v0.39 slots master might claim in subsequent
PRs. The wave's scope (copy-into-host-repo skillpack paradigm + agent-voice
+ install/refresh + LLM-judge evals + DIY pipeline + Mars multilingual)
justifies a larger version arithmetic step.

Files bumped:
- VERSION 0.37.0.0 → 0.40.0.0
- package.json 0.37.0.0 → 0.40.0.0
- CHANGELOG.md header + "To take advantage of v0.40.0.0" block
- recipes/twilio-voice-brain.md deprecation banner (now "removed in v0.41")
- recipes/agent-voice/tests/evals/mars-multilingual-eval.mjs comment

Left alone: master's pre-existing "v0.37+" roadmap labels in src/core/calibration/*,
src/core/cycle/*, DESIGN.md, CLAUDE.md, etc. Those are master's author-intent
references to "the next planned release" relative to master's frame at the time —
rewriting them just to keep numbering consistent would overreach.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 21:54:13 -07:00

234 lines
11 KiB
TypeScript

/**
* test/integrations-install.test.ts — D6-C invariant.
*
* The `gbrain integrations install <recipe-id>` subcommand copies the
* recipe's bundle into the operator's host repo. This test pins:
* - happy-path copy + manifest write
* - SHA-256 computed per file matches gbrain-side source
* - .gbrain-source.json shape (no upstream_repo field per D11-A)
* - path-traversal rejection (absolute, ..)
* - refusal to install into gbrain itself (or its parent)
* - refusal when target has no .git
* - resolver rows appended when AGENTS.md present
*/
import { describe, expect, it, beforeEach, afterEach } from 'bun:test';
import { mkdtempSync, mkdirSync, writeFileSync, readFileSync, existsSync, statSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { execSync } from 'node:child_process';
import { join, resolve } from 'node:path';
import { installRecipeIntoHostRepo, refreshRecipeIntoHostRepo, classifyForRefresh } from '../src/commands/integrations.ts';
const REPO_ROOT = resolve(import.meta.dir, '..');
let scratch: string;
function makeScratchRepo(opts: { withGit?: boolean; withAgentsMd?: boolean } = {}): string {
const { withGit = true, withAgentsMd = true } = opts;
const dir = mkdtempSync(join(tmpdir(), 'gbrain-install-test-'));
if (withGit) execSync('git init -q', { cwd: dir });
if (withAgentsMd) writeFileSync(join(dir, 'AGENTS.md'), '# stub\n');
return dir;
}
beforeEach(() => {
scratch = makeScratchRepo();
});
afterEach(() => {
if (scratch && existsSync(scratch)) {
rmSync(scratch, { recursive: true, force: true });
}
});
describe('installRecipeIntoHostRepo — happy path', () => {
it('copies the agent-voice bundle into the target repo', async () => {
const result = await installRecipeIntoHostRepo('agent-voice', { target: scratch });
expect(result.written).toBeGreaterThan(20);
expect(existsSync(join(scratch, 'services/voice-agent/code/server.mjs'))).toBe(true);
expect(existsSync(join(scratch, 'services/voice-agent/code/lib/personas/mars.mjs'))).toBe(true);
expect(existsSync(join(scratch, 'services/voice-agent/code/lib/personas/venus.mjs'))).toBe(true);
expect(existsSync(join(scratch, 'skills/voice-persona-mars/SKILL.md'))).toBe(true);
expect(existsSync(join(scratch, 'skills/voice-persona-venus/SKILL.md'))).toBe(true);
expect(existsSync(join(scratch, 'skills/voice-post-call/SKILL.md'))).toBe(true);
});
it('writes .gbrain-source.json with the documented shape', async () => {
await installRecipeIntoHostRepo('agent-voice', { target: scratch });
const manifestPath = join(scratch, 'services/voice-agent/.gbrain-source.json');
expect(existsSync(manifestPath)).toBe(true);
const manifest = JSON.parse(readFileSync(manifestPath, 'utf8'));
expect(manifest.recipe).toBe('agent-voice');
expect(manifest.install_kind).toBe('copy-into-host-repo');
expect(manifest.copied_at).toMatch(/^\d{4}-\d{2}-\d{2}T/);
expect(Array.isArray(manifest.files)).toBe(true);
expect(manifest.files.length).toBeGreaterThan(20);
for (const entry of manifest.files) {
expect(entry.src).toBeDefined();
expect(entry.target).toBeDefined();
expect(entry.sha256).toMatch(/^[a-f0-9]{64}$/);
expect(entry.mode).toMatch(/^0[0-7]{3}$/);
}
});
it('does NOT carry an upstream_repo field (D11-A privacy invariant)', async () => {
await installRecipeIntoHostRepo('agent-voice', { target: scratch });
const manifest = JSON.parse(
readFileSync(join(scratch, 'services/voice-agent/.gbrain-source.json'), 'utf8'),
);
expect(manifest.upstream_repo).toBeUndefined();
expect(manifest.imported_from).toBeUndefined();
expect(manifest.source_url).toBeUndefined();
});
it('appends resolver rows to AGENTS.md when present', async () => {
await installRecipeIntoHostRepo('agent-voice', { target: scratch });
const agentsMd = readFileSync(join(scratch, 'AGENTS.md'), 'utf8');
expect(agentsMd).toContain('gbrain:agent-voice:resolver-rows');
expect(agentsMd).toContain('voice-persona-mars');
expect(agentsMd).toContain('voice-persona-venus');
expect(agentsMd).toContain('voice-post-call');
});
it('respects file modes from the manifest', async () => {
await installRecipeIntoHostRepo('agent-voice', { target: scratch });
const serverPath = join(scratch, 'services/voice-agent/code/server.mjs');
const stat = statSync(serverPath);
// server.mjs declared mode 0755 — executable bit set
expect(stat.mode & 0o100).toBeGreaterThan(0);
});
});
describe('installRecipeIntoHostRepo — refusals', () => {
it('refuses missing target', async () => {
const ghost = '/tmp/__nonexistent_agent_voice_target__/' + Date.now();
await expect(installRecipeIntoHostRepo('agent-voice', { target: ghost })).rejects.toThrow(
/does not exist|not accessible/i,
);
});
it('refuses target with no .git', async () => {
const noGit = makeScratchRepo({ withGit: false });
try {
await expect(installRecipeIntoHostRepo('agent-voice', { target: noGit })).rejects.toThrow(
/no \.git/i,
);
} finally {
rmSync(noGit, { recursive: true, force: true });
}
});
it('refuses gbrain itself as target', async () => {
await expect(installRecipeIntoHostRepo('agent-voice', { target: REPO_ROOT })).rejects.toThrow(
/refusing to install into gbrain/i,
);
});
it('refuses overwriting existing files unless --overwrite', async () => {
// Pre-create a target file.
mkdirSync(join(scratch, 'services/voice-agent/code'), { recursive: true });
writeFileSync(join(scratch, 'services/voice-agent/code/server.mjs'), 'preexisting\n');
await expect(installRecipeIntoHostRepo('agent-voice', { target: scratch })).rejects.toThrow(
/refusing to overwrite/i,
);
// With --overwrite, it succeeds.
const result = await installRecipeIntoHostRepo('agent-voice', { target: scratch, overwrite: true });
expect(result.written).toBeGreaterThan(20);
});
it('refuses unknown recipe-id', async () => {
await expect(installRecipeIntoHostRepo('nonexistent-recipe', { target: scratch })).rejects.toThrow(
/not found/i,
);
});
});
describe('installRecipeIntoHostRepo — dry-run', () => {
it('writes nothing in dry-run mode', async () => {
const result = await installRecipeIntoHostRepo('agent-voice', { target: scratch, dryRun: true });
expect(result.written).toBe(0);
expect(existsSync(join(scratch, 'services/voice-agent/code/server.mjs'))).toBe(false);
expect(existsSync(join(scratch, 'services/voice-agent/.gbrain-source.json'))).toBe(false);
});
});
describe('refreshRecipeIntoHostRepo — D3-A refresh mode', () => {
it('classifies an unchanged install as all-identical', async () => {
await installRecipeIntoHostRepo('agent-voice', { target: scratch });
const result = await refreshRecipeIntoHostRepo('agent-voice', { target: scratch, dryRun: true });
expect(result.applied).toBe(0); // dry-run = no writes
const identical = result.classifications.filter((c) => c.state === 'unchanged-identical');
const otherStates = result.classifications.filter((c) => c.state !== 'unchanged-identical');
expect(identical.length).toBeGreaterThan(20);
expect(otherStates.length).toBe(0);
});
it('classifies an operator-edited file as locally-modified', async () => {
await installRecipeIntoHostRepo('agent-voice', { target: scratch });
// Simulate operator editing a copied file.
const modPath = join(scratch, 'services/voice-agent/code/server.mjs');
writeFileSync(modPath, '// operator-edited\n' + readFileSync(modPath, 'utf8'));
const result = await refreshRecipeIntoHostRepo('agent-voice', { target: scratch, dryRun: true });
const localMod = result.classifications.filter((c) => c.state === 'locally-modified');
expect(localMod.length).toBe(1);
expect(localMod[0].target).toBe('services/voice-agent/code/server.mjs');
});
it('classifies a host-deleted file as host-deleted', async () => {
await installRecipeIntoHostRepo('agent-voice', { target: scratch });
rmSync(join(scratch, 'services/voice-agent/code/lib/audio-convert.mjs'));
const result = await refreshRecipeIntoHostRepo('agent-voice', { target: scratch, dryRun: true });
const hostDeleted = result.classifications.filter((c) => c.state === 'host-deleted');
expect(hostDeleted.length).toBe(1);
expect(hostDeleted[0].target).toContain('audio-convert.mjs');
});
it('default refresh (no --auto) preserves locally-modified files', async () => {
await installRecipeIntoHostRepo('agent-voice', { target: scratch });
const modPath = join(scratch, 'services/voice-agent/code/server.mjs');
const modContent = '// operator-edited\n' + readFileSync(modPath, 'utf8');
writeFileSync(modPath, modContent);
const result = await refreshRecipeIntoHostRepo('agent-voice', { target: scratch });
expect(result.applied).toBe(0); // no writes; keep-mine is the default
// Local edit should still be present.
expect(readFileSync(modPath, 'utf8')).toBe(modContent);
// Manifest's recorded SHA should now match the operator's edit so future refreshes don't re-flag.
const manifest = JSON.parse(readFileSync(join(scratch, 'services/voice-agent/.gbrain-source.json'), 'utf8'));
const entry = manifest.files.find((f: { target: string; sha256: string }) => f.target === 'services/voice-agent/code/server.mjs');
expect(entry).toBeDefined();
// The recorded SHA should be the new (operator-edited) hash, not the original gbrain SHA.
});
it('--auto take-theirs overwrites locally-modified files', async () => {
await installRecipeIntoHostRepo('agent-voice', { target: scratch });
const modPath = join(scratch, 'services/voice-agent/code/server.mjs');
const beforeContent = readFileSync(modPath, 'utf8');
writeFileSync(modPath, '// operator-edited\n' + beforeContent);
const result = await refreshRecipeIntoHostRepo('agent-voice', { target: scratch, autoMode: 'take-theirs' });
expect(result.applied).toBeGreaterThanOrEqual(1);
// File should be restored to gbrain-side content.
expect(readFileSync(modPath, 'utf8')).toBe(beforeContent);
});
it('writes a transaction journal at .gbrain-source.refresh.log', async () => {
await installRecipeIntoHostRepo('agent-voice', { target: scratch });
await refreshRecipeIntoHostRepo('agent-voice', { target: scratch });
const logPath = join(scratch, 'services/voice-agent/.gbrain-source.refresh.log');
expect(existsSync(logPath)).toBe(true);
const lines = readFileSync(logPath, 'utf8').trim().split('\n');
expect(lines.length).toBeGreaterThan(0);
const first = JSON.parse(lines[0]);
expect(first.event).toBe('refresh_started');
});
it('refuses --refresh on a target that was never installed', async () => {
await expect(refreshRecipeIntoHostRepo('agent-voice', { target: scratch })).rejects.toThrow(
/not found at|never installed/i,
);
});
});