Files
gbrain/recipes/agent-voice/install/refresh-algorithm.md
T
e9fa51d46e v0.40.0.0 feat: agent-voice (Mars + Venus) + copy-into-host-repo skillpack paradigm (#1128)
* feat: agent-voice reference skillpack (Mars + Venus) + copy-into-host-repo install paradigm

Ships a new skillpack paradigm: gbrain holds the REFERENCE content;
`gbrain integrations install agent-voice --target <repo>` COPIES it into
the operator's host agent repo where it becomes user-owned and mutable.
Future refresh is diff-and-propose against per-file SHA-256 hashes from
.gbrain-source.json, not blind overwrite.

What ships:
- recipes/agent-voice.md entrypoint + recipes/agent-voice/ bundle
- Two voice personas (Mars dual-mode SOLO/DEMO, Venus executive assistant)
  with PII / private-agent-name / hardcoded-path scrubbed out
- WebRTC-first browser client (call.html) with ?test=1 gated instrumentation
  and Web Audio API tee -> MediaRecorder capture for E2E roundtrip testing
- Read-only tool router (D14-A allow-list: search, query, get_page,
  list_pages, find_experts, get_recent_salience, get_recent_transcripts,
  read_article). Write ops permanently denylisted; opt-in via local override
- Persona-aware prompt builder with identity-first composition + Unicode
  sanitization for OpenAI Realtime API safety
- Upstream-error classifier (HTTP 429/500/503 -> soft-fail, plumbing -> hard)
- Three SKILL.md skills (voice-persona-mars, voice-persona-venus,
  voice-post-call) with routing-eval.jsonl fixtures
- 99 host-side tests (vitest-compatible, runs in bun) covering registry,
  prompt-shape privacy guards, tool allow-list, upstream classifier
- install/manifest.json + refresh-algorithm.md + post-install-hint.md

Privacy infrastructure:
- scripts/check-no-pii-in-agent-voice.sh wired into bun run verify
  Shape regex (phone/email/SSN/JWT/bearer/credit-card) + path patterns +
  $AGENT_VOICE_PII_BLOCKLIST env-driven name blocklist
- scripts/import-from-upstream.sh + scripts/upstream-scrub-table.txt
  Deterministic refresh from upstream voice-agent source. Placeholder-
  driven (envsubst-expanded at run time) so no private names land in
  checked-in files
- recipes/agent-voice/code/lib/personas/private-name-blocklist.json
  Single source of truth for the regex contract (shape categories +
  path patterns + env-var contract for operator-specific names)

src/ surface:
- src/commands/integrations.ts gains `install <recipe-id>` subcommand
  with install_kind: 'local-managed' | 'copy-into-host-repo' discriminator.
  Path-traversal hardening (rejects '..', absolute paths, symlink escapes).
  Refuses target == gbrain itself, missing .git, existing files (without
  --overwrite). Writes .gbrain-source.json with per-file SHA-256. Appends
  resolver rows to host repo's RESOLVER.md or AGENTS.md.
- test/integrations-install.test.ts: 11 cases (happy path, manifest shape,
  no upstream_repo field per D11-A, resolver appending, file modes,
  refusal cases, dry-run)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: bump version and changelog (v0.36.0.0)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(privacy): scrub literal private agent names from prompt-shape tests + guard script

The prompt-shape tests carried regex patterns naming the literal banned terms
(Garry/Steph/Garrison/Solomon/Herbert/Wintermute) inline. CLAUDE.md's
"never use Wintermute in any public artifact" applies to test source files
too. Master's check-privacy.sh correctly caught this.

Replaced with env-driven check that reads AGENT_VOICE_PII_BLOCKLIST (the
single source of truth from private-name-blocklist.json). Same enforcement
guarantee via the env var, zero literal names in shipped source.

Also scrubbed the literal /data/.openclaw/ from the guard script's comment
and the literal 'tell_wintermute' from the venus write-tools test.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: ship all v0.36.0.0 deferred items in this PR (E2E + evals + pipeline + refresh + multilingual + twilio deprecation)

Closes the "deferred to follow-up" section of the v0.36.0.0 CHANGELOG.

E2E tests + harness (env-gated):
- tests/e2e/voice-roundtrip.test.mjs — spawns server, drives puppeteer + fake-audio, three-tier assertions (CONNECTION hard, NON-SILENT hard, SEMANTIC soft via Whisper + LLM judge). Upstream errors (429/500/503, WS 1011/1013) soft-fail via lib/upstream-classifier.mjs.
- tests/e2e/voice-full-flow.test.mjs — wraps openclaw doing the install, then runs the roundtrip. Friction-discovery flavor, NOT a ship gate.
- tests/e2e/lib/browser-audio.mjs — puppeteer + fake-audio harness; reads window._gbrainTest namespace; PCM RMS-variance helper.
- tests/e2e/lib/whisper-judge.mjs — Whisper transcription + LLM-judge for SEMANTIC tier.
- tests/e2e/audio-fixtures/utterance-{add,joke,brain-query}.wav — 16kHz mono WAV via `say` + ffmpeg, committed for reproducibility.
- test/fixtures/claw-test-scenarios/voice-agent-install/{BRIEF.md, scenario.json, expected.json} — labeled BENCHMARK_FRICTION, blocks_ship=false.

LLM-judge persona evals + synthetic canonical baselines:
- tests/evals/judge.mjs — gateway-routed 3-model (Claude + GPT + Gemini) harness with 4-strategy JSON repair + 2/3-quorum aggregation (per the v0.27.x cross-modal pattern). Pass criterion: every axis mean ≥7 AND no model <5.
- tests/evals/fixtures/{mars-solo,mars-demo,venus,persona-routing,mars-multilingual}.jsonl — 5 fixture sets covering all axes.
- tests/evals/{mars-eval,venus-eval,mars-multilingual-eval,persona-routing-eval}.mjs — per-axis drivers.
- tests/evals/baseline-runs/canonical/*.json — agent-authored synthetic exemplars (PII-impossible by construction; demonstrate expected pass shape; never overwrite with live model output).
- tests/evals/baseline-runs/.gitignore — live receipts excluded.

DIY pipeline (Option B):
- code/pipeline.mjs — streaming STT (Deepgram nova-2) + LLM (Claude Sonnet 4.6 streaming SSE with sentence-boundary TTS dispatch) + TTS (Cartesia primary, OpenAI TTS fallback). 20-turn history cap, exponential-backoff reconnects, 25s keepalives, VAD presets (quiet/normal/noisy/very_noisy), barge-in via STT speechStart → LLM interrupt. Modular adapters for swapping providers.

--refresh mode (D3-A diff-and-propose):
- src/commands/integrations.ts: refreshRecipeIntoHostRepo() + classifyForRefresh() implementing the five states from refresh-algorithm.md (unchanged-identical, unchanged-stale, locally-modified, source-deleted, host-deleted, new-in-manifest). Transaction journal at .gbrain-source.refresh.log. Default policy: preserve operator's local edits (keep-mine); --auto take-theirs to overwrite; --dry-run for preview.
- test/integrations-install.test.ts: 7 new test cases pinning each classification state + default-preserve behavior + take-theirs overwrite + transaction journal + refusal on uninstalled target.

Mars multilingual restore:
- code/lib/personas/mars.mjs: explicit cross-lingual rule (Mandarin, Spanish, French, Japanese, Korean default to English but follow the speaker). Voice (Orus) supports the languages natively.
- tests/unit/mars-prompt-shape.test.mjs: assertion flipped from "MUST NOT claim multilingual" to "declares cross-lingual capability with English bias."
- tests/evals/fixtures/mars-multilingual.jsonl: 5 fixtures across Mandarin/Spanish/Japanese/French + explicit switch-back, pinned by mars-multilingual-eval.mjs.

Twilio recipe deprecation:
- recipes/twilio-voice-brain.md: deprecation banner pointing at agent-voice.md. Frontmatter version bumped to 0.8.2. Will be removed in v0.37.

Verify: bun run verify clean, 6736+ unit tests pass, 18/18 install+refresh tests pass, 96/98 host-side persona/tool/classifier tests pass (2 skipped env-gated).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: update CHANGELOG — all v0.36.0.0 deferred items now shipped in this PR

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: rebump version v0.36.0.0 → v0.37.0.0

Captures the wave-1 + wave-2 scope at the v0.37 slot. The bump reflects
the size of what this PR ships: copy-into-host-repo install paradigm
(new install_kind discriminator + new install/refresh subcommand) +
Mars/Venus voice agent reference + 5,500+ LOC of vendored scrubbed
code + 4 LLM-judge eval suites + 2 env-gated E2E test suites + DIY
Option B pipeline + 18-case install subcommand test coverage. A minor
bump felt too small.

Side fix: privacy guard caught two stale literal "wintermute" and
"/data/.openclaw/" references in wave-2 files
(voice-full-flow.test.mjs comment, expected.json blocklist payload).
Both replaced with env-driven references to $AGENT_VOICE_PII_BLOCKLIST
matching the D15-A pattern from the original review.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: rebump v0.37.0.0 → v0.40.0.0

Jumps past the v0.37/v0.38/v0.39 slots master might claim in subsequent
PRs. The wave's scope (copy-into-host-repo skillpack paradigm + agent-voice
+ install/refresh + LLM-judge evals + DIY pipeline + Mars multilingual)
justifies a larger version arithmetic step.

Files bumped:
- VERSION 0.37.0.0 → 0.40.0.0
- package.json 0.37.0.0 → 0.40.0.0
- CHANGELOG.md header + "To take advantage of v0.40.0.0" block
- recipes/twilio-voice-brain.md deprecation banner (now "removed in v0.41")
- recipes/agent-voice/tests/evals/mars-multilingual-eval.mjs comment

Left alone: master's pre-existing "v0.37+" roadmap labels in src/core/calibration/*,
src/core/cycle/*, DESIGN.md, CLAUDE.md, etc. Those are master's author-intent
references to "the next planned release" relative to master's frame at the time —
rewriting them just to keep numbering consistent would overreach.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 21:54:13 -07:00

4.4 KiB

Refresh algorithm (diff-and-propose)

gbrain integrations install agent-voice --refresh re-walks the manifest, classifies every file into one of five states, and lets the operator decide per-file. The reference implementation is in src/commands/integrations.ts under the install_kind: copy-into-host-repo branch.

State machine

For each file declared in install/manifest.json:

Let src_hash    = SHA-256 of gbrain-side file at manifest.src
Let host_path   = <target-repo>/<manifest.target>
Let recorded   = .gbrain-source.json.files[].sha256 for this entry (or absent if first refresh)
Let host_hash  = SHA-256 of host_path (or absent if file deleted on host side)

State:
  - "unchanged-identical"  iff host_hash == src_hash
                           → no-op
  - "unchanged-stale"      iff host_hash == recorded AND host_hash != src_hash
                           → operator unmodified, source moved → offer update
  - "locally-modified"     iff host_hash != recorded AND host_hash != src_hash AND host_hash is defined
                           → operator edited locally; offer three options (see below)
  - "host-deleted"         iff host_hash is absent AND src exists
                           → operator removed the file; offer to restore or to remove from manifest
  - "source-deleted"       iff src is absent AND host_hash is defined
                           → gbrain reference removed the file; offer cleanup (remove from host)

A path-mapping renames table in the manifest (renames: [{from, to}], not yet shipped) allows the refresh algorithm to detect a source-renamed file as a logical update rather than a delete+add.

"Locally-modified" decision

When a file shows locally-modified, the operator picks one of three options:

  • keep-mine — leave host file untouched. The manifest entry's sha256 is updated to the current host hash (the operator's edit becomes the new "recorded" baseline; future refreshes won't re-flag it until they edit it again OR the source changes).
  • take-theirs — copy the gbrain reference over the host file. The recorded SHA becomes the new src_hash.
  • merge — print a unified diff. Operator hand-merges in their editor; the refresh command exits without writing. Re-run --refresh after the merge to confirm.

Transaction journal

<target-repo>/services/voice-agent/.gbrain-source.refresh.log is a JSONL append-only file. Each line records:

{"ts": "2026-05-17T12:34:56Z", "src": "code/server.mjs", "state": "locally-modified", "decision": "keep-mine"}

The journal exists for two reasons:

  1. Partial-apply recovery. If the refresh is interrupted mid-loop (Ctrl-C, crash, machine reboot), re-running --refresh reads the journal and resumes where it stopped.
  2. Audit. Operators can grep the journal to see which files were touched and why.

The journal is rotated by file size (>1MB triggers rename to .gbrain-source.refresh.log.1) and ignored by --refresh's own scan (the journal is host-only metadata, not a managed file).

Concurrent refresh guard

--refresh acquires an advisory file lock at <target-repo>/services/voice-agent/.gbrain-source.refresh.lock for the duration of the run. Concurrent --refresh invocations on the same host repo fail-fast with "refresh already in progress."

CLI surface

gbrain integrations install agent-voice --target <repo> --refresh
gbrain integrations install agent-voice --target <repo> --refresh --dry-run     # report-only
gbrain integrations install agent-voice --target <repo> --refresh --auto take-theirs   # non-interactive
gbrain integrations install agent-voice --target <repo> --refresh --auto keep-mine     # bias toward operator's edits

--auto <decision> applies the named decision to ALL locally-modified files without prompting. Useful for CI lanes that want either "always take upstream" or "always preserve local" without operator interaction.

What this v0 deliberately skips

  • Conflict resolution for files that exist in both manifests but at different paths (treated as add+delete).
  • Concurrent edits on the SAME file mid-refresh (the advisory lock + per-file atomic write covers this).
  • Semantic merges (we offer file-level diff only; no per-hunk picking).
  • Manifest schema migration (v0.1.0 → v0.2.0 changes are handled by the install command refusing to refresh old manifests and asking the operator to re-install).

Each of those is a follow-up TODO.