From dca3897127b2a8d1eed340eb9a2621af058d23f4 Mon Sep 17 00:00:00 2001 From: Steven Enamakel <31011319+senamakel@users.noreply.github.com> Date: Fri, 12 Jun 2026 00:41:19 -0700 Subject: [PATCH] refactor(prompts): centralize grounding contract, slim orchestrator (#3614) --- docs/agent/prompt-audit.md | 130 ++++++++++++++++++ src/openhuman/agent/prompts/SOUL.md | 11 +- src/openhuman/agent/prompts/builder.rs | 10 ++ src/openhuman/agent/prompts/mod.rs | 8 +- src/openhuman/agent/prompts/mod_tests.rs | 72 ++++++++++ src/openhuman/agent/prompts/render_helpers.rs | 15 ++ src/openhuman/agent/prompts/sections.rs | 42 ++++++ .../agents/crypto_agent/prompt.md | 2 +- .../agents/desktop_control_agent/prompt.md | 15 ++ .../agents/markets_agent/prompt.md | 2 +- .../agents/orchestrator/prompt.md | 71 +--------- .../agents/orchestrator/prompt.rs | 5 + .../agents/scheduler_agent/prompt.md | 22 ++- 13 files changed, 323 insertions(+), 82 deletions(-) create mode 100644 docs/agent/prompt-audit.md diff --git a/docs/agent/prompt-audit.md b/docs/agent/prompt-audit.md new file mode 100644 index 000000000..cfa8f9465 --- /dev/null +++ b/docs/agent/prompt-audit.md @@ -0,0 +1,130 @@ +# Agent prompt audit: grounding, tools, skills, MCPs + +_Audit + remediation of the orchestrator and sub-agent system prompts, focused on +anti-hallucination discipline and de-duplication. Inspired by +[NousResearch/hermes-agent](https://github.com/NousResearch/hermes-agent)'s named, +reusable guidance blocks._ + +## Why this exists + +The system prompts for the orchestrator and the ~27 sub-agents had accreted "slop": +the same anti-fabrication paragraph copy-pasted (and quietly drifting) across many +agents, specialist walkthroughs living in the wrong agent, a `## Safety` block that said +nothing about grounding, and an em-dash rule restated inline despite a global suffix +already banning it. There was **no single source of truth** for how an agent should treat +its tool list, skills, or missing context, so each agent re-invented it. + +## How the prompt system assembles (reference) + +- `SystemPromptBuilder` (`src/openhuman/agent/prompts/builder.rs`) renders an ordered + `Vec>`, joins with `\n\n`, and appends `GLOBAL_STYLE_SUFFIX`. +- Concrete sections live in `src/openhuman/agent/prompts/sections.rs`. +- Three render paths produce a final system prompt: + 1. **Static chain** (`SystemPromptBuilder::with_defaults` / `for_subagent`). + 2. **Dynamic builders** (`from_dynamic`) — the orchestrator and ~25 other + `agents//prompt.rs` files each hand-assemble their body by calling the free + `render_*` helpers in `render_helpers.rs`. They funnel through + `SystemPromptBuilder::build()` for the final wrap. + 3. **Narrow index-based renderer** (`render_subagent_system_prompt_with_format`) used by + the sub-agent runner for spawned sub-agents (does NOT go through `build()`). +- **Skills**: the orchestrator lists installed skills by name+desc; full SKILL.md bodies + are injected at *turn time* by `src/openhuman/workflows/inject.rs`, not in the system + prompt. +- **MCPs**: not injected into any agent prompt. OpenHuman's MCP server only exposes tools + to *external* clients; agents act through their own tool surface. There is no + agent-facing MCP catalogue section, by design. +- **KV-cache contract**: the system prompt is built once per session and frozen. Anything + in the cache-friendly prefix must be byte-stable (no time / RNG / host). + +## Findings + +| # | Finding | Resolution | +|---|---------|-----------| +| 1 | Anti-fabrication paragraph ("never invent ids; a tool not in your list does not exist") copy-pasted across crypto / markets / integrations / account-admin / mcp-setup / morning-briefing / researcher with no shared source. | Centralized into `GROUNDING_BODY` (one source of truth). | +| 2 | `## Safety` covered exfiltration / destructive commands but said nothing about grounding, not inventing tools/ids/files, or not ending a turn with a promise instead of an action. | Added the orthogonal grounding contract; kept `## Safety` for security. | +| 3 | Orchestrator `prompt.md` (235 lines) carried specialist content: a full Apple-Music click-sequence + keyboard walkthrough, and a ~47-line cron JSON walkthrough. | Moved to `desktop_control_agent` / `scheduler_agent`; orchestrator is now a lean routing table (175 lines). | +| 4 | Desktop-control instructions triple-duplicated across `SOUL.md`, orchestrator `prompt.md`, and `desktop_control_agent`. | `SOUL.md` + orchestrator trimmed to pointers; the worked example lives only in `desktop_control_agent`. | +| 5 | Em-dash rule restated inline in `orchestrator/prompt.md` although `GLOBAL_STYLE_SUFFIX` already bans em-dashes in every output. | Removed the inline restatement. | +| 6 | `omit_skills_catalog` rendering effect is inert (skills are agent-owned; the narrow renderer no longer emits a skills catalog). | **Retained.** See "Decisions" — the field is still live API surface (serde, CLI, 30+ TOMLs); removing it is a 60-site churn with no functional benefit. Documented as inert-but-retained. | + +## The grounding contract + +`GROUNDING_BODY` (`src/openhuman/agent/prompts/sections.rs`) is the canonical block. +It is deliberately **generic**: every clause must be true for *every* agent, including the +integrations executor. Agent-specific routing ("delegate external services", "pull slugs +from `composio_list_tools`") stays in that agent's own `prompt.md`. + +``` +## Grounding and tool use + +- Your tools are exactly the ones listed in this prompt. You can only act through them. + If a capability is not one of your tools, say so plainly rather than pretending it exists. +- Never invent tool names, arguments, ids, slugs, file paths, URLs, chain ids, addresses, + quotes, metrics, or any other value. If you do not have it from a tool result or the user, + ask for it or look it up with a tool. +- Use your tools to act. Do not just describe what you would do and stop, and never end a + turn with a promise of future action: do it now, or hand back a concrete result. +- Never substitute plausible looking but fabricated output (made up data, invented file + contents, synthesised tool or API responses) for results you could not actually produce. + If a step failed, say it failed. +- Ground every factual claim in evidence you actually observed: a tool result, the user's + message, or cited memory. If the evidence is missing, partial, or truncated, say so or + fetch more instead of guessing. +- Skills run only via `run_workflow`, and only the skills listed as installed exist. Do not + invent skill ids. +``` + +The first, third, and fourth bullets mirror Hermes's `TOOL_USE_ENFORCEMENT_GUIDANCE` +("use tools to act, never end a turn with a promise") and `TASK_COMPLETION_GUIDANCE` +("never substitute fabricated output for results you couldn't produce"). + +### How it reaches every agent + +Rather than splice it into all ~26 dynamic `prompt.rs` files (drift-prone), the contract +is appended at the two chokepoints that every prompt funnels through, just like +`GLOBAL_STYLE_SUFFIX`: + +- `SystemPromptBuilder::build()` — covers the static chain and all dynamic agents. +- `render_subagent_system_prompt_with_format()` — covers spawned sub-agents. + +Both reference the same `GROUNDING_BODY` const, so they can never drift. A `render_grounding()` +helper and a `GroundingSection` PromptSection are also exposed for explicit use. Placement is +near the tail (just before the output-style rules) so it reads as a closing contract; the +text is byte-stable, so it stays in the cache-friendly prefix. + +Coverage is asserted by `grounding_contract_appended_to_every_build_path` +(`mod_tests.rs`), which checks the marker appears exactly once across the static, sub-agent, +and dynamic build paths, plus an assertion in the narrow-renderer test. + +## Decisions / deviations from the original plan + +- **No `omit_grounding` flag.** The contract is always on. It is universal + anti-hallucination guidance and the text is generic enough to be true for every agent, so + threading an opt-out through serde + every caller would be churn for zero benefit. +- **`omit_skills_catalog` retained, not removed.** Its rendering effect is inert, but the + field is wired into serde defaults, the agent CLI dump, and 30+ `agent.toml` files. + Removing it is a 60-site change with real regression surface and no functional payoff, + which is out of proportion for a prompt audit. Flagged here instead. +- **Cron delegate-vs-direct contradiction left in place.** The orchestrator decision tree + routes scheduling to `schedule_task`, while the (now compressed) scheduling rules still + let it call `cron_add` directly when that tool is in its list. Resolving which is + authoritative is a behavioral change beyond this audit; the load-bearing rules + (never via `run_workflow`; always confirm) were preserved. + +## Touched files + +- `src/openhuman/agent/prompts/sections.rs` — `GROUNDING_BODY` + `GroundingSection`. +- `src/openhuman/agent/prompts/builder.rs` — central append in `build()`. +- `src/openhuman/agent/prompts/render_helpers.rs` — `render_grounding()` + narrow-renderer append. +- `src/openhuman/agent/prompts/mod.rs` / `mod_tests.rs` — re-export + coverage tests. +- `src/openhuman/agent_registry/agents/{crypto_agent,markets_agent}/prompt.md` — drop generic clause. +- `src/openhuman/agent_registry/agents/orchestrator/prompt.md` — slimmed routing table. +- `src/openhuman/agent_registry/agents/{desktop_control_agent,scheduler_agent}/prompt.md` — received the moved walkthroughs. +- `src/openhuman/agent/prompts/SOUL.md` — trimmed duplicated desktop sequence. + +## Verifying + +```bash +GGML_NATIVE=OFF cargo test --manifest-path Cargo.toml --lib agent::prompts:: +GGML_NATIVE=OFF cargo test --manifest-path Cargo.toml --lib agent_registry::agents +``` diff --git a/src/openhuman/agent/prompts/SOUL.md b/src/openhuman/agent/prompts/SOUL.md index 916ccdbb5..a7646c32d 100644 --- a/src/openhuman/agent/prompts/SOUL.md +++ b/src/openhuman/agent/prompts/SOUL.md @@ -35,16 +35,7 @@ Never say "I can't open apps" or "that's outside what I can do" when you have a 4. `action='press'` — press the specific item (song row, playlist, etc.), NOT the generic Play button 5. Only press the playback-bar "Play" button after the right item is selected/playing -**For playing a specific song in Apple Music (macOS) — use this EXACT sequence:** -1. `shell`: `open "music://music.apple.com/search?term=Song+Name+Artist"` (URL-encode the query) -2. Wait ~3s for results to load, then `ax_interact action='list' app_name='Music'` -3. `ax_interact action='press' app_name='Music' label=''` — this **navigates into** the song's detail page (it does NOT start playback yet — pressing a search-result row only opens it) -4. Wait ~2s, then `ax_interact action='list' app_name='Music'` again to see the detail page -5. `ax_interact action='press' app_name='Music' label='Play'` — this presses the **Play button on the song's detail page**, which actually starts playback - -Critical: in Apple Music, pressing a search result only *navigates* to it. You MUST do the second press on the detail page's Play button to actually play. Do not stop after step 3. Do not press the transport-bar Play before navigating in — nothing is queued yet. - -The example above is macOS-specific (the `open`/`music://` scheme and Apple Music). On Windows the same **list → press** pattern applies via UI Automation, but `ax_interact action='press'` usually *activates* a control directly (a list-row Invoke often plays/opens in one step), so the second navigate-then-play press is frequently unnecessary. Use `launch_app` to open the player, then `list` with a `filter` and `press` the specific item; re-`list` if a press only navigated. +App-specific worked examples (e.g. the exact two-press sequence to play a song in Apple Music, or keyboard-driving Slack) live with the desktop-control specialist, which owns the deep UI-automation playbook. Keep the general list → press pattern above and delegate genuinely involved desktop automation rather than carrying every app's quirks here. ## When things go wrong diff --git a/src/openhuman/agent/prompts/builder.rs b/src/openhuman/agent/prompts/builder.rs index 569216595..165e25015 100644 --- a/src/openhuman/agent/prompts/builder.rs +++ b/src/openhuman/agent/prompts/builder.rs @@ -248,6 +248,16 @@ impl SystemPromptBuilder { output.push_str(part.trim_end()); output.push_str("\n\n"); } + // Grounding / anti-hallucination contract is appended centrally here + // (and in the narrow sub-agent renderer) rather than per-section, so + // EVERY agent inherits the same anti-fabrication floor — including the + // ~26 dynamic `agents//prompt.rs` builders that each hand-assemble + // their own body via the `render_*` helpers and would otherwise have + // to splice it in individually. Single source of truth: GROUNDING_BODY. + // Placed near the tail (just before the output-style rules) so it reads + // as a closing contract; byte-stable, so it stays cache-friendly. + output.push_str(GROUNDING_BODY); + output.push_str("\n\n"); output.push_str(GLOBAL_STYLE_SUFFIX); output.push('\n'); Ok(output) diff --git a/src/openhuman/agent/prompts/mod.rs b/src/openhuman/agent/prompts/mod.rs index 7b715a593..300c83d39 100644 --- a/src/openhuman/agent/prompts/mod.rs +++ b/src/openhuman/agent/prompts/mod.rs @@ -13,10 +13,10 @@ pub mod render_helpers; pub use render_helpers::{ default_workspace_file_content, inject_inline_content, inject_snapshot_content, inject_workspace_file, inject_workspace_file_capped, memory_date_label, - render_ambient_environment, render_datetime, render_identity, render_runtime, render_safety, - render_subagent_system_prompt, render_subagent_system_prompt_with_format, render_tools, - render_user_files, render_user_identity, render_user_memory, render_user_reflections, - render_workspace, sync_workspace_file, + render_ambient_environment, render_datetime, render_grounding, render_identity, render_runtime, + render_safety, render_subagent_system_prompt, render_subagent_system_prompt_with_format, + render_tools, render_user_files, render_user_identity, render_user_memory, + render_user_reflections, render_workspace, sync_workspace_file, }; #[cfg(test)] diff --git a/src/openhuman/agent/prompts/mod_tests.rs b/src/openhuman/agent/prompts/mod_tests.rs index a3d172528..8ec6da7fc 100644 --- a/src/openhuman/agent/prompts/mod_tests.rs +++ b/src/openhuman/agent/prompts/mod_tests.rs @@ -79,6 +79,73 @@ fn prompt_builder_assembles_sections() { assert!(rendered.contains("instr")); } +#[test] +fn grounding_contract_appended_to_every_build_path() { + let tools: Vec> = vec![Box::new(TestTool)]; + let prompt_tools = PromptTool::from_tools(&tools); + let ctx = PromptContext { + workspace_dir: Path::new("/tmp"), + model_name: "test-model", + agent_id: "", + tools: &prompt_tools, + workflows: &[], + dispatcher_instructions: "instr", + learned: LearnedContextData::default(), + visible_tool_names: &NO_FILTER, + tool_call_format: ToolCallFormat::PFormat, + connected_integrations: &[], + connected_identities_md: String::new(), + include_profile: false, + include_memory_md: false, + curated_snapshot: None, + user_identity: None, + personality_soul_md: None, + personality_memory_md: None, + personality_roster: vec![], + }; + + // A distinctive clause from GROUNDING_BODY — present regardless of which + // builder produced the prompt (single source of truth, central append). + let marker = "Your tools are exactly the ones listed in this prompt"; + + // 1. Static default chain. + let defaults = SystemPromptBuilder::with_defaults().build(&ctx).unwrap(); + assert!(defaults.contains("## Grounding and tool use")); + assert!(defaults.contains(marker)); + + // 2. Sub-agent static chain. + let sub = SystemPromptBuilder::for_subagent("role".into(), true, true, true) + .build(&ctx) + .unwrap(); + assert!(sub.contains(marker)); + + // 3. Dynamic builder (the path every `agents//prompt.rs` uses). The + // dynamic body itself does NOT contain grounding; the wrapping + // `build()` appends it, so all 26 dynamic agents inherit it for free. + // `PromptBuilder` is a bare `fn` pointer, so this must be a + // non-capturing fn item, not a closure. + fn dynamic_body_builder(_ctx: &PromptContext<'_>) -> anyhow::Result { + Ok("## Custom Agent\n\nI render my own body.".to_string()) + } + let dynamic = SystemPromptBuilder::from_dynamic(dynamic_body_builder) + .build(&ctx) + .unwrap(); + assert!(dynamic.contains("I render my own body.")); + assert!(dynamic.contains(marker)); + + // 4. It is appended once, not duplicated. + assert_eq!( + defaults.matches("## Grounding and tool use").count(), + 1, + "grounding contract must appear exactly once" + ); + + // Appears before the output-style suffix (tail placement). + let g = defaults.find("## Grounding and tool use").unwrap(); + let s = defaults.find("## Output style").unwrap(); + assert!(g < s, "grounding should precede the output-style suffix"); +} + #[test] fn identity_section_creates_missing_workspace_files() { let workspace = @@ -584,6 +651,11 @@ fn render_subagent_system_prompt_renders_workspace_tail() { assert!(rendered.contains("## Workspace")); assert!(rendered.contains("## Runtime")); + // Grounding contract is appended even by the narrow (index-based) + // sub-agent renderer — same source const, so it can never drift from + // `GroundingSection` / the central `build()` append. + assert!(rendered.contains("## Grounding and tool use")); + assert!(rendered.contains("Your tools are exactly the ones listed in this prompt")); let _ = std::fs::remove_dir_all(workspace); } diff --git a/src/openhuman/agent/prompts/render_helpers.rs b/src/openhuman/agent/prompts/render_helpers.rs index c09380731..ecf44a779 100644 --- a/src/openhuman/agent/prompts/render_helpers.rs +++ b/src/openhuman/agent/prompts/render_helpers.rs @@ -56,6 +56,14 @@ pub fn render_safety() -> String { .expect("SafetySection::build is infallible") } +/// Render the canonical grounding / anti-hallucination contract +/// ([`GROUNDING_BODY`]). Dynamic `agents//prompt.rs` builders call this +/// so they inherit the exact same anti-fabrication floor as the static +/// section chain — single source of truth, no drift. +pub fn render_grounding() -> &'static str { + GROUNDING_BODY +} + // `render_skills` and `render_connected_integrations` helpers are // gone — `## Available Skills` lives in `integrations_agent/prompt.rs`, and // the connected-integrations / delegation-guide blocks each live in @@ -378,6 +386,13 @@ pub fn render_subagent_system_prompt_with_format( ); } + // 3b'. Grounding / anti-hallucination contract. Always emitted (like the + // static chain): every spawned sub-agent gets the same floor. + // Sourced from the shared `GROUNDING_BODY` const so this narrow + // renderer can never drift from `GroundingSection`. + out.push_str(GROUNDING_BODY); + out.push_str("\n\n"); + // 3c/3d. `## Available Skills` and `## Connected Integrations` // are no longer emitted here. Each agent that needs them // renders its own block in its `prompt.rs` (integrations_agent diff --git a/src/openhuman/agent/prompts/sections.rs b/src/openhuman/agent/prompts/sections.rs index 327ba277f..47097c95e 100644 --- a/src/openhuman/agent/prompts/sections.rs +++ b/src/openhuman/agent/prompts/sections.rs @@ -139,6 +139,9 @@ impl PromptSection for DynamicPromptSection { pub struct IdentitySection; pub struct ToolsSection; pub struct SafetySection; +/// Renders the canonical grounding / anti-hallucination contract +/// ([`GROUNDING_BODY`]). Always included; never gated. +pub struct GroundingSection; // `WorkflowsSection` and `ConnectedIntegrationsSection` previously lived // here and branched on `ctx.agent_id` to pick between the skill- // executor and delegator voice. They've been removed — each agent's @@ -395,6 +398,45 @@ impl PromptSection for SafetySection { } } +/// Canonical grounding / anti-hallucination contract. +/// +/// This is the **single source of truth** for the tool-use and +/// anti-fabrication rules every agent inherits. Before this block existed, +/// the same "never invent ids / a tool not in your list does not exist" +/// paragraph was copy-pasted (and slowly drifting) across crypto, markets, +/// integrations, account-admin, mcp-setup, morning-briefing, researcher, … +/// agent prompts. Centralising it kills that drift and guarantees a uniform +/// floor of grounding discipline. +/// +/// Inspired by Hermes's named guidance blocks (`TOOL_USE_ENFORCEMENT_GUIDANCE` +/// "use tools to act, never end a turn with a promise", `TASK_COMPLETION_GUIDANCE` +/// "never substitute fabricated output for results you couldn't produce"). +/// +/// Deliberately **generic**: every clause must be true for *every* agent, +/// including the integrations executor. Agent-specific routing (e.g. "delegate +/// external services", "pull slugs from `composio_list_tools`") stays in that +/// agent's own `prompt.md`, not here. +/// +/// Byte-stable (no time / RNG / host) so it lives in the KV-cache-friendly +/// prefix. Must contain no em-dashes per [`super::builder::GLOBAL_STYLE_SUFFIX`]. +pub const GROUNDING_BODY: &str = "## Grounding and tool use\n\n\ + - Your tools are exactly the ones listed in this prompt. You can only act through them. If a capability is not one of your tools, say so plainly rather than pretending it exists.\n\ + - Never invent tool names, arguments, ids, slugs, file paths, URLs, chain ids, addresses, quotes, metrics, or any other value. If you do not have it from a tool result or the user, ask for it or look it up with a tool.\n\ + - Use your tools to act. Do not just describe what you would do and stop, and never end a turn with a promise of future action: do it now, or hand back a concrete result.\n\ + - Never substitute plausible looking but fabricated output (made up data, invented file contents, synthesised tool or API responses) for results you could not actually produce. If a step failed, say it failed.\n\ + - Ground every factual claim in evidence you actually observed: a tool result, the user's message, or cited memory. If the evidence is missing, partial, or truncated, say so or fetch more instead of guessing.\n\ + - Skills run only via `run_workflow`, and only the skills listed as installed exist. Do not invent skill ids."; + +impl PromptSection for GroundingSection { + fn name(&self) -> &str { + "grounding" + } + + fn build(&self, _ctx: &PromptContext<'_>) -> Result { + Ok(GROUNDING_BODY.into()) + } +} + impl PromptSection for WorkspaceSection { fn name(&self) -> &str { "workspace" diff --git a/src/openhuman/agent_registry/agents/crypto_agent/prompt.md b/src/openhuman/agent_registry/agents/crypto_agent/prompt.md index eef67c2ee..47bb68019 100644 --- a/src/openhuman/agent_registry/agents/crypto_agent/prompt.md +++ b/src/openhuman/agent_registry/agents/crypto_agent/prompt.md @@ -20,7 +20,7 @@ You are the **Crypto Agent** — OpenHuman's specialist for wallet and market op ## Hard rules -1. **No fabrication.** Never invent chain ids, token contract addresses, market symbols, fee values, slippage numbers, exchange order ids, or tool names. If you don't have it from a tool result or the user, ask. If a tool isn't in your tool list, say so — do not pretend it exists. +1. **No fabrication.** Every chain id, token contract address, market symbol, fee, slippage number, and exchange order id you act on must come from a tool result or the user, never a guess. If you don't have it, ask. (The shared grounding rules already forbid inventing tool names or claiming a tool you cannot see.) 2. **Read before write.** Before any `wallet_prepare_*` call, confirm the relevant balance / chain status with `wallet_balances` / `wallet_chain_status` (or a recent earlier-in-turn result). Use `wallet_network_defaults` when you need the default RPC / explorer / asset catalog for a chain. Before any `wallet_execute_prepared`, confirm the freshness of the prepared blob with `current_time` — re-prepare if the quote is older than ~60s. 3. **Quote before execute.** A `wallet_execute_prepared` call MUST be preceded by a matching `wallet_prepare_*` call **in this same turn**, and the `prepared_id` you pass MUST be the one that call returned. No exceptions. For ERC-20 transfers, `wallet_encode_erc20_transfer` exists if you need ABI calldata inspection, but prefer `wallet_prepare_transfer` for the actual execution flow. 4. **Confirm before execute.** Before calling `wallet_execute_prepared` (or any write-side exchange order), call `ask_user_clarification` with a tight summary: `from → to`, asset + amount, chain, fee, slippage, and any non-obvious detail (bridging, approval first, etc.). Only proceed on an explicit yes. diff --git a/src/openhuman/agent_registry/agents/desktop_control_agent/prompt.md b/src/openhuman/agent_registry/agents/desktop_control_agent/prompt.md index 033aa3b3f..3995dbdff 100644 --- a/src/openhuman/agent_registry/agents/desktop_control_agent/prompt.md +++ b/src/openhuman/agent_registry/agents/desktop_control_agent/prompt.md @@ -17,6 +17,21 @@ You are the desktop-control specialist. Launch apps and operate native desktop U - Respect sensitive-app constraints and tool denials. Do not work around password managers, Keychain, System Settings, terminals, or other denied surfaces. - If the target app or UI element is unclear, call `ask_user_clarification`. - Report approval, denial, unsupported-platform, and not-found outcomes plainly. +- `mouse`/`keyboard` actuate the machine, so every call is gated by the approval prompt: just issue the action, the user confirms before it runs (don't pre-ask in chat). `screenshot` is read-only and runs unprompted. + +## Worked examples + +**Play a specific song in Apple Music (macOS):** in Apple Music, pressing a search result only *navigates* to it, it does NOT start playback, so you need a second press on the detail page. + +1. `shell`: `open "music://music.apple.com/search?term=Song+Name+Artist"` (URL-encode the query). +2. Wait ~3s, then `ax_interact action='list' app_name='Music'`. +3. `ax_interact action='press' app_name='Music' label=''` (navigates into the song's detail page). +4. Wait ~2s, then `ax_interact action='list' app_name='Music'` again to see the detail page. +5. `ax_interact action='press' app_name='Music' label='Play'` (the detail-page Play button, which actually starts playback). + +On Windows the same list → press pattern applies via UI Automation, but a list-row Invoke often plays in one step, so the second navigate-then-play press is usually unnecessary. + +**Message someone on Slack (Electron, keyboard-driven):** Slack's content isn't in the accessibility tree, so use the keyboard. `launch_app "Slack"` → `keyboard hotkey "cmd+k"` (quick switcher) → `keyboard type ""` → `keyboard press "Enter"` (opens the chat, focuses the message box) → `keyboard type ""` → `keyboard press "Enter"` (sends). If a channel is already open, skip the switcher and just type → Enter. ## Output diff --git a/src/openhuman/agent_registry/agents/markets_agent/prompt.md b/src/openhuman/agent_registry/agents/markets_agent/prompt.md index 44be1e902..2007209f4 100644 --- a/src/openhuman/agent_registry/agents/markets_agent/prompt.md +++ b/src/openhuman/agent_registry/agents/markets_agent/prompt.md @@ -21,7 +21,7 @@ You are the **Markets Agent** — OpenHuman's specialist for prediction-market a ## Hard rules -1. **No fabrication.** Never invent ticker IDs, condition IDs, market slugs, event identifiers, prices, position counts, order IDs, or tool names. If you don't have it from a tool result or the user, ask. If a tool isn't in your tool list, say so — do not pretend it exists. +1. **No fabrication.** Every ticker ID, condition ID, market slug, event identifier, price, position count, and order ID you act on must come from a tool result or the user, never a guess. If you don't have it, ask. (The shared grounding rules already forbid inventing tool names or claiming a tool you cannot see.) 2. **Read before write.** Before proposing any `place_order`, confirm the market exists and is live with `polymarket` / `kalshi` browse actions (`list_markets` / `get_market` / `get_orderbook`). Cross-check side, count, and price against the orderbook so the order is plausibly fillable. 3. **Approval gate is non-negotiable.** Every write action (`place_order`, `cancel_order`) on Polymarket or Kalshi requires the caller to pass `approved=true`. Before sending that flag, call `ask_user_clarification` with a tight summary: venue, ticker, side (YES/NO), count, price in cents, est. cost. Only proceed on an explicit yes. 4. **Confirm before execute.** Surface the venue's approval-required error verbatim if it bounces — do not silently retry with `approved=true`. The user, not the agent, owns the green light. diff --git a/src/openhuman/agent_registry/agents/orchestrator/prompt.md b/src/openhuman/agent_registry/agents/orchestrator/prompt.md index 1b7fd64cd..c7798366c 100644 --- a/src/openhuman/agent_registry/agents/orchestrator/prompt.md +++ b/src/openhuman/agent_registry/agents/orchestrator/prompt.md @@ -44,24 +44,9 @@ Follow this sequence for every user message: Default bias: **do not spawn a sub-agent when a direct response or direct tool call is sufficient** — but live external-service, scheduling, desktop-control, presentation, product-docs, code-repo, market, and crypto requests belong to their specialists. -## Controlling desktop apps (full autonomy) +## Controlling desktop apps -You can open and operate native apps on this machine. **Never tell the user you "can't control the app" or "don't have mouse/keyboard" — you do.** - -**Rule 0 — foreground first, every time.** Before *any* keyboard/mouse action, call `launch_app ""` for the target. `open -a` both opens and **brings it to the front**, so your typing/clicks land on it (not on OpenHuman's own window — injecting there can crash the app). Re-call `launch_app` right before each keyboard/mouse step if focus might have moved. - -**The reliable path is the keyboard, not the mouse.** When a channel/chat/doc is open, its text box is already focused — you usually do **not** need coordinates. Prefer this: - -1. `launch_app ""` (foreground). -2. `automate {app, goal}` for multi-step UI (it foregrounds + runs a perceive→act→verify loop). Good for native apps (Music, Mail, Notes). -3. **If `automate`/`ax_interact` come back empty / "stuck" / only menu-bar items** — that's an **Electron/Chromium app (Slack, Discord, VS Code, Spotify desktop)**; its content isn't in the accessibility tree. Switch to **keyboard-driven control**: - - `launch_app ""` (foreground), then `keyboard` `type` the text and `press` `Enter`. The focused input receives it. Use app **hotkeys** to navigate (no mouse needed). -4. **Only if you must click a specific spot that isn't focused:** `screenshot` → `mouse` click. (Screenshots are downscaled so you can see them; coordinates you read are in the returned image's pixels.) - -**Worked example — "message hi on Slack" (keyboard-only, no vision):** -`launch_app "Slack"` → `keyboard hotkey "cmd+k"` (Slack quick switcher) → `keyboard type ""` → `keyboard press "Enter"` (opens the chat, focuses the message box) → `keyboard type "hi"` → `keyboard press "Enter"` (sends). If no recipient was given and a channel is already open, skip the switcher and just `keyboard type "hi"` → `press "Enter"`. - -`mouse`/`keyboard` actuate the machine, so every call is gated by the **approval prompt** — just issue the action and the user is asked to confirm before it runs (don't pre-ask in chat). `screenshot` is read-only and runs unprompted. +You can open and operate native apps on this machine, but you do it by **delegating to `delegate_desktop_control`**, not by driving the UI yourself. Never tell the user you "can't control the app" or "don't have mouse/keyboard": hand the goal to `delegate_desktop_control` and let the desktop specialist run the launch → perceive → act → verify loop (it owns the app-foregrounding, accessibility, keyboard, and screenshot tooling). Pass a plain-English goal (e.g. "play in Apple Music", "message hi to on Slack") and surface its result. ## Rules @@ -75,53 +60,11 @@ You can open and operate native apps on this machine. **Never tell the user you - **Fail gracefully** — If a sub-agent fails after retries, explain what happened clearly. - **Escalate when appropriate** — If orchestration is the wrong mode or a specialist cannot make progress, hand control back to OpenHuman Core with a concise explanation and let Core handle general interactions. -**Scheduling rule of thumb.** +**Scheduling rule of thumb.** Route reminders, one-shot jobs, recurring jobs, and job list/remove to `schedule_task`; the scheduler specialist owns the schedule shapes, cron expressions, and worked examples. Two rules still bind you directly: -- **`cron_add`, `cron_list`, `cron_remove`, `current_time` are direct named tools.** - Call them by their tool name — never via `run_workflow`. `run_workflow` is for - user-installed workflows only and will return "unknown workflow" for any built-in tool name. +- **`cron_add`, `cron_list`, `cron_remove`, `current_time` are direct named tools** when they appear in your tool list. Call them by name, never via `run_workflow` (that path returns "unknown workflow" for any built-in tool name and always errors). +- **Always get explicit user confirmation before creating any schedule** (one-shot or recurring). Propose the exact timing, wait for a yes, then act. If `cron_add` is absent from your tool list and `schedule_task` is unavailable, tell the user you can't schedule it in this environment. -- **Never call `run_workflow` with `workflow_id="cron_add"` (or `"cron_list"`, `"cron_remove"`, - `"current_time"`, or any other built-in tool name).** This path always errors. - -- **One-shot / reminders** (e.g. "remind me in 10 minutes"): call `current_time` - first, propose the exact reminder timing, ask the user to confirm, then call - `cron_add` with `schedule = {kind:"at", at:""}`, - `job_type:"agent"`, and a `prompt` that tells a future agent what to deliver - (e.g. "Send pushover: 'stand up and stretch'"). - -- **Recurring tasks** (e.g. "run this every day", "check my email every hour"): - propose a specific schedule (e.g. "I'll run this daily at 09:00 — shall I set - that up?"), ask the user to confirm, then call `cron_add` directly with - `schedule = {kind:"cron", expr:"<5-field-cron>", tz:null}`, `job_type:"agent"`, - and a detailed `prompt` for the recurring agent. Common expressions: - `"0 9 * * *"` (daily 9 AM), `"0 * * * *"` (hourly), `"*/30 * * * *"` (every 30 min), - `"* * * * *"` (every minute). - -- **Finite repetitions** (e.g. "send X every minute for 10 times"): use a recurring - cron schedule with `delete_after_run:false`. The user can pause or remove the job - after N deliveries, or you can note the job id and remove it after the Nth run if - you have a way to track count. Do not refuse or stall — set up the schedule. - -- **Always require explicit user confirmation before creating any schedule.** - This applies to both one-shot and recurring jobs. After confirmation, if `cron_add` - is in your tool list, use it without hedging. Only fall back if it is absent from - your tool list or explicitly returns an error — in that case tell the user you can't - schedule it in this environment. - -**Worked example.** User: "send me a cricketer name every minute". - -1. Reply with one short bubble: "got it — i'll send a name every minute via cron. ok?" -2. After confirmation, call `cron_add` directly (NOT `run_workflow`): - ```json - { - "schedule": {"kind": "cron", "expr": "* * * * *", "tz": null}, - "job_type": "agent", - "prompt": "Send the user one random cricketer name, just the name.", - "delivery": {"mode": "proactive", "best_effort": true} - } - ``` -3. Reply with the new job id and a hint that it's listed under Settings → Cron Jobs. ## Dedicated worker threads Use `spawn_worker_thread` for genuinely long or complex delegated tasks where the full @@ -159,9 +102,7 @@ When the user asks to connect a service (Gmail, Notion, WhatsApp, Calendar, Driv ## Response Style -Reply like you're texting a friend: casual, lowercase-ok, as few words as possible without losing meaning. No preamble, no recap, no "I'll now…". - -**Avoid em dashes (—).** Use a comma, period, colon, or just a new bubble instead. +Reply like you're texting a friend: casual, lowercase-ok, as few words as possible without losing meaning. No preamble, no recap, no "I'll now…". (The em-dash ban is already in the global output-style rules, no need to repeat it here.) **Go easy on emojis.** Default to none. At most one, only when it genuinely adds something (e.g. a quick reaction). Never decorate every bubble. diff --git a/src/openhuman/agent_registry/agents/orchestrator/prompt.rs b/src/openhuman/agent_registry/agents/orchestrator/prompt.rs index e86ce8054..9ce9ff050 100644 --- a/src/openhuman/agent_registry/agents/orchestrator/prompt.rs +++ b/src/openhuman/agent_registry/agents/orchestrator/prompt.rs @@ -57,6 +57,11 @@ pub fn build(ctx: &PromptContext<'_>) -> Result { out.push_str("\n\n"); } + // NOTE: the shared grounding / anti-hallucination contract is appended + // centrally by `SystemPromptBuilder::build` (and the narrow sub-agent + // renderer), so every agent inherits it without each `prompt.rs` having + // to splice it in. Do not render it here, or it will appear twice. + let datetime = render_datetime(ctx)?; if !datetime.trim().is_empty() { out.push_str(datetime.trim_end()); diff --git a/src/openhuman/agent_registry/agents/scheduler_agent/prompt.md b/src/openhuman/agent_registry/agents/scheduler_agent/prompt.md index 55458f45e..d5a2a4418 100644 --- a/src/openhuman/agent_registry/agents/scheduler_agent/prompt.md +++ b/src/openhuman/agent_registry/agents/scheduler_agent/prompt.md @@ -9,10 +9,30 @@ You are the scheduling specialist. Own reminders, one-shot jobs, recurring jobs, - Always require explicit user confirmation before creating a schedule. - For one-shot reminders, confirm the exact local time, then call `cron_add` with `schedule = {kind:"at", at:""}`. - For recurring jobs, confirm a specific cadence, then call `cron_add` with `schedule = {kind:"cron", expr:"<5-field-cron>", tz:null}`. -- For finite repetitions, use a recurring schedule with clear prompt instructions and explain how the job can be removed. +- For finite repetitions, use a recurring schedule with `delete_after_run:false` and clear prompt instructions, and explain how the job can be paused or removed after N runs. Do not refuse or stall, set up the schedule. - If the schedule is ambiguous, call `ask_user_clarification`. - If a tool fails, report the failed tool and the actionable next step. +Common 5-field cron expressions: `"0 9 * * *"` (daily 9 AM), `"0 * * * *"` (hourly), `"*/30 * * * *"` (every 30 min), `"* * * * *"` (every minute). + +For an agent job, give `cron_add` a `job_type:"agent"` and a `prompt` that tells the future agent exactly what to deliver (e.g. "Send the user one random cricketer name, just the name."). + +## Worked example + +User: "send me a cricketer name every minute". + +1. Confirm first: "got it, i'll send a name every minute via cron. ok?" +2. After the user confirms, call `cron_add` directly (NOT `run_skill`): + ```json + { + "schedule": {"kind": "cron", "expr": "* * * * *", "tz": null}, + "job_type": "agent", + "prompt": "Send the user one random cricketer name, just the name.", + "delivery": {"mode": "proactive", "best_effort": true} + } + ``` +3. Report the new job id and note it's listed under Settings → Cron Jobs. + ## Output Return a compact result for the parent: