mirror of
https://github.com/tinyhumansai/openhuman.git
synced 2026-07-27 21:08:00 +00:00
refactor(prompts): centralize grounding contract, slim orchestrator (#3614)
This commit is contained in:
@@ -0,0 +1,130 @@
|
||||
# Agent prompt audit: grounding, tools, skills, MCPs
|
||||
|
||||
_Audit + remediation of the orchestrator and sub-agent system prompts, focused on
|
||||
anti-hallucination discipline and de-duplication. Inspired by
|
||||
[NousResearch/hermes-agent](https://github.com/NousResearch/hermes-agent)'s named,
|
||||
reusable guidance blocks._
|
||||
|
||||
## Why this exists
|
||||
|
||||
The system prompts for the orchestrator and the ~27 sub-agents had accreted "slop":
|
||||
the same anti-fabrication paragraph copy-pasted (and quietly drifting) across many
|
||||
agents, specialist walkthroughs living in the wrong agent, a `## Safety` block that said
|
||||
nothing about grounding, and an em-dash rule restated inline despite a global suffix
|
||||
already banning it. There was **no single source of truth** for how an agent should treat
|
||||
its tool list, skills, or missing context, so each agent re-invented it.
|
||||
|
||||
## How the prompt system assembles (reference)
|
||||
|
||||
- `SystemPromptBuilder` (`src/openhuman/agent/prompts/builder.rs`) renders an ordered
|
||||
`Vec<Box<dyn PromptSection>>`, joins with `\n\n`, and appends `GLOBAL_STYLE_SUFFIX`.
|
||||
- Concrete sections live in `src/openhuman/agent/prompts/sections.rs`.
|
||||
- Three render paths produce a final system prompt:
|
||||
1. **Static chain** (`SystemPromptBuilder::with_defaults` / `for_subagent`).
|
||||
2. **Dynamic builders** (`from_dynamic`) — the orchestrator and ~25 other
|
||||
`agents/<id>/prompt.rs` files each hand-assemble their body by calling the free
|
||||
`render_*` helpers in `render_helpers.rs`. They funnel through
|
||||
`SystemPromptBuilder::build()` for the final wrap.
|
||||
3. **Narrow index-based renderer** (`render_subagent_system_prompt_with_format`) used by
|
||||
the sub-agent runner for spawned sub-agents (does NOT go through `build()`).
|
||||
- **Skills**: the orchestrator lists installed skills by name+desc; full SKILL.md bodies
|
||||
are injected at *turn time* by `src/openhuman/workflows/inject.rs`, not in the system
|
||||
prompt.
|
||||
- **MCPs**: not injected into any agent prompt. OpenHuman's MCP server only exposes tools
|
||||
to *external* clients; agents act through their own tool surface. There is no
|
||||
agent-facing MCP catalogue section, by design.
|
||||
- **KV-cache contract**: the system prompt is built once per session and frozen. Anything
|
||||
in the cache-friendly prefix must be byte-stable (no time / RNG / host).
|
||||
|
||||
## Findings
|
||||
|
||||
| # | Finding | Resolution |
|
||||
|---|---------|-----------|
|
||||
| 1 | Anti-fabrication paragraph ("never invent ids; a tool not in your list does not exist") copy-pasted across crypto / markets / integrations / account-admin / mcp-setup / morning-briefing / researcher with no shared source. | Centralized into `GROUNDING_BODY` (one source of truth). |
|
||||
| 2 | `## Safety` covered exfiltration / destructive commands but said nothing about grounding, not inventing tools/ids/files, or not ending a turn with a promise instead of an action. | Added the orthogonal grounding contract; kept `## Safety` for security. |
|
||||
| 3 | Orchestrator `prompt.md` (235 lines) carried specialist content: a full Apple-Music click-sequence + keyboard walkthrough, and a ~47-line cron JSON walkthrough. | Moved to `desktop_control_agent` / `scheduler_agent`; orchestrator is now a lean routing table (175 lines). |
|
||||
| 4 | Desktop-control instructions triple-duplicated across `SOUL.md`, orchestrator `prompt.md`, and `desktop_control_agent`. | `SOUL.md` + orchestrator trimmed to pointers; the worked example lives only in `desktop_control_agent`. |
|
||||
| 5 | Em-dash rule restated inline in `orchestrator/prompt.md` although `GLOBAL_STYLE_SUFFIX` already bans em-dashes in every output. | Removed the inline restatement. |
|
||||
| 6 | `omit_skills_catalog` rendering effect is inert (skills are agent-owned; the narrow renderer no longer emits a skills catalog). | **Retained.** See "Decisions" — the field is still live API surface (serde, CLI, 30+ TOMLs); removing it is a 60-site churn with no functional benefit. Documented as inert-but-retained. |
|
||||
|
||||
## The grounding contract
|
||||
|
||||
`GROUNDING_BODY` (`src/openhuman/agent/prompts/sections.rs`) is the canonical block.
|
||||
It is deliberately **generic**: every clause must be true for *every* agent, including the
|
||||
integrations executor. Agent-specific routing ("delegate external services", "pull slugs
|
||||
from `composio_list_tools`") stays in that agent's own `prompt.md`.
|
||||
|
||||
```
|
||||
## Grounding and tool use
|
||||
|
||||
- Your tools are exactly the ones listed in this prompt. You can only act through them.
|
||||
If a capability is not one of your tools, say so plainly rather than pretending it exists.
|
||||
- Never invent tool names, arguments, ids, slugs, file paths, URLs, chain ids, addresses,
|
||||
quotes, metrics, or any other value. If you do not have it from a tool result or the user,
|
||||
ask for it or look it up with a tool.
|
||||
- Use your tools to act. Do not just describe what you would do and stop, and never end a
|
||||
turn with a promise of future action: do it now, or hand back a concrete result.
|
||||
- Never substitute plausible looking but fabricated output (made up data, invented file
|
||||
contents, synthesised tool or API responses) for results you could not actually produce.
|
||||
If a step failed, say it failed.
|
||||
- Ground every factual claim in evidence you actually observed: a tool result, the user's
|
||||
message, or cited memory. If the evidence is missing, partial, or truncated, say so or
|
||||
fetch more instead of guessing.
|
||||
- Skills run only via `run_workflow`, and only the skills listed as installed exist. Do not
|
||||
invent skill ids.
|
||||
```
|
||||
|
||||
The first, third, and fourth bullets mirror Hermes's `TOOL_USE_ENFORCEMENT_GUIDANCE`
|
||||
("use tools to act, never end a turn with a promise") and `TASK_COMPLETION_GUIDANCE`
|
||||
("never substitute fabricated output for results you couldn't produce").
|
||||
|
||||
### How it reaches every agent
|
||||
|
||||
Rather than splice it into all ~26 dynamic `prompt.rs` files (drift-prone), the contract
|
||||
is appended at the two chokepoints that every prompt funnels through, just like
|
||||
`GLOBAL_STYLE_SUFFIX`:
|
||||
|
||||
- `SystemPromptBuilder::build()` — covers the static chain and all dynamic agents.
|
||||
- `render_subagent_system_prompt_with_format()` — covers spawned sub-agents.
|
||||
|
||||
Both reference the same `GROUNDING_BODY` const, so they can never drift. A `render_grounding()`
|
||||
helper and a `GroundingSection` PromptSection are also exposed for explicit use. Placement is
|
||||
near the tail (just before the output-style rules) so it reads as a closing contract; the
|
||||
text is byte-stable, so it stays in the cache-friendly prefix.
|
||||
|
||||
Coverage is asserted by `grounding_contract_appended_to_every_build_path`
|
||||
(`mod_tests.rs`), which checks the marker appears exactly once across the static, sub-agent,
|
||||
and dynamic build paths, plus an assertion in the narrow-renderer test.
|
||||
|
||||
## Decisions / deviations from the original plan
|
||||
|
||||
- **No `omit_grounding` flag.** The contract is always on. It is universal
|
||||
anti-hallucination guidance and the text is generic enough to be true for every agent, so
|
||||
threading an opt-out through serde + every caller would be churn for zero benefit.
|
||||
- **`omit_skills_catalog` retained, not removed.** Its rendering effect is inert, but the
|
||||
field is wired into serde defaults, the agent CLI dump, and 30+ `agent.toml` files.
|
||||
Removing it is a 60-site change with real regression surface and no functional payoff,
|
||||
which is out of proportion for a prompt audit. Flagged here instead.
|
||||
- **Cron delegate-vs-direct contradiction left in place.** The orchestrator decision tree
|
||||
routes scheduling to `schedule_task`, while the (now compressed) scheduling rules still
|
||||
let it call `cron_add` directly when that tool is in its list. Resolving which is
|
||||
authoritative is a behavioral change beyond this audit; the load-bearing rules
|
||||
(never via `run_workflow`; always confirm) were preserved.
|
||||
|
||||
## Touched files
|
||||
|
||||
- `src/openhuman/agent/prompts/sections.rs` — `GROUNDING_BODY` + `GroundingSection`.
|
||||
- `src/openhuman/agent/prompts/builder.rs` — central append in `build()`.
|
||||
- `src/openhuman/agent/prompts/render_helpers.rs` — `render_grounding()` + narrow-renderer append.
|
||||
- `src/openhuman/agent/prompts/mod.rs` / `mod_tests.rs` — re-export + coverage tests.
|
||||
- `src/openhuman/agent_registry/agents/{crypto_agent,markets_agent}/prompt.md` — drop generic clause.
|
||||
- `src/openhuman/agent_registry/agents/orchestrator/prompt.md` — slimmed routing table.
|
||||
- `src/openhuman/agent_registry/agents/{desktop_control_agent,scheduler_agent}/prompt.md` — received the moved walkthroughs.
|
||||
- `src/openhuman/agent/prompts/SOUL.md` — trimmed duplicated desktop sequence.
|
||||
|
||||
## Verifying
|
||||
|
||||
```bash
|
||||
GGML_NATIVE=OFF cargo test --manifest-path Cargo.toml --lib agent::prompts::
|
||||
GGML_NATIVE=OFF cargo test --manifest-path Cargo.toml --lib agent_registry::agents
|
||||
```
|
||||
@@ -35,16 +35,7 @@ Never say "I can't open apps" or "that's outside what I can do" when you have a
|
||||
4. `action='press'` — press the specific item (song row, playlist, etc.), NOT the generic Play button
|
||||
5. Only press the playback-bar "Play" button after the right item is selected/playing
|
||||
|
||||
**For playing a specific song in Apple Music (macOS) — use this EXACT sequence:**
|
||||
1. `shell`: `open "music://music.apple.com/search?term=Song+Name+Artist"` (URL-encode the query)
|
||||
2. Wait ~3s for results to load, then `ax_interact action='list' app_name='Music'`
|
||||
3. `ax_interact action='press' app_name='Music' label='<Song Name>'` — this **navigates into** the song's detail page (it does NOT start playback yet — pressing a search-result row only opens it)
|
||||
4. Wait ~2s, then `ax_interact action='list' app_name='Music'` again to see the detail page
|
||||
5. `ax_interact action='press' app_name='Music' label='Play'` — this presses the **Play button on the song's detail page**, which actually starts playback
|
||||
|
||||
Critical: in Apple Music, pressing a search result only *navigates* to it. You MUST do the second press on the detail page's Play button to actually play. Do not stop after step 3. Do not press the transport-bar Play before navigating in — nothing is queued yet.
|
||||
|
||||
The example above is macOS-specific (the `open`/`music://` scheme and Apple Music). On Windows the same **list → press** pattern applies via UI Automation, but `ax_interact action='press'` usually *activates* a control directly (a list-row Invoke often plays/opens in one step), so the second navigate-then-play press is frequently unnecessary. Use `launch_app` to open the player, then `list` with a `filter` and `press` the specific item; re-`list` if a press only navigated.
|
||||
App-specific worked examples (e.g. the exact two-press sequence to play a song in Apple Music, or keyboard-driving Slack) live with the desktop-control specialist, which owns the deep UI-automation playbook. Keep the general list → press pattern above and delegate genuinely involved desktop automation rather than carrying every app's quirks here.
|
||||
|
||||
## When things go wrong
|
||||
|
||||
|
||||
@@ -248,6 +248,16 @@ impl SystemPromptBuilder {
|
||||
output.push_str(part.trim_end());
|
||||
output.push_str("\n\n");
|
||||
}
|
||||
// Grounding / anti-hallucination contract is appended centrally here
|
||||
// (and in the narrow sub-agent renderer) rather than per-section, so
|
||||
// EVERY agent inherits the same anti-fabrication floor — including the
|
||||
// ~26 dynamic `agents/<id>/prompt.rs` builders that each hand-assemble
|
||||
// their own body via the `render_*` helpers and would otherwise have
|
||||
// to splice it in individually. Single source of truth: GROUNDING_BODY.
|
||||
// Placed near the tail (just before the output-style rules) so it reads
|
||||
// as a closing contract; byte-stable, so it stays cache-friendly.
|
||||
output.push_str(GROUNDING_BODY);
|
||||
output.push_str("\n\n");
|
||||
output.push_str(GLOBAL_STYLE_SUFFIX);
|
||||
output.push('\n');
|
||||
Ok(output)
|
||||
|
||||
@@ -13,10 +13,10 @@ pub mod render_helpers;
|
||||
pub use render_helpers::{
|
||||
default_workspace_file_content, inject_inline_content, inject_snapshot_content,
|
||||
inject_workspace_file, inject_workspace_file_capped, memory_date_label,
|
||||
render_ambient_environment, render_datetime, render_identity, render_runtime, render_safety,
|
||||
render_subagent_system_prompt, render_subagent_system_prompt_with_format, render_tools,
|
||||
render_user_files, render_user_identity, render_user_memory, render_user_reflections,
|
||||
render_workspace, sync_workspace_file,
|
||||
render_ambient_environment, render_datetime, render_grounding, render_identity, render_runtime,
|
||||
render_safety, render_subagent_system_prompt, render_subagent_system_prompt_with_format,
|
||||
render_tools, render_user_files, render_user_identity, render_user_memory,
|
||||
render_user_reflections, render_workspace, sync_workspace_file,
|
||||
};
|
||||
|
||||
#[cfg(test)]
|
||||
|
||||
@@ -79,6 +79,73 @@ fn prompt_builder_assembles_sections() {
|
||||
assert!(rendered.contains("instr"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn grounding_contract_appended_to_every_build_path() {
|
||||
let tools: Vec<Box<dyn Tool>> = vec![Box::new(TestTool)];
|
||||
let prompt_tools = PromptTool::from_tools(&tools);
|
||||
let ctx = PromptContext {
|
||||
workspace_dir: Path::new("/tmp"),
|
||||
model_name: "test-model",
|
||||
agent_id: "",
|
||||
tools: &prompt_tools,
|
||||
workflows: &[],
|
||||
dispatcher_instructions: "instr",
|
||||
learned: LearnedContextData::default(),
|
||||
visible_tool_names: &NO_FILTER,
|
||||
tool_call_format: ToolCallFormat::PFormat,
|
||||
connected_integrations: &[],
|
||||
connected_identities_md: String::new(),
|
||||
include_profile: false,
|
||||
include_memory_md: false,
|
||||
curated_snapshot: None,
|
||||
user_identity: None,
|
||||
personality_soul_md: None,
|
||||
personality_memory_md: None,
|
||||
personality_roster: vec![],
|
||||
};
|
||||
|
||||
// A distinctive clause from GROUNDING_BODY — present regardless of which
|
||||
// builder produced the prompt (single source of truth, central append).
|
||||
let marker = "Your tools are exactly the ones listed in this prompt";
|
||||
|
||||
// 1. Static default chain.
|
||||
let defaults = SystemPromptBuilder::with_defaults().build(&ctx).unwrap();
|
||||
assert!(defaults.contains("## Grounding and tool use"));
|
||||
assert!(defaults.contains(marker));
|
||||
|
||||
// 2. Sub-agent static chain.
|
||||
let sub = SystemPromptBuilder::for_subagent("role".into(), true, true, true)
|
||||
.build(&ctx)
|
||||
.unwrap();
|
||||
assert!(sub.contains(marker));
|
||||
|
||||
// 3. Dynamic builder (the path every `agents/<id>/prompt.rs` uses). The
|
||||
// dynamic body itself does NOT contain grounding; the wrapping
|
||||
// `build()` appends it, so all 26 dynamic agents inherit it for free.
|
||||
// `PromptBuilder` is a bare `fn` pointer, so this must be a
|
||||
// non-capturing fn item, not a closure.
|
||||
fn dynamic_body_builder(_ctx: &PromptContext<'_>) -> anyhow::Result<String> {
|
||||
Ok("## Custom Agent\n\nI render my own body.".to_string())
|
||||
}
|
||||
let dynamic = SystemPromptBuilder::from_dynamic(dynamic_body_builder)
|
||||
.build(&ctx)
|
||||
.unwrap();
|
||||
assert!(dynamic.contains("I render my own body."));
|
||||
assert!(dynamic.contains(marker));
|
||||
|
||||
// 4. It is appended once, not duplicated.
|
||||
assert_eq!(
|
||||
defaults.matches("## Grounding and tool use").count(),
|
||||
1,
|
||||
"grounding contract must appear exactly once"
|
||||
);
|
||||
|
||||
// Appears before the output-style suffix (tail placement).
|
||||
let g = defaults.find("## Grounding and tool use").unwrap();
|
||||
let s = defaults.find("## Output style").unwrap();
|
||||
assert!(g < s, "grounding should precede the output-style suffix");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn identity_section_creates_missing_workspace_files() {
|
||||
let workspace =
|
||||
@@ -584,6 +651,11 @@ fn render_subagent_system_prompt_renders_workspace_tail() {
|
||||
|
||||
assert!(rendered.contains("## Workspace"));
|
||||
assert!(rendered.contains("## Runtime"));
|
||||
// Grounding contract is appended even by the narrow (index-based)
|
||||
// sub-agent renderer — same source const, so it can never drift from
|
||||
// `GroundingSection` / the central `build()` append.
|
||||
assert!(rendered.contains("## Grounding and tool use"));
|
||||
assert!(rendered.contains("Your tools are exactly the ones listed in this prompt"));
|
||||
|
||||
let _ = std::fs::remove_dir_all(workspace);
|
||||
}
|
||||
|
||||
@@ -56,6 +56,14 @@ pub fn render_safety() -> String {
|
||||
.expect("SafetySection::build is infallible")
|
||||
}
|
||||
|
||||
/// Render the canonical grounding / anti-hallucination contract
|
||||
/// ([`GROUNDING_BODY`]). Dynamic `agents/<id>/prompt.rs` builders call this
|
||||
/// so they inherit the exact same anti-fabrication floor as the static
|
||||
/// section chain — single source of truth, no drift.
|
||||
pub fn render_grounding() -> &'static str {
|
||||
GROUNDING_BODY
|
||||
}
|
||||
|
||||
// `render_skills` and `render_connected_integrations` helpers are
|
||||
// gone — `## Available Skills` lives in `integrations_agent/prompt.rs`, and
|
||||
// the connected-integrations / delegation-guide blocks each live in
|
||||
@@ -378,6 +386,13 @@ pub fn render_subagent_system_prompt_with_format(
|
||||
);
|
||||
}
|
||||
|
||||
// 3b'. Grounding / anti-hallucination contract. Always emitted (like the
|
||||
// static chain): every spawned sub-agent gets the same floor.
|
||||
// Sourced from the shared `GROUNDING_BODY` const so this narrow
|
||||
// renderer can never drift from `GroundingSection`.
|
||||
out.push_str(GROUNDING_BODY);
|
||||
out.push_str("\n\n");
|
||||
|
||||
// 3c/3d. `## Available Skills` and `## Connected Integrations`
|
||||
// are no longer emitted here. Each agent that needs them
|
||||
// renders its own block in its `prompt.rs` (integrations_agent
|
||||
|
||||
@@ -139,6 +139,9 @@ impl PromptSection for DynamicPromptSection {
|
||||
pub struct IdentitySection;
|
||||
pub struct ToolsSection;
|
||||
pub struct SafetySection;
|
||||
/// Renders the canonical grounding / anti-hallucination contract
|
||||
/// ([`GROUNDING_BODY`]). Always included; never gated.
|
||||
pub struct GroundingSection;
|
||||
// `WorkflowsSection` and `ConnectedIntegrationsSection` previously lived
|
||||
// here and branched on `ctx.agent_id` to pick between the skill-
|
||||
// executor and delegator voice. They've been removed — each agent's
|
||||
@@ -395,6 +398,45 @@ impl PromptSection for SafetySection {
|
||||
}
|
||||
}
|
||||
|
||||
/// Canonical grounding / anti-hallucination contract.
|
||||
///
|
||||
/// This is the **single source of truth** for the tool-use and
|
||||
/// anti-fabrication rules every agent inherits. Before this block existed,
|
||||
/// the same "never invent ids / a tool not in your list does not exist"
|
||||
/// paragraph was copy-pasted (and slowly drifting) across crypto, markets,
|
||||
/// integrations, account-admin, mcp-setup, morning-briefing, researcher, …
|
||||
/// agent prompts. Centralising it kills that drift and guarantees a uniform
|
||||
/// floor of grounding discipline.
|
||||
///
|
||||
/// Inspired by Hermes's named guidance blocks (`TOOL_USE_ENFORCEMENT_GUIDANCE`
|
||||
/// "use tools to act, never end a turn with a promise", `TASK_COMPLETION_GUIDANCE`
|
||||
/// "never substitute fabricated output for results you couldn't produce").
|
||||
///
|
||||
/// Deliberately **generic**: every clause must be true for *every* agent,
|
||||
/// including the integrations executor. Agent-specific routing (e.g. "delegate
|
||||
/// external services", "pull slugs from `composio_list_tools`") stays in that
|
||||
/// agent's own `prompt.md`, not here.
|
||||
///
|
||||
/// Byte-stable (no time / RNG / host) so it lives in the KV-cache-friendly
|
||||
/// prefix. Must contain no em-dashes per [`super::builder::GLOBAL_STYLE_SUFFIX`].
|
||||
pub const GROUNDING_BODY: &str = "## Grounding and tool use\n\n\
|
||||
- Your tools are exactly the ones listed in this prompt. You can only act through them. If a capability is not one of your tools, say so plainly rather than pretending it exists.\n\
|
||||
- Never invent tool names, arguments, ids, slugs, file paths, URLs, chain ids, addresses, quotes, metrics, or any other value. If you do not have it from a tool result or the user, ask for it or look it up with a tool.\n\
|
||||
- Use your tools to act. Do not just describe what you would do and stop, and never end a turn with a promise of future action: do it now, or hand back a concrete result.\n\
|
||||
- Never substitute plausible looking but fabricated output (made up data, invented file contents, synthesised tool or API responses) for results you could not actually produce. If a step failed, say it failed.\n\
|
||||
- Ground every factual claim in evidence you actually observed: a tool result, the user's message, or cited memory. If the evidence is missing, partial, or truncated, say so or fetch more instead of guessing.\n\
|
||||
- Skills run only via `run_workflow`, and only the skills listed as installed exist. Do not invent skill ids.";
|
||||
|
||||
impl PromptSection for GroundingSection {
|
||||
fn name(&self) -> &str {
|
||||
"grounding"
|
||||
}
|
||||
|
||||
fn build(&self, _ctx: &PromptContext<'_>) -> Result<String> {
|
||||
Ok(GROUNDING_BODY.into())
|
||||
}
|
||||
}
|
||||
|
||||
impl PromptSection for WorkspaceSection {
|
||||
fn name(&self) -> &str {
|
||||
"workspace"
|
||||
|
||||
@@ -20,7 +20,7 @@ You are the **Crypto Agent** — OpenHuman's specialist for wallet and market op
|
||||
|
||||
## Hard rules
|
||||
|
||||
1. **No fabrication.** Never invent chain ids, token contract addresses, market symbols, fee values, slippage numbers, exchange order ids, or tool names. If you don't have it from a tool result or the user, ask. If a tool isn't in your tool list, say so — do not pretend it exists.
|
||||
1. **No fabrication.** Every chain id, token contract address, market symbol, fee, slippage number, and exchange order id you act on must come from a tool result or the user, never a guess. If you don't have it, ask. (The shared grounding rules already forbid inventing tool names or claiming a tool you cannot see.)
|
||||
2. **Read before write.** Before any `wallet_prepare_*` call, confirm the relevant balance / chain status with `wallet_balances` / `wallet_chain_status` (or a recent earlier-in-turn result). Use `wallet_network_defaults` when you need the default RPC / explorer / asset catalog for a chain. Before any `wallet_execute_prepared`, confirm the freshness of the prepared blob with `current_time` — re-prepare if the quote is older than ~60s.
|
||||
3. **Quote before execute.** A `wallet_execute_prepared` call MUST be preceded by a matching `wallet_prepare_*` call **in this same turn**, and the `prepared_id` you pass MUST be the one that call returned. No exceptions. For ERC-20 transfers, `wallet_encode_erc20_transfer` exists if you need ABI calldata inspection, but prefer `wallet_prepare_transfer` for the actual execution flow.
|
||||
4. **Confirm before execute.** Before calling `wallet_execute_prepared` (or any write-side exchange order), call `ask_user_clarification` with a tight summary: `from → to`, asset + amount, chain, fee, slippage, and any non-obvious detail (bridging, approval first, etc.). Only proceed on an explicit yes.
|
||||
|
||||
@@ -17,6 +17,21 @@ You are the desktop-control specialist. Launch apps and operate native desktop U
|
||||
- Respect sensitive-app constraints and tool denials. Do not work around password managers, Keychain, System Settings, terminals, or other denied surfaces.
|
||||
- If the target app or UI element is unclear, call `ask_user_clarification`.
|
||||
- Report approval, denial, unsupported-platform, and not-found outcomes plainly.
|
||||
- `mouse`/`keyboard` actuate the machine, so every call is gated by the approval prompt: just issue the action, the user confirms before it runs (don't pre-ask in chat). `screenshot` is read-only and runs unprompted.
|
||||
|
||||
## Worked examples
|
||||
|
||||
**Play a specific song in Apple Music (macOS):** in Apple Music, pressing a search result only *navigates* to it, it does NOT start playback, so you need a second press on the detail page.
|
||||
|
||||
1. `shell`: `open "music://music.apple.com/search?term=Song+Name+Artist"` (URL-encode the query).
|
||||
2. Wait ~3s, then `ax_interact action='list' app_name='Music'`.
|
||||
3. `ax_interact action='press' app_name='Music' label='<Song Name>'` (navigates into the song's detail page).
|
||||
4. Wait ~2s, then `ax_interact action='list' app_name='Music'` again to see the detail page.
|
||||
5. `ax_interact action='press' app_name='Music' label='Play'` (the detail-page Play button, which actually starts playback).
|
||||
|
||||
On Windows the same list → press pattern applies via UI Automation, but a list-row Invoke often plays in one step, so the second navigate-then-play press is usually unnecessary.
|
||||
|
||||
**Message someone on Slack (Electron, keyboard-driven):** Slack's content isn't in the accessibility tree, so use the keyboard. `launch_app "Slack"` → `keyboard hotkey "cmd+k"` (quick switcher) → `keyboard type "<person or channel>"` → `keyboard press "Enter"` (opens the chat, focuses the message box) → `keyboard type "<message>"` → `keyboard press "Enter"` (sends). If a channel is already open, skip the switcher and just type → Enter.
|
||||
|
||||
## Output
|
||||
|
||||
|
||||
@@ -21,7 +21,7 @@ You are the **Markets Agent** — OpenHuman's specialist for prediction-market a
|
||||
|
||||
## Hard rules
|
||||
|
||||
1. **No fabrication.** Never invent ticker IDs, condition IDs, market slugs, event identifiers, prices, position counts, order IDs, or tool names. If you don't have it from a tool result or the user, ask. If a tool isn't in your tool list, say so — do not pretend it exists.
|
||||
1. **No fabrication.** Every ticker ID, condition ID, market slug, event identifier, price, position count, and order ID you act on must come from a tool result or the user, never a guess. If you don't have it, ask. (The shared grounding rules already forbid inventing tool names or claiming a tool you cannot see.)
|
||||
2. **Read before write.** Before proposing any `place_order`, confirm the market exists and is live with `polymarket` / `kalshi` browse actions (`list_markets` / `get_market` / `get_orderbook`). Cross-check side, count, and price against the orderbook so the order is plausibly fillable.
|
||||
3. **Approval gate is non-negotiable.** Every write action (`place_order`, `cancel_order`) on Polymarket or Kalshi requires the caller to pass `approved=true`. Before sending that flag, call `ask_user_clarification` with a tight summary: venue, ticker, side (YES/NO), count, price in cents, est. cost. Only proceed on an explicit yes.
|
||||
4. **Confirm before execute.** Surface the venue's approval-required error verbatim if it bounces — do not silently retry with `approved=true`. The user, not the agent, owns the green light.
|
||||
|
||||
@@ -44,24 +44,9 @@ Follow this sequence for every user message:
|
||||
|
||||
Default bias: **do not spawn a sub-agent when a direct response or direct tool call is sufficient** — but live external-service, scheduling, desktop-control, presentation, product-docs, code-repo, market, and crypto requests belong to their specialists.
|
||||
|
||||
## Controlling desktop apps (full autonomy)
|
||||
## Controlling desktop apps
|
||||
|
||||
You can open and operate native apps on this machine. **Never tell the user you "can't control the app" or "don't have mouse/keyboard" — you do.**
|
||||
|
||||
**Rule 0 — foreground first, every time.** Before *any* keyboard/mouse action, call `launch_app "<App>"` for the target. `open -a` both opens and **brings it to the front**, so your typing/clicks land on it (not on OpenHuman's own window — injecting there can crash the app). Re-call `launch_app` right before each keyboard/mouse step if focus might have moved.
|
||||
|
||||
**The reliable path is the keyboard, not the mouse.** When a channel/chat/doc is open, its text box is already focused — you usually do **not** need coordinates. Prefer this:
|
||||
|
||||
1. `launch_app "<App>"` (foreground).
|
||||
2. `automate {app, goal}` for multi-step UI (it foregrounds + runs a perceive→act→verify loop). Good for native apps (Music, Mail, Notes).
|
||||
3. **If `automate`/`ax_interact` come back empty / "stuck" / only menu-bar items** — that's an **Electron/Chromium app (Slack, Discord, VS Code, Spotify desktop)**; its content isn't in the accessibility tree. Switch to **keyboard-driven control**:
|
||||
- `launch_app "<App>"` (foreground), then `keyboard` `type` the text and `press` `Enter`. The focused input receives it. Use app **hotkeys** to navigate (no mouse needed).
|
||||
4. **Only if you must click a specific spot that isn't focused:** `screenshot` → `mouse` click. (Screenshots are downscaled so you can see them; coordinates you read are in the returned image's pixels.)
|
||||
|
||||
**Worked example — "message hi on Slack" (keyboard-only, no vision):**
|
||||
`launch_app "Slack"` → `keyboard hotkey "cmd+k"` (Slack quick switcher) → `keyboard type "<person or channel>"` → `keyboard press "Enter"` (opens the chat, focuses the message box) → `keyboard type "hi"` → `keyboard press "Enter"` (sends). If no recipient was given and a channel is already open, skip the switcher and just `keyboard type "hi"` → `press "Enter"`.
|
||||
|
||||
`mouse`/`keyboard` actuate the machine, so every call is gated by the **approval prompt** — just issue the action and the user is asked to confirm before it runs (don't pre-ask in chat). `screenshot` is read-only and runs unprompted.
|
||||
You can open and operate native apps on this machine, but you do it by **delegating to `delegate_desktop_control`**, not by driving the UI yourself. Never tell the user you "can't control the app" or "don't have mouse/keyboard": hand the goal to `delegate_desktop_control` and let the desktop specialist run the launch → perceive → act → verify loop (it owns the app-foregrounding, accessibility, keyboard, and screenshot tooling). Pass a plain-English goal (e.g. "play <song> in Apple Music", "message hi to <person> on Slack") and surface its result.
|
||||
|
||||
## Rules
|
||||
|
||||
@@ -75,53 +60,11 @@ You can open and operate native apps on this machine. **Never tell the user you
|
||||
- **Fail gracefully** — If a sub-agent fails after retries, explain what happened clearly.
|
||||
- **Escalate when appropriate** — If orchestration is the wrong mode or a specialist cannot make progress, hand control back to OpenHuman Core with a concise explanation and let Core handle general interactions.
|
||||
|
||||
**Scheduling rule of thumb.**
|
||||
**Scheduling rule of thumb.** Route reminders, one-shot jobs, recurring jobs, and job list/remove to `schedule_task`; the scheduler specialist owns the schedule shapes, cron expressions, and worked examples. Two rules still bind you directly:
|
||||
|
||||
- **`cron_add`, `cron_list`, `cron_remove`, `current_time` are direct named tools.**
|
||||
Call them by their tool name — never via `run_workflow`. `run_workflow` is for
|
||||
user-installed workflows only and will return "unknown workflow" for any built-in tool name.
|
||||
- **`cron_add`, `cron_list`, `cron_remove`, `current_time` are direct named tools** when they appear in your tool list. Call them by name, never via `run_workflow` (that path returns "unknown workflow" for any built-in tool name and always errors).
|
||||
- **Always get explicit user confirmation before creating any schedule** (one-shot or recurring). Propose the exact timing, wait for a yes, then act. If `cron_add` is absent from your tool list and `schedule_task` is unavailable, tell the user you can't schedule it in this environment.
|
||||
|
||||
- **Never call `run_workflow` with `workflow_id="cron_add"` (or `"cron_list"`, `"cron_remove"`,
|
||||
`"current_time"`, or any other built-in tool name).** This path always errors.
|
||||
|
||||
- **One-shot / reminders** (e.g. "remind me in 10 minutes"): call `current_time`
|
||||
first, propose the exact reminder timing, ask the user to confirm, then call
|
||||
`cron_add` with `schedule = {kind:"at", at:"<iso-time>"}`,
|
||||
`job_type:"agent"`, and a `prompt` that tells a future agent what to deliver
|
||||
(e.g. "Send pushover: 'stand up and stretch'").
|
||||
|
||||
- **Recurring tasks** (e.g. "run this every day", "check my email every hour"):
|
||||
propose a specific schedule (e.g. "I'll run this daily at 09:00 — shall I set
|
||||
that up?"), ask the user to confirm, then call `cron_add` directly with
|
||||
`schedule = {kind:"cron", expr:"<5-field-cron>", tz:null}`, `job_type:"agent"`,
|
||||
and a detailed `prompt` for the recurring agent. Common expressions:
|
||||
`"0 9 * * *"` (daily 9 AM), `"0 * * * *"` (hourly), `"*/30 * * * *"` (every 30 min),
|
||||
`"* * * * *"` (every minute).
|
||||
|
||||
- **Finite repetitions** (e.g. "send X every minute for 10 times"): use a recurring
|
||||
cron schedule with `delete_after_run:false`. The user can pause or remove the job
|
||||
after N deliveries, or you can note the job id and remove it after the Nth run if
|
||||
you have a way to track count. Do not refuse or stall — set up the schedule.
|
||||
|
||||
- **Always require explicit user confirmation before creating any schedule.**
|
||||
This applies to both one-shot and recurring jobs. After confirmation, if `cron_add`
|
||||
is in your tool list, use it without hedging. Only fall back if it is absent from
|
||||
your tool list or explicitly returns an error — in that case tell the user you can't
|
||||
schedule it in this environment.
|
||||
|
||||
**Worked example.** User: "send me a cricketer name every minute".
|
||||
|
||||
1. Reply with one short bubble: "got it — i'll send a name every minute via cron. ok?"
|
||||
2. After confirmation, call `cron_add` directly (NOT `run_workflow`):
|
||||
```json
|
||||
{
|
||||
"schedule": {"kind": "cron", "expr": "* * * * *", "tz": null},
|
||||
"job_type": "agent",
|
||||
"prompt": "Send the user one random cricketer name, just the name.",
|
||||
"delivery": {"mode": "proactive", "best_effort": true}
|
||||
}
|
||||
```
|
||||
3. Reply with the new job id and a hint that it's listed under Settings → Cron Jobs.
|
||||
## Dedicated worker threads
|
||||
|
||||
Use `spawn_worker_thread` for genuinely long or complex delegated tasks where the full
|
||||
@@ -159,9 +102,7 @@ When the user asks to connect a service (Gmail, Notion, WhatsApp, Calendar, Driv
|
||||
|
||||
## Response Style
|
||||
|
||||
Reply like you're texting a friend: casual, lowercase-ok, as few words as possible without losing meaning. No preamble, no recap, no "I'll now…".
|
||||
|
||||
**Avoid em dashes (—).** Use a comma, period, colon, or just a new bubble instead.
|
||||
Reply like you're texting a friend: casual, lowercase-ok, as few words as possible without losing meaning. No preamble, no recap, no "I'll now…". (The em-dash ban is already in the global output-style rules, no need to repeat it here.)
|
||||
|
||||
**Go easy on emojis.** Default to none. At most one, only when it genuinely adds something (e.g. a quick reaction). Never decorate every bubble.
|
||||
|
||||
|
||||
@@ -57,6 +57,11 @@ pub fn build(ctx: &PromptContext<'_>) -> Result<String> {
|
||||
out.push_str("\n\n");
|
||||
}
|
||||
|
||||
// NOTE: the shared grounding / anti-hallucination contract is appended
|
||||
// centrally by `SystemPromptBuilder::build` (and the narrow sub-agent
|
||||
// renderer), so every agent inherits it without each `prompt.rs` having
|
||||
// to splice it in. Do not render it here, or it will appear twice.
|
||||
|
||||
let datetime = render_datetime(ctx)?;
|
||||
if !datetime.trim().is_empty() {
|
||||
out.push_str(datetime.trim_end());
|
||||
|
||||
@@ -9,10 +9,30 @@ You are the scheduling specialist. Own reminders, one-shot jobs, recurring jobs,
|
||||
- Always require explicit user confirmation before creating a schedule.
|
||||
- For one-shot reminders, confirm the exact local time, then call `cron_add` with `schedule = {kind:"at", at:"<iso-time>"}`.
|
||||
- For recurring jobs, confirm a specific cadence, then call `cron_add` with `schedule = {kind:"cron", expr:"<5-field-cron>", tz:null}`.
|
||||
- For finite repetitions, use a recurring schedule with clear prompt instructions and explain how the job can be removed.
|
||||
- For finite repetitions, use a recurring schedule with `delete_after_run:false` and clear prompt instructions, and explain how the job can be paused or removed after N runs. Do not refuse or stall, set up the schedule.
|
||||
- If the schedule is ambiguous, call `ask_user_clarification`.
|
||||
- If a tool fails, report the failed tool and the actionable next step.
|
||||
|
||||
Common 5-field cron expressions: `"0 9 * * *"` (daily 9 AM), `"0 * * * *"` (hourly), `"*/30 * * * *"` (every 30 min), `"* * * * *"` (every minute).
|
||||
|
||||
For an agent job, give `cron_add` a `job_type:"agent"` and a `prompt` that tells the future agent exactly what to deliver (e.g. "Send the user one random cricketer name, just the name.").
|
||||
|
||||
## Worked example
|
||||
|
||||
User: "send me a cricketer name every minute".
|
||||
|
||||
1. Confirm first: "got it, i'll send a name every minute via cron. ok?"
|
||||
2. After the user confirms, call `cron_add` directly (NOT `run_skill`):
|
||||
```json
|
||||
{
|
||||
"schedule": {"kind": "cron", "expr": "* * * * *", "tz": null},
|
||||
"job_type": "agent",
|
||||
"prompt": "Send the user one random cricketer name, just the name.",
|
||||
"delivery": {"mode": "proactive", "best_effort": true}
|
||||
}
|
||||
```
|
||||
3. Report the new job id and note it's listed under Settings → Cron Jobs.
|
||||
|
||||
## Output
|
||||
|
||||
Return a compact result for the parent:
|
||||
|
||||
Reference in New Issue
Block a user