From 15df9855c35a8188dd08890515517b963c6ad1ca Mon Sep 17 00:00:00 2001 From: Aqil Aziz Date: Thu, 21 May 2026 05:21:23 +0700 Subject: [PATCH] fix(orchestrator): route live facts to research ## Summary - Routes live/current factual requests (weather, forecasts, recent web facts, "use Grok/live data") to the available `research` tool. - Replaces the stale `delegate_researcher` prompt reference with the synthesized tool name `research`. - Updates the chat harness subagent fixture to emit `research` so the forced tool call matches the current delegation surface. - Adds a prompt-builder regression test that locks the live-data rule and rejects the stale tool name. ## Problem - The orchestrator could acknowledge live weather/research requests with "on it" without actually calling the research tool. - The prompt named `delegate_researcher`, but the current researcher tool is exposed as `research` via `delegate_name = "research"`. - Users who named an unwired provider like Grok could stall the agent instead of falling back to the available live research path. ## Solution - Teach the orchestrator to call `research` for live/current/time-sensitive facts that direct tools cannot answer. - Explicitly tell it not to stop at "on it" and not to wait for the exact named provider when that provider is not wired in. - Align the E2E harness fixture with the real `research` tool name. ## Submission Checklist - [x] Tests added or updated (happy path + at least one failure / edge case) per [Testing Strategy](../gitbooks/developing/testing-strategy.md#failure-path-requirement) - added prompt regression assertion and updated E2E forced tool fixture. - [x] **Diff coverage >= 80%** - N/A locally: tiny prompt/test fixture change; CI diff coverage will be authoritative. - [x] Coverage matrix updated - N/A: prompt routing change, no feature row added/removed/renamed. - [x] All affected feature IDs from the matrix are listed in the PR description under `## Related` - N/A: no matrix feature ID applies. - [x] No new external network dependencies introduced (mock backend used per [Testing Strategy](../gitbooks/developing/testing-strategy.md#mock-policy)) - [x] Manual smoke checklist updated if this touches release-cut surfaces ([`docs/RELEASE-MANUAL-SMOKE.md`](../docs/RELEASE-MANUAL-SMOKE.md)) - N/A: no release-cut surface. - [x] Linked issue closed via `Closes #NNN` in the `## Related` section ## Impact - Runtime: orchestrator prompt only, plus E2E harness fixture alignment. - User-visible effect: live weather/current fact requests should call research instead of ending with an acknowledgement. - No migration, security, or dependency impact. ## Related - Closes #2164 - Follow-up PR(s)/TODOs: N/A --- ## AI Authored PR Metadata (required for Codex/Linear PRs) ### Linear Issue - Key: N/A - URL: N/A ### Commit & Branch - Branch: `codex/2164-live-data-research` - Commit SHA: `dc63d1d3e1ea531a1609594debd1e0ca95329b27` ### Validation Run - [x] `pnpm --filter openhuman-app exec prettier --check test/e2e/specs/chat-harness-subagent.spec.ts` - passed - [x] `pnpm typecheck` - N/A: comments/string fixture plus Rust prompt text only - [x] Focused tests: `cargo test --lib build_routes_live_facts_to_research_tool` attempted; blocked before test execution by local `libclang` setup - [x] Rust fmt/check (if changed): `cargo fmt --all --check` - passed; `git diff --check` - passed - [x] Tauri fmt/check (if changed): N/A ### Validation Blocked - `command:` `cargo test --lib build_routes_live_facts_to_research_tool` - `error:` `whisper-rs-sys` build could not find `clang.dll` / `libclang.dll`; `LIBCLANG_PATH` is unset in this Windows environment - `impact:` Rust prompt unit test did not execute locally; remote CI environment should validate it ### Behavior Changes - Intended behavior change: live/current factual requests route to `research` when no direct tool can answer. - User-visible effect: requests like "forecast for Bremerhaven today" should produce a researched answer instead of a bare "on it" acknowledgement. ### Parity Contract - Legacy behavior preserved: direct answers, connected integration routing, crypto/code/planner/critic/archive routing remain unchanged. - Guard/fallback/dispatch parity checks: prompt test asserts live facts mention weather/forecast/Grok fallback and rejects stale `delegate_researcher` naming. ### Duplicate / Superseded PR Handling - Duplicate PR(s): N/A - Canonical PR: this PR - Resolution (closed/superseded/updated): N/A ## Summary by CodeRabbit ## Release Notes * **Tests** * Enhanced test coverage for research request handling and subagent communication. * **Chores** * Improved internal routing and terminology for research operations to support live, time-sensitive fact queries more effectively. [![Review Change Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/tinyhumansai/openhuman/pull/2299?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) Co-authored-by: aqilaziz Co-authored-by: Steven Enamakel --- app/test/e2e/specs/chat-harness-subagent.spec.ts | 14 +++++++------- src/openhuman/agent/agents/orchestrator/prompt.md | 3 ++- src/openhuman/agent/agents/orchestrator/prompt.rs | 13 +++++++++++++ 3 files changed, 22 insertions(+), 8 deletions(-) diff --git a/app/test/e2e/specs/chat-harness-subagent.spec.ts b/app/test/e2e/specs/chat-harness-subagent.spec.ts index a63df08bc..5b95df662 100644 --- a/app/test/e2e/specs/chat-harness-subagent.spec.ts +++ b/app/test/e2e/specs/chat-harness-subagent.spec.ts @@ -3,17 +3,17 @@ * * The default chat agent after onboarding is the **orchestrator** * (`src/openhuman/channels/providers/web.rs::pick_target_agent_id`). - * Its `subagents = [...]` list synthesises one `delegate_` tool per - * archetype at build time (see + * Its `subagents = [...]` list synthesises one delegated archetype tool + * per archetype at build time (see * `src/openhuman/tools/orchestrator_tools.rs`). When the LLM calls - * `delegate_researcher` (or any other `delegate_*`), the tool dispatches + * `research` (or any other delegated archetype tool), the tool dispatches * to a sub-agent which runs the agent harness loop a level deeper — * which means the LLM gets hit at least once more for the sub-agent. * * What this spec scripts and verifies: * * 1. Configure `llmForcedResponses` with THREE responses in order: - * A) orchestrator turn — emits `delegate_researcher` tool_call + * A) orchestrator turn — emits `research` tool_call * B) researcher turn — answers with a plain text finding * C) orchestrator turn — final synthesis text (canary marker) * @@ -56,13 +56,13 @@ const RESEARCHER_REPLY = 'The researcher answer is 42.'; // Three forced responses, popped in order by the mock LLM streamer. const FORCED_RESPONSES = [ - // 1. Orchestrator: emit a delegate_researcher tool call. + // 1. Orchestrator: emit a research tool call. { content: '', toolCalls: [ { - id: 'call_delegate_researcher_1', - name: 'delegate_researcher', + id: 'call_research_1', + name: 'research', arguments: JSON.stringify({ prompt: 'Tell me a marker phrase' }), }, ], diff --git a/src/openhuman/agent/agents/orchestrator/prompt.md b/src/openhuman/agent/agents/orchestrator/prompt.md index c07e521d1..95eb36ba0 100644 --- a/src/openhuman/agent/agents/orchestrator/prompt.md +++ b/src/openhuman/agent/agents/orchestrator/prompt.md @@ -28,7 +28,8 @@ Follow this sequence for every user message: 4. **Does this need other specialised execution?** - If the request is about a **crypto wallet or market action** — balances, transfers, swaps, contract calls, on-chain positions, or trading on a connected exchange — use `delegate_do_crypto`. It enforces read → simulate → confirm → execute and refuses to fabricate chain ids, token addresses, market symbols, or unsupported tools. **Do not** route crypto write operations through `delegate_to_integrations_agent` or `delegate_run_code`. - If code writing/execution/debugging is required, use `delegate_run_code`. - - If web/doc crawling is required, use `delegate_researcher`. + - If web/doc crawling is required, use `research`. + - If the user asks for live/current/time-sensitive facts that are not covered by a direct tool — weather, forecasts, current temperatures, recent news, fresh web facts, or "use Grok/web/live data" — call `research` with a prompt that asks for live sources. Do **not** stop at "on it", and do **not** wait for the exact named provider if it is not wired in. Use the available research tool and then answer with the result. - If complex multi-step decomposition is required, use `delegate_plan`. - If code review is requested, use `delegate_critic`. - If memory archiving or distillation is required, use `delegate_archivist`. diff --git a/src/openhuman/agent/agents/orchestrator/prompt.rs b/src/openhuman/agent/agents/orchestrator/prompt.rs index 874ee2718..ccc224a36 100644 --- a/src/openhuman/agent/agents/orchestrator/prompt.rs +++ b/src/openhuman/agent/agents/orchestrator/prompt.rs @@ -216,6 +216,19 @@ mod tests { assert!(body.contains("Do this even if `memory_tree` could plausibly answer")); } + #[test] + fn build_routes_live_facts_to_research_tool() { + let body = build(&ctx_with(&[])).unwrap(); + assert!(body.contains("use `research`")); + assert!(body.contains("weather, forecasts, current temperatures")); + assert!(body.contains("\"use Grok/web/live data\"")); + assert!(body.contains("Do **not** stop at \"on it\"")); + assert!( + !body.contains("delegate_researcher"), + "orchestrator prompt should name the synthesized researcher tool" + ); + } + #[test] fn build_emits_delegation_guide_with_collapsed_tool() { let integrations = vec![ConnectedIntegration {