mirror of
https://github.com/tinyhumansai/openhuman.git
synced 2026-07-27 21:08:00 +00:00
fix(orchestrator): route live facts to research
## Summary - Routes live/current factual requests (weather, forecasts, recent web facts, "use Grok/live data") to the available `research` tool. - Replaces the stale `delegate_researcher` prompt reference with the synthesized tool name `research`. - Updates the chat harness subagent fixture to emit `research` so the forced tool call matches the current delegation surface. - Adds a prompt-builder regression test that locks the live-data rule and rejects the stale tool name. ## Problem - The orchestrator could acknowledge live weather/research requests with "on it" without actually calling the research tool. - The prompt named `delegate_researcher`, but the current researcher tool is exposed as `research` via `delegate_name = "research"`. - Users who named an unwired provider like Grok could stall the agent instead of falling back to the available live research path. ## Solution - Teach the orchestrator to call `research` for live/current/time-sensitive facts that direct tools cannot answer. - Explicitly tell it not to stop at "on it" and not to wait for the exact named provider when that provider is not wired in. - Align the E2E harness fixture with the real `research` tool name. ## Submission Checklist - [x] Tests added or updated (happy path + at least one failure / edge case) per [Testing Strategy](../gitbooks/developing/testing-strategy.md#failure-path-requirement) - added prompt regression assertion and updated E2E forced tool fixture. - [x] **Diff coverage >= 80%** - N/A locally: tiny prompt/test fixture change; CI diff coverage will be authoritative. - [x] Coverage matrix updated - N/A: prompt routing change, no feature row added/removed/renamed. - [x] All affected feature IDs from the matrix are listed in the PR description under `## Related` - N/A: no matrix feature ID applies. - [x] No new external network dependencies introduced (mock backend used per [Testing Strategy](../gitbooks/developing/testing-strategy.md#mock-policy)) - [x] Manual smoke checklist updated if this touches release-cut surfaces ([`docs/RELEASE-MANUAL-SMOKE.md`](../docs/RELEASE-MANUAL-SMOKE.md)) - N/A: no release-cut surface. - [x] Linked issue closed via `Closes #NNN` in the `## Related` section ## Impact - Runtime: orchestrator prompt only, plus E2E harness fixture alignment. - User-visible effect: live weather/current fact requests should call research instead of ending with an acknowledgement. - No migration, security, or dependency impact. ## Related - Closes #2164 - Follow-up PR(s)/TODOs: N/A --- ## AI Authored PR Metadata (required for Codex/Linear PRs) ### Linear Issue - Key: N/A - URL: N/A ### Commit & Branch - Branch: `codex/2164-live-data-research` - Commit SHA: `dc63d1d3e1ea531a1609594debd1e0ca95329b27` ### Validation Run - [x] `pnpm --filter openhuman-app exec prettier --check test/e2e/specs/chat-harness-subagent.spec.ts` - passed - [x] `pnpm typecheck` - N/A: comments/string fixture plus Rust prompt text only - [x] Focused tests: `cargo test --lib build_routes_live_facts_to_research_tool` attempted; blocked before test execution by local `libclang` setup - [x] Rust fmt/check (if changed): `cargo fmt --all --check` - passed; `git diff --check` - passed - [x] Tauri fmt/check (if changed): N/A ### Validation Blocked - `command:` `cargo test --lib build_routes_live_facts_to_research_tool` - `error:` `whisper-rs-sys` build could not find `clang.dll` / `libclang.dll`; `LIBCLANG_PATH` is unset in this Windows environment - `impact:` Rust prompt unit test did not execute locally; remote CI environment should validate it ### Behavior Changes - Intended behavior change: live/current factual requests route to `research` when no direct tool can answer. - User-visible effect: requests like "forecast for Bremerhaven today" should produce a researched answer instead of a bare "on it" acknowledgement. ### Parity Contract - Legacy behavior preserved: direct answers, connected integration routing, crypto/code/planner/critic/archive routing remain unchanged. - Guard/fallback/dispatch parity checks: prompt test asserts live facts mention weather/forecast/Grok fallback and rejects stale `delegate_researcher` naming. ### Duplicate / Superseded PR Handling - Duplicate PR(s): N/A - Canonical PR: this PR - Resolution (closed/superseded/updated): N/A <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Release Notes * **Tests** * Enhanced test coverage for research request handling and subagent communication. * **Chores** * Improved internal routing and terminology for research operations to support live, time-sensitive fact queries more effectively. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/tinyhumansai/openhuman/pull/2299?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> Co-authored-by: aqilaziz <gonzes7@gmail.com> Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai>
This commit is contained in:
co-authored by
Steven Enamakel
parent
5a0a5a7e02
commit
15df9855c3
@@ -3,17 +3,17 @@
|
||||
*
|
||||
* The default chat agent after onboarding is the **orchestrator**
|
||||
* (`src/openhuman/channels/providers/web.rs::pick_target_agent_id`).
|
||||
* Its `subagents = [...]` list synthesises one `delegate_<id>` tool per
|
||||
* archetype at build time (see
|
||||
* Its `subagents = [...]` list synthesises one delegated archetype tool
|
||||
* per archetype at build time (see
|
||||
* `src/openhuman/tools/orchestrator_tools.rs`). When the LLM calls
|
||||
* `delegate_researcher` (or any other `delegate_*`), the tool dispatches
|
||||
* `research` (or any other delegated archetype tool), the tool dispatches
|
||||
* to a sub-agent which runs the agent harness loop a level deeper —
|
||||
* which means the LLM gets hit at least once more for the sub-agent.
|
||||
*
|
||||
* What this spec scripts and verifies:
|
||||
*
|
||||
* 1. Configure `llmForcedResponses` with THREE responses in order:
|
||||
* A) orchestrator turn — emits `delegate_researcher` tool_call
|
||||
* A) orchestrator turn — emits `research` tool_call
|
||||
* B) researcher turn — answers with a plain text finding
|
||||
* C) orchestrator turn — final synthesis text (canary marker)
|
||||
*
|
||||
@@ -56,13 +56,13 @@ const RESEARCHER_REPLY = 'The researcher answer is 42.';
|
||||
|
||||
// Three forced responses, popped in order by the mock LLM streamer.
|
||||
const FORCED_RESPONSES = [
|
||||
// 1. Orchestrator: emit a delegate_researcher tool call.
|
||||
// 1. Orchestrator: emit a research tool call.
|
||||
{
|
||||
content: '',
|
||||
toolCalls: [
|
||||
{
|
||||
id: 'call_delegate_researcher_1',
|
||||
name: 'delegate_researcher',
|
||||
id: 'call_research_1',
|
||||
name: 'research',
|
||||
arguments: JSON.stringify({ prompt: 'Tell me a marker phrase' }),
|
||||
},
|
||||
],
|
||||
|
||||
@@ -28,7 +28,8 @@ Follow this sequence for every user message:
|
||||
4. **Does this need other specialised execution?**
|
||||
- If the request is about a **crypto wallet or market action** — balances, transfers, swaps, contract calls, on-chain positions, or trading on a connected exchange — use `delegate_do_crypto`. It enforces read → simulate → confirm → execute and refuses to fabricate chain ids, token addresses, market symbols, or unsupported tools. **Do not** route crypto write operations through `delegate_to_integrations_agent` or `delegate_run_code`.
|
||||
- If code writing/execution/debugging is required, use `delegate_run_code`.
|
||||
- If web/doc crawling is required, use `delegate_researcher`.
|
||||
- If web/doc crawling is required, use `research`.
|
||||
- If the user asks for live/current/time-sensitive facts that are not covered by a direct tool — weather, forecasts, current temperatures, recent news, fresh web facts, or "use Grok/web/live data" — call `research` with a prompt that asks for live sources. Do **not** stop at "on it", and do **not** wait for the exact named provider if it is not wired in. Use the available research tool and then answer with the result.
|
||||
- If complex multi-step decomposition is required, use `delegate_plan`.
|
||||
- If code review is requested, use `delegate_critic`.
|
||||
- If memory archiving or distillation is required, use `delegate_archivist`.
|
||||
|
||||
@@ -216,6 +216,19 @@ mod tests {
|
||||
assert!(body.contains("Do this even if `memory_tree` could plausibly answer"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn build_routes_live_facts_to_research_tool() {
|
||||
let body = build(&ctx_with(&[])).unwrap();
|
||||
assert!(body.contains("use `research`"));
|
||||
assert!(body.contains("weather, forecasts, current temperatures"));
|
||||
assert!(body.contains("\"use Grok/web/live data\""));
|
||||
assert!(body.contains("Do **not** stop at \"on it\""));
|
||||
assert!(
|
||||
!body.contains("delegate_researcher"),
|
||||
"orchestrator prompt should name the synthesized researcher tool"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn build_emits_delegation_guide_with_collapsed_tool() {
|
||||
let integrations = vec![ConnectedIntegration {
|
||||
|
||||
Reference in New Issue
Block a user