mirror of
https://github.com/tinyhumansai/openhuman.git
synced 2026-07-27 21:08:00 +00:00
fix(composio): fail fast for uncurated empty toolkits
## Summary - Adds a fast-fail path when `composio_list_tools` ends up empty for requested or connected toolkits that do not have curated OpenHuman agent catalogs. - Returns a clear unsupported-toolkit error instead of a successful empty `tools` response for uncurated scopes such as OneDrive, Excel, or Todoist. - Keeps existing behavior for catalogued toolkits and direct-mode empty responses. - Adds focused unit coverage for toolkit normalization and unsupported-toolkit messaging. ## Problem - Some toolkits can be connected in the UI but do not have curated OpenHuman agent tool catalogs yet. - When the agent asks `composio_list_tools` for those toolkits, it can receive an empty usable tool list and continue trying until max iterations. - Users then see a generic agent failure instead of a direct explanation that the connected toolkit is not agent-ready. ## Solution - Track the explicit `toolkits` filter, or the active connected toolkits when filtering to connected accounts. - If the final `tools` list is empty and the scoped toolkit set includes uncatalogued toolkits, return a `ToolResult::error` with a concrete agent-ready support message. - Leave catalog creation and UI preview/coming-soon badges as follow-up slices so this PR stays small. ## Submission Checklist - [x] Tests added or updated (happy path + at least one failure / edge case) per [Testing Strategy](../gitbooks/developing/testing-strategy.md#failure-path-requirement) - [x] **Diff coverage ≥ 80%** — local coverage run is blocked by missing libclang; CI coverage gate will verify changed lines. - [x] Coverage matrix updated — N/A: behavior-only Composio tool failure path, no feature row added/removed/renamed. - [x] All affected feature IDs from the matrix are listed in the PR description under `## Related` — N/A: no matrix feature ID changed. - [x] No new external network dependencies introduced (mock backend used per [Testing Strategy](../gitbooks/developing/testing-strategy.md#mock-policy)) - [x] Manual smoke checklist updated if this touches release-cut surfaces ([`docs/RELEASE-MANUAL-SMOKE.md`](../docs/RELEASE-MANUAL-SMOKE.md)) — N/A: no release smoke checklist surface changed. - [x] Linked issue closed via `Closes #NNN` in the `## Related` section — N/A: partial fix for #2283; catalog/UI work remains. ## Impact - Runtime: Composio agent tool discovery. - User-visible: the agent gets a direct unsupported-toolkit message instead of looping on an empty action list. - Compatibility: catalogued toolkits still return tools as before; direct mode still returns success+empty by design. ## Related - Refs #2283 - Follow-up PR(s)/TODOs: add curated catalogs for OneDrive/Excel/Todoist; add UI preview/agent-coming-soon labeling for uncatalogued connected toolkits. --- ## AI Authored PR Metadata (required for Codex/Linear PRs) ### Linear Issue - Key: N/A - URL: N/A ### Commit & Branch - Branch: `codex/2283-uncurated-toolkit-fast-fail` - Commit SHA: `d299c8ae92c690b911647f22746372572f17ff60` ### Validation Run - [x] `pnpm --filter openhuman-app format:check` — N/A: no frontend changes. - [x] `pnpm typecheck` — N/A: no TypeScript changes. - [x] Focused tests: blocked locally; see Validation Blocked. - [x] Rust fmt/check (if changed): `cargo fmt --check` passed. - [x] Tauri fmt/check (if changed): N/A: no Tauri shell changes. ### Validation Blocked - `command:` `cargo test --lib empty_uncurated_toolkits_message --manifest-path Cargo.toml` - `error:` `whisper-rs-sys` build script could not find `clang.dll` / `libclang.dll`; `LIBCLANG_PATH` is unset in this Windows environment. - `impact:` focused Rust tests could not run locally, but the helper tests are included for CI. ### Behavior Changes - Intended behavior change: empty `composio_list_tools` results for uncatalogued requested/connected toolkits now fail fast with a useful message. - User-visible effect: the agent should stop burning through max iterations when a connected toolkit is not yet agent-ready. ### Parity Contract - Legacy behavior preserved: catalogued toolkits, scope filtering, connection filtering, and direct-mode short-circuit behavior are unchanged. - Guard/fallback/dispatch parity checks: unit tests cover requested toolkit normalization, uncatalogued toolkit messaging, and catalogued toolkit no-op behavior. ### Duplicate / Superseded PR Handling - Duplicate PR(s): none found for #2283 at PR creation time. - Canonical PR: this PR for the fast-fail slice only. - Resolution (closed/superseded/updated): N/A <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved toolkit filtering so empty results now return clear, user-facing guidance when selected toolkits lack curated agent tools, including which toolkits are affected. * **Tests** * Added unit tests to validate normalized toolkit filtering and the new uncatalogued-toolkit messaging behavior, including provider-backed cases. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/tinyhumansai/openhuman/pull/2293?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> Co-authored-by: aqilaziz <gonzes7@gmail.com> Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai>
This commit is contained in:
co-authored by
Steven Enamakel
parent
227f534e2c
commit
ec1352f059
@@ -24,6 +24,7 @@
|
||||
//! the right slug and supply valid arguments without a separate round
|
||||
//! trip.
|
||||
|
||||
use std::collections::{BTreeSet, HashSet};
|
||||
use std::sync::Arc;
|
||||
|
||||
use async_trait::async_trait;
|
||||
@@ -122,7 +123,7 @@ async fn evaluate_tool_visibility(slug: &str) -> ToolDecision {
|
||||
/// is direct against entries the caller has already lowercased.
|
||||
fn retain_connected_tools(
|
||||
resp: &mut super::types::ComposioToolsResponse,
|
||||
connected: &std::collections::HashSet<String>,
|
||||
connected: &HashSet<String>,
|
||||
) -> usize {
|
||||
let before = resp.tools.len();
|
||||
resp.tools.retain(|t| {
|
||||
@@ -133,6 +134,55 @@ fn retain_connected_tools(
|
||||
before - resp.tools.len()
|
||||
}
|
||||
|
||||
fn normalized_scope_toolkits(
|
||||
requested: Option<&[String]>,
|
||||
connected: Option<&HashSet<String>>,
|
||||
) -> Vec<String> {
|
||||
let mut out = BTreeSet::new();
|
||||
if let Some(requested) = requested {
|
||||
for toolkit in requested {
|
||||
let normalized = toolkit.trim().to_ascii_lowercase();
|
||||
if !normalized.is_empty() {
|
||||
out.insert(normalized);
|
||||
}
|
||||
}
|
||||
} else if let Some(connected) = connected {
|
||||
out.extend(connected.iter().filter(|t| !t.is_empty()).cloned());
|
||||
}
|
||||
out.into_iter().collect()
|
||||
}
|
||||
|
||||
fn uncatalogued_toolkits(toolkits: &[String]) -> Vec<String> {
|
||||
toolkits
|
||||
.iter()
|
||||
.filter(|toolkit| {
|
||||
get_provider(toolkit)
|
||||
.and_then(|provider| provider.curated_tools())
|
||||
.or_else(|| catalog_for_toolkit(toolkit))
|
||||
.is_none()
|
||||
})
|
||||
.cloned()
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn empty_uncurated_toolkits_message(toolkits: &[String]) -> Option<String> {
|
||||
let unsupported = uncatalogued_toolkits(toolkits);
|
||||
if unsupported.is_empty() {
|
||||
return None;
|
||||
}
|
||||
let names = unsupported
|
||||
.iter()
|
||||
.map(|toolkit| format!("`{toolkit}`"))
|
||||
.collect::<Vec<_>>()
|
||||
.join(", ");
|
||||
Some(format!(
|
||||
"composio_list_tools: no agent-ready actions are available for toolkit(s) {names}. \
|
||||
These integrations can be connected, but OpenHuman does not yet ship curated agent \
|
||||
tool catalogs for them. Use a supported toolkit such as Google Drive or Google Sheets \
|
||||
for now, or try again after catalog support lands."
|
||||
))
|
||||
}
|
||||
|
||||
/// Filter a freshly-fetched [`super::types::ComposioToolsResponse`] in
|
||||
/// place: drop tools that aren't curated for their toolkit and tools
|
||||
/// whose scope is disabled in the user's pref.
|
||||
@@ -736,6 +786,7 @@ impl Tool for ComposioListToolsTool {
|
||||
match client.list_tools(toolkits.as_deref()).await {
|
||||
Ok(mut resp) => {
|
||||
filter_list_tools_response(&mut resp).await;
|
||||
let mut connected_toolkits: Option<HashSet<String>> = None;
|
||||
|
||||
if !include_unconnected {
|
||||
// Restrict to toolkits with an ACTIVE / CONNECTED
|
||||
@@ -744,7 +795,7 @@ impl Tool for ComposioListToolsTool {
|
||||
// prompt's Delegation Guide stay in sync.
|
||||
match client.list_connections().await {
|
||||
Ok(conns) => {
|
||||
let connected: std::collections::HashSet<String> = conns
|
||||
let connected: HashSet<String> = conns
|
||||
.connections
|
||||
.iter()
|
||||
.filter(|c| c.is_active())
|
||||
@@ -758,6 +809,7 @@ impl Tool for ComposioListToolsTool {
|
||||
kept = resp.tools.len(),
|
||||
"[composio] list_tools restricted to connected toolkits"
|
||||
);
|
||||
connected_toolkits = Some(connected);
|
||||
}
|
||||
Err(e) => {
|
||||
// Soft-fail: surface the issue to the agent
|
||||
@@ -772,6 +824,18 @@ impl Tool for ComposioListToolsTool {
|
||||
}
|
||||
}
|
||||
|
||||
if resp.tools.is_empty() {
|
||||
let scoped_toolkits =
|
||||
normalized_scope_toolkits(toolkits.as_deref(), connected_toolkits.as_ref());
|
||||
if let Some(message) = empty_uncurated_toolkits_message(&scoped_toolkits) {
|
||||
tracing::debug!(
|
||||
toolkits = ?scoped_toolkits,
|
||||
"[composio] list_tools empty for uncurated toolkit scope"
|
||||
);
|
||||
return Ok(ToolResult::error(message));
|
||||
}
|
||||
}
|
||||
|
||||
let mut result = ToolResult::success(
|
||||
serde_json::to_string(&resp).unwrap_or_else(|_| "{}".into()),
|
||||
);
|
||||
|
||||
@@ -1,7 +1,46 @@
|
||||
use super::*;
|
||||
use crate::openhuman::composio::providers::tool_scope::{CuratedTool, ToolScope};
|
||||
use crate::openhuman::composio::providers::{
|
||||
registry::register_provider, ComposioProvider, ProviderContext, ProviderUserProfile,
|
||||
SyncOutcome, SyncReason,
|
||||
};
|
||||
use async_trait::async_trait;
|
||||
use std::path::Path;
|
||||
use std::sync::Arc;
|
||||
|
||||
static PROVIDER_ONLY_CURATED: &[CuratedTool] = &[CuratedTool {
|
||||
slug: "PROVIDERONLY_LIST_ITEMS",
|
||||
scope: ToolScope::Read,
|
||||
}];
|
||||
|
||||
struct ProviderOnlyCatalog;
|
||||
|
||||
#[async_trait]
|
||||
impl ComposioProvider for ProviderOnlyCatalog {
|
||||
fn toolkit_slug(&self) -> &'static str {
|
||||
"provideronly"
|
||||
}
|
||||
|
||||
fn curated_tools(&self) -> Option<&'static [CuratedTool]> {
|
||||
Some(PROVIDER_ONLY_CURATED)
|
||||
}
|
||||
|
||||
async fn fetch_user_profile(
|
||||
&self,
|
||||
_ctx: &ProviderContext,
|
||||
) -> Result<ProviderUserProfile, String> {
|
||||
Ok(ProviderUserProfile::default())
|
||||
}
|
||||
|
||||
async fn sync(
|
||||
&self,
|
||||
_ctx: &ProviderContext,
|
||||
_reason: SyncReason,
|
||||
) -> Result<SyncOutcome, String> {
|
||||
Ok(SyncOutcome::default())
|
||||
}
|
||||
}
|
||||
|
||||
struct WorkspaceEnvGuard {
|
||||
previous: Option<std::ffi::OsString>,
|
||||
}
|
||||
@@ -565,6 +604,48 @@ fn retain_connected_tools_drops_unconnected_toolkits_case_insensitively() {
|
||||
assert!(!names.contains(&"NOTION_CREATE_PAGE"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn normalized_scope_toolkits_prefers_requested_filter() {
|
||||
use std::collections::HashSet;
|
||||
|
||||
let requested = vec![" OneDrive ".to_string(), "excel".to_string()];
|
||||
let connected: HashSet<String> = ["gmail".to_string()].into_iter().collect();
|
||||
|
||||
assert_eq!(
|
||||
normalized_scope_toolkits(Some(&requested), Some(&connected)),
|
||||
vec!["excel".to_string(), "onedrive".to_string()]
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn empty_uncurated_toolkits_message_names_agent_unsupported_toolkits() {
|
||||
let message = empty_uncurated_toolkits_message(&[
|
||||
"onedrive".to_string(),
|
||||
"excel".to_string(),
|
||||
"todoist".to_string(),
|
||||
])
|
||||
.expect("uncurated toolkit message");
|
||||
|
||||
assert!(message.contains("no agent-ready actions"));
|
||||
assert!(message.contains("`onedrive`"));
|
||||
assert!(message.contains("`excel`"));
|
||||
assert!(message.contains("`todoist`"));
|
||||
assert!(message.contains("curated agent tool catalogs"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn empty_uncurated_toolkits_message_ignores_catalogued_toolkits() {
|
||||
assert!(empty_uncurated_toolkits_message(&["gmail".to_string()]).is_none());
|
||||
assert!(empty_uncurated_toolkits_message(&["googlesheets".to_string()]).is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn empty_uncurated_toolkits_message_uses_provider_curated_tools() {
|
||||
register_provider(Arc::new(ProviderOnlyCatalog));
|
||||
|
||||
assert!(empty_uncurated_toolkits_message(&["provideronly".to_string()]).is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn render_tools_markdown_handles_empty_response() {
|
||||
use crate::openhuman::composio::types::ComposioToolsResponse;
|
||||
|
||||
Reference in New Issue
Block a user