fix(composio): fail fast for uncurated empty toolkits

## Summary

- Adds a fast-fail path when `composio_list_tools` ends up empty for requested or connected toolkits that do not have curated OpenHuman agent catalogs.
- Returns a clear unsupported-toolkit error instead of a successful empty `tools` response for uncurated scopes such as OneDrive, Excel, or Todoist.
- Keeps existing behavior for catalogued toolkits and direct-mode empty responses.
- Adds focused unit coverage for toolkit normalization and unsupported-toolkit messaging.

## Problem

- Some toolkits can be connected in the UI but do not have curated OpenHuman agent tool catalogs yet.
- When the agent asks `composio_list_tools` for those toolkits, it can receive an empty usable tool list and continue trying until max iterations.
- Users then see a generic agent failure instead of a direct explanation that the connected toolkit is not agent-ready.

## Solution

- Track the explicit `toolkits` filter, or the active connected toolkits when filtering to connected accounts.
- If the final `tools` list is empty and the scoped toolkit set includes uncatalogued toolkits, return a `ToolResult::error` with a concrete agent-ready support message.
- Leave catalog creation and UI preview/coming-soon badges as follow-up slices so this PR stays small.

## Submission Checklist

- [x] Tests added or updated (happy path + at least one failure / edge case) per [Testing Strategy](../gitbooks/developing/testing-strategy.md#failure-path-requirement)
- [x] **Diff coverage ≥ 80%** — local coverage run is blocked by missing libclang; CI coverage gate will verify changed lines.
- [x] Coverage matrix updated — N/A: behavior-only Composio tool failure path, no feature row added/removed/renamed.
- [x] All affected feature IDs from the matrix are listed in the PR description under `## Related` — N/A: no matrix feature ID changed.
- [x] No new external network dependencies introduced (mock backend used per [Testing Strategy](../gitbooks/developing/testing-strategy.md#mock-policy))
- [x] Manual smoke checklist updated if this touches release-cut surfaces ([`docs/RELEASE-MANUAL-SMOKE.md`](../docs/RELEASE-MANUAL-SMOKE.md)) — N/A: no release smoke checklist surface changed.
- [x] Linked issue closed via `Closes #NNN` in the `## Related` section — N/A: partial fix for #2283; catalog/UI work remains.

## Impact

- Runtime: Composio agent tool discovery.
- User-visible: the agent gets a direct unsupported-toolkit message instead of looping on an empty action list.
- Compatibility: catalogued toolkits still return tools as before; direct mode still returns success+empty by design.

## Related

- Refs #2283
- Follow-up PR(s)/TODOs: add curated catalogs for OneDrive/Excel/Todoist; add UI preview/agent-coming-soon labeling for uncatalogued connected toolkits.

---

## AI Authored PR Metadata (required for Codex/Linear PRs)

### Linear Issue
- Key: N/A
- URL: N/A

### Commit & Branch
- Branch: `codex/2283-uncurated-toolkit-fast-fail`
- Commit SHA: `d299c8ae92c690b911647f22746372572f17ff60`

### Validation Run
- [x] `pnpm --filter openhuman-app format:check` — N/A: no frontend changes.
- [x] `pnpm typecheck` — N/A: no TypeScript changes.
- [x] Focused tests: blocked locally; see Validation Blocked.
- [x] Rust fmt/check (if changed): `cargo fmt --check` passed.
- [x] Tauri fmt/check (if changed): N/A: no Tauri shell changes.

### Validation Blocked
- `command:` `cargo test --lib empty_uncurated_toolkits_message --manifest-path Cargo.toml`
- `error:` `whisper-rs-sys` build script could not find `clang.dll` / `libclang.dll`; `LIBCLANG_PATH` is unset in this Windows environment.
- `impact:` focused Rust tests could not run locally, but the helper tests are included for CI.

### Behavior Changes
- Intended behavior change: empty `composio_list_tools` results for uncatalogued requested/connected toolkits now fail fast with a useful message.
- User-visible effect: the agent should stop burning through max iterations when a connected toolkit is not yet agent-ready.

### Parity Contract
- Legacy behavior preserved: catalogued toolkits, scope filtering, connection filtering, and direct-mode short-circuit behavior are unchanged.
- Guard/fallback/dispatch parity checks: unit tests cover requested toolkit normalization, uncatalogued toolkit messaging, and catalogued toolkit no-op behavior.

### Duplicate / Superseded PR Handling
- Duplicate PR(s): none found for #2283 at PR creation time.
- Canonical PR: this PR for the fast-fail slice only.
- Resolution (closed/superseded/updated): N/A

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

* **Bug Fixes**
  * Improved toolkit filtering so empty results now return clear, user-facing guidance when selected toolkits lack curated agent tools, including which toolkits are affected.

* **Tests**
  * Added unit tests to validate normalized toolkit filtering and the new uncatalogued-toolkit messaging behavior, including provider-backed cases.

<!-- review_stack_entry_start -->

[![Review Change Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/tinyhumansai/openhuman/pull/2293?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Co-authored-by: aqilaziz <gonzes7@gmail.com>
Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai>
This commit is contained in:
Aqil Aziz
2026-05-20 16:50:33 -07:00
committed by GitHub
co-authored by Steven Enamakel
parent 227f534e2c
commit ec1352f059
2 changed files with 147 additions and 2 deletions
+66 -2
View File
@@ -24,6 +24,7 @@
//! the right slug and supply valid arguments without a separate round
//! trip.
use std::collections::{BTreeSet, HashSet};
use std::sync::Arc;
use async_trait::async_trait;
@@ -122,7 +123,7 @@ async fn evaluate_tool_visibility(slug: &str) -> ToolDecision {
/// is direct against entries the caller has already lowercased.
fn retain_connected_tools(
resp: &mut super::types::ComposioToolsResponse,
connected: &std::collections::HashSet<String>,
connected: &HashSet<String>,
) -> usize {
let before = resp.tools.len();
resp.tools.retain(|t| {
@@ -133,6 +134,55 @@ fn retain_connected_tools(
before - resp.tools.len()
}
fn normalized_scope_toolkits(
requested: Option<&[String]>,
connected: Option<&HashSet<String>>,
) -> Vec<String> {
let mut out = BTreeSet::new();
if let Some(requested) = requested {
for toolkit in requested {
let normalized = toolkit.trim().to_ascii_lowercase();
if !normalized.is_empty() {
out.insert(normalized);
}
}
} else if let Some(connected) = connected {
out.extend(connected.iter().filter(|t| !t.is_empty()).cloned());
}
out.into_iter().collect()
}
fn uncatalogued_toolkits(toolkits: &[String]) -> Vec<String> {
toolkits
.iter()
.filter(|toolkit| {
get_provider(toolkit)
.and_then(|provider| provider.curated_tools())
.or_else(|| catalog_for_toolkit(toolkit))
.is_none()
})
.cloned()
.collect()
}
fn empty_uncurated_toolkits_message(toolkits: &[String]) -> Option<String> {
let unsupported = uncatalogued_toolkits(toolkits);
if unsupported.is_empty() {
return None;
}
let names = unsupported
.iter()
.map(|toolkit| format!("`{toolkit}`"))
.collect::<Vec<_>>()
.join(", ");
Some(format!(
"composio_list_tools: no agent-ready actions are available for toolkit(s) {names}. \
These integrations can be connected, but OpenHuman does not yet ship curated agent \
tool catalogs for them. Use a supported toolkit such as Google Drive or Google Sheets \
for now, or try again after catalog support lands."
))
}
/// Filter a freshly-fetched [`super::types::ComposioToolsResponse`] in
/// place: drop tools that aren't curated for their toolkit and tools
/// whose scope is disabled in the user's pref.
@@ -736,6 +786,7 @@ impl Tool for ComposioListToolsTool {
match client.list_tools(toolkits.as_deref()).await {
Ok(mut resp) => {
filter_list_tools_response(&mut resp).await;
let mut connected_toolkits: Option<HashSet<String>> = None;
if !include_unconnected {
// Restrict to toolkits with an ACTIVE / CONNECTED
@@ -744,7 +795,7 @@ impl Tool for ComposioListToolsTool {
// prompt's Delegation Guide stay in sync.
match client.list_connections().await {
Ok(conns) => {
let connected: std::collections::HashSet<String> = conns
let connected: HashSet<String> = conns
.connections
.iter()
.filter(|c| c.is_active())
@@ -758,6 +809,7 @@ impl Tool for ComposioListToolsTool {
kept = resp.tools.len(),
"[composio] list_tools restricted to connected toolkits"
);
connected_toolkits = Some(connected);
}
Err(e) => {
// Soft-fail: surface the issue to the agent
@@ -772,6 +824,18 @@ impl Tool for ComposioListToolsTool {
}
}
if resp.tools.is_empty() {
let scoped_toolkits =
normalized_scope_toolkits(toolkits.as_deref(), connected_toolkits.as_ref());
if let Some(message) = empty_uncurated_toolkits_message(&scoped_toolkits) {
tracing::debug!(
toolkits = ?scoped_toolkits,
"[composio] list_tools empty for uncurated toolkit scope"
);
return Ok(ToolResult::error(message));
}
}
let mut result = ToolResult::success(
serde_json::to_string(&resp).unwrap_or_else(|_| "{}".into()),
);
+81
View File
@@ -1,7 +1,46 @@
use super::*;
use crate::openhuman::composio::providers::tool_scope::{CuratedTool, ToolScope};
use crate::openhuman::composio::providers::{
registry::register_provider, ComposioProvider, ProviderContext, ProviderUserProfile,
SyncOutcome, SyncReason,
};
use async_trait::async_trait;
use std::path::Path;
use std::sync::Arc;
static PROVIDER_ONLY_CURATED: &[CuratedTool] = &[CuratedTool {
slug: "PROVIDERONLY_LIST_ITEMS",
scope: ToolScope::Read,
}];
struct ProviderOnlyCatalog;
#[async_trait]
impl ComposioProvider for ProviderOnlyCatalog {
fn toolkit_slug(&self) -> &'static str {
"provideronly"
}
fn curated_tools(&self) -> Option<&'static [CuratedTool]> {
Some(PROVIDER_ONLY_CURATED)
}
async fn fetch_user_profile(
&self,
_ctx: &ProviderContext,
) -> Result<ProviderUserProfile, String> {
Ok(ProviderUserProfile::default())
}
async fn sync(
&self,
_ctx: &ProviderContext,
_reason: SyncReason,
) -> Result<SyncOutcome, String> {
Ok(SyncOutcome::default())
}
}
struct WorkspaceEnvGuard {
previous: Option<std::ffi::OsString>,
}
@@ -565,6 +604,48 @@ fn retain_connected_tools_drops_unconnected_toolkits_case_insensitively() {
assert!(!names.contains(&"NOTION_CREATE_PAGE"));
}
#[test]
fn normalized_scope_toolkits_prefers_requested_filter() {
use std::collections::HashSet;
let requested = vec![" OneDrive ".to_string(), "excel".to_string()];
let connected: HashSet<String> = ["gmail".to_string()].into_iter().collect();
assert_eq!(
normalized_scope_toolkits(Some(&requested), Some(&connected)),
vec!["excel".to_string(), "onedrive".to_string()]
);
}
#[test]
fn empty_uncurated_toolkits_message_names_agent_unsupported_toolkits() {
let message = empty_uncurated_toolkits_message(&[
"onedrive".to_string(),
"excel".to_string(),
"todoist".to_string(),
])
.expect("uncurated toolkit message");
assert!(message.contains("no agent-ready actions"));
assert!(message.contains("`onedrive`"));
assert!(message.contains("`excel`"));
assert!(message.contains("`todoist`"));
assert!(message.contains("curated agent tool catalogs"));
}
#[test]
fn empty_uncurated_toolkits_message_ignores_catalogued_toolkits() {
assert!(empty_uncurated_toolkits_message(&["gmail".to_string()]).is_none());
assert!(empty_uncurated_toolkits_message(&["googlesheets".to_string()]).is_none());
}
#[test]
fn empty_uncurated_toolkits_message_uses_provider_curated_tools() {
register_provider(Arc::new(ProviderOnlyCatalog));
assert!(empty_uncurated_toolkits_message(&["provideronly".to_string()]).is_none());
}
#[test]
fn render_tools_markdown_handles_empty_response() {
use crate::openhuman::composio::types::ComposioToolsResponse;