From ee3f6472ca3a3d6b1c3fdba433ded6a4b28ee04e Mon Sep 17 00:00:00 2001 From: Steven Enamakel <31011319+senamakel@users.noreply.github.com> Date: Sat, 18 Apr 2026 00:44:31 -0700 Subject: [PATCH] feat: improve sub-agent tooling, conversation timeline UX, and Tauri setup (#646) * refactor(tauri): update development scripts and configuration for CEF integration - Modified `package.json` scripts to consistently export `CEF_PATH` for all `cargo tauri` commands, ensuring a unified CEF binary distribution location. - Removed the overlay window configuration from `tauri.conf.json` and updated related Rust functions to reflect this change, while retaining helper functions for potential future use. - Updated documentation in `install.md` to clarify the importance of setting `CEF_PATH` for consistent CEF integration across builds. - Enhanced the `ensure-tauri-cli.sh` script to set `CEF_PATH` and ensure proper installation of the vendored CEF-aware `tauri-cli`. These changes streamline the development workflow and improve the reliability of CEF integration in the application. * refactor(release): enhance macOS signing script for nested frameworks and helper apps - Introduced a new `codesign_hardened` function to streamline the signing process with consistent options. - Improved the signing logic for nested frameworks and helper applications, ensuring all binaries are signed correctly. - Updated output messages for better clarity during the signing process, including detailed listings of bundle contents. - Disabled the summarizer payload threshold in the configuration to prevent recursive invocations until the issue is resolved. These changes improve the reliability and maintainability of the macOS signing and notarization workflow. * feat(orchestrator): add current_time tool and enhance agent capabilities - Introduced the `current_time` tool to provide the current date and time in UTC and local time zones, facilitating scheduling and reminders. - Updated `agent.toml` to include new tools: `current_time`, `cron_add`, `cron_list`, `cron_remove`, and `schedule`, enhancing the orchestrator's functionality. - Expanded documentation in `prompt.md` to guide users on utilizing the new direct tools effectively. These changes improve the orchestrator's ability to handle time-related queries and scheduling tasks directly, enhancing user experience. * feat(gmail): implement post-processing for Gmail responses to convert HTML to markdown - Added a new `post_process` module specifically for Gmail, which modifies action responses to convert HTML content into markdown format, improving usability and reducing context token usage. - Enhanced the `ComposioProvider` trait with a `post_process_action_result` method to allow providers to handle response modifications. - Introduced a `post_process` function that checks for a `raw_html` flag in the arguments to determine whether to apply the conversion. - Implemented tests to validate the HTML detection and conversion logic, ensuring the integrity of the post-processing functionality. These changes enhance the handling of Gmail responses, making them more suitable for further processing and display in the application. * refactor(gmail): improve HTML to markdown conversion and tool result budget - Enhanced the `post_process` module for Gmail to handle large HTML payloads more efficiently, implementing a fallback mechanism for oversized content. - Updated the `DEFAULT_TOOL_RESULT_BUDGET_BYTES` to `0`, disabling the budget temporarily while reworking the oversized-output path. - Refined the `extract_markdown_body` function to better manage HTML content, ensuring cleaner markdown output and improved performance. - Added utility functions for stripping HTML noise and handling large email bodies, enhancing the overall robustness of the email processing logic. These changes optimize the handling of Gmail responses, improving usability and performance in processing large HTML content. * feat(thread): implement thread title generation from user and assistant messages - Added a new `generateTitleIfNeeded` function in the `threadApi` to create a thread title based on the first user message and the assistant's reply. - Introduced a new `GenerateConversationThreadTitleRequest` struct to handle requests for title generation. - Updated the `ChatRuntimeProvider` to dispatch the title generation action after processing inference responses. - Enhanced the `threadSlice` with a new async thunk for generating thread titles, ensuring proper error handling and thread loading. - Added tests for the new title generation functionality to validate the integration with the threads RPC. These changes improve the user experience by automatically generating relevant thread titles, enhancing the organization of conversations. * feat(conversations): add thread title update functionality - Implemented `update_thread_title` method in `ConversationStore` to allow updating the title of existing conversation threads. - Added a corresponding public function `update_thread_title` for external access. - Enhanced tests to verify that thread titles are correctly updated and persisted in the store. These changes improve the management of conversation threads by enabling dynamic title updates, enhancing user experience and organization. * feat(docs): add comprehensive agent and subagent tool flow documentation - Introduced a new document detailing the runtime flow of the agent harness, including execution paths for main agents and tools. - Explained the differences between typed and fork subagents, and provided guidance for debugging harness and delegation issues. - Included a file map outlining key components and their roles within the Rust implementation. - Added a flow diagram to visually represent the interaction between agents, tools, and subagents. These changes enhance the understanding of the agent architecture and improve the documentation for developers working with the system. * feat(subagent): implement extraction tool and handoff cache for oversized results - Introduced `extract_from_result` tool to allow targeted queries against oversized tool outputs, improving efficiency by directly interacting with the extraction model. - Added `ResultHandoffCache` to manage oversized payloads, enabling progressive disclosure and reducing context length issues in sub-agent history. - Implemented hygiene helpers for cleaning tool outputs before caching, ensuring only relevant data is stored. - Enhanced the sub-agent runner with new modules for tool preparation and execution, streamlining the overall agent workflow. These changes enhance the sub-agent's ability to handle large tool results effectively, improving performance and user experience. * feat(conversations): enhance message bubble rendering and timeline entry formatting - Introduced new utility functions for parsing and rendering agent messages, including `splitAgentMessageIntoBubbles` and `parseMarkdownTable`, to improve the display of messages in conversation threads. - Implemented `BubbleMarkdown` and `TableCellMarkdown` components for better formatting of user and agent messages, ensuring consistent styling and interaction. - Enhanced the `formatTimelineEntry` function to provide clearer titles and details for tool timeline entries, improving the user experience during interactions with subagents. - Updated the `ChatRuntimeProvider` to utilize the new formatting functions, ensuring that tool timeline entries are displayed with relevant context and detail. These changes improve the overall presentation and usability of conversation messages and tool interactions, enhancing user engagement and clarity. * feat(subagent): enforce restrictions on sub-agent spawning tools - Introduced a filter to prevent sub-agents from invoking their own spawning tools, specifically `spawn_subagent` and `delegate_*`, to avoid recursion issues and ensure proper delegation by the top-level orchestrator. - Updated the `is_subagent_spawn_tool` function to identify these tools and integrated checks in both `run_typed_mode` and `run_fork_mode` to maintain the integrity of the sub-agent execution environment. - Enhanced logging to track the removal of restricted tools from the sub-agent's tool surface, improving observability and debugging capabilities. These changes strengthen the sub-agent architecture by enforcing strict boundaries on tool invocation, enhancing stability and performance. * feat(conversations): add ToolTimelineBlock component and enhance timeline entry formatting - Introduced the `ToolTimelineBlock` component to display tool timeline entries with improved formatting and user interaction, including auto-expansion for running entries. - Enhanced the `formatTimelineEntry` function to include user-friendly titles for specific tool actions, such as 'Viewing your Integrations'. - Updated the rendering logic in the `Conversations` component to filter and display visible messages more effectively, improving user experience during conversations. These changes enhance the clarity and usability of tool interactions within conversation threads, providing users with better context and engagement. * refactor(conversations): streamline component imports and enhance formatting consistency - Consolidated import statements in `Conversations.tsx` and `ChatRuntimeProvider.tsx` for improved readability. - Refactored the `ToolTimelineBlock` component to simplify its props structure. - Enhanced formatting consistency in the rendering logic of agent message bubbles and timeline entries, ensuring cleaner code and better maintainability. - Updated test cases for `splitAgentMessageIntoBubbles` and `formatTimelineEntry` to reflect formatting changes and ensure accuracy. These changes improve code clarity and maintainability while enhancing the overall user experience in conversation threads. * fix: satisfy pre-push lint on fix/tauri * fix(chat): refresh usage counters after responses * refactor(ChatRuntimeProvider): remove redundant import of requestUsageRefresh - Eliminated the duplicate import statement for `requestUsageRefresh` in `ChatRuntimeProvider.tsx`, streamlining the code for better readability and maintainability. * fix(chat): satisfy pre-push checks * refactor(conversations): improve error handling and remove unused title generation logic - Removed the unused `generateThreadTitleIfNeeded` function call from the `Conversations` component, simplifying the message dispatch logic. - Enhanced error handling during message dispatch by using `unwrap()` to catch and log errors, providing clearer feedback on send failures. - Updated the `ChatRuntimeProvider` to ensure proper error logging when generating thread titles, improving observability of issues related to title generation. These changes streamline the conversation handling process and improve the robustness of error management in the chat system. * refactor(conversations): improve formatting of title redaction and streamline test assertions - Reformatted the `redact_title_for_log` function for better readability by adjusting the formatting of the output string. - Simplified the assertion in the timezone test to enhance clarity and maintainability. These changes contribute to cleaner code and improved test structure in the conversations module. * refactor(tokenjuice): sort fact parts for improved formatting in inline summary - Modified the `format_inline` function to sort the fact parts before generating the summary string. This change enhances the readability and consistency of the output by ensuring that facts are presented in a stable order. These changes contribute to better formatted summaries in the token juice module. * fix: address remaining CodeRabbit review comments * refactor: streamline error handling and improve code readability - Updated error handling in prompt loading to use `std::io::Error::other` for better clarity. - Simplified conditional checks using `is_none_or` and `is_some_and` for improved readability. - Refactored string trimming logic to utilize array syntax for better clarity. - Enhanced default implementations for several structs to reduce boilerplate code. These changes contribute to cleaner code and improved maintainability across various modules. * refactor: improve code formatting and structure - Adjusted formatting in `post_process.rs` for better readability by aligning the conditional block. - Combined derive attributes in `tools.rs` for the `ComputerControlConfig` struct to reduce redundancy. - Streamlined entry retrieval in `compatible.rs` by condensing multiple lines into a single line for clarity. These changes enhance code readability and maintainability across the affected modules. --- app/package.json | 16 +- app/src-tauri/src/lib.rs | 36 +- app/src-tauri/tauri.conf.json | 14 - app/src-tauri/vendor/tauri-cef | 2 +- app/src/hooks/usageRefresh.ts | 28 + app/src/hooks/useUsageState.test.ts | 60 +- app/src/hooks/useUsageState.ts | 3 + app/src/pages/Conversations.tsx | 378 ++++-- app/src/providers/ChatRuntimeProvider.tsx | 185 ++- app/src/services/api/threadApi.test.ts | 28 + app/src/services/api/threadApi.ts | 25 + app/src/store/chatRuntimeSlice.ts | 3 + app/src/store/threadSlice.ts | 38 + .../__tests__/agentMessageBubbles.test.ts | 105 ++ .../__tests__/toolTimelineFormatting.test.ts | 67 ++ app/src/utils/agentMessageBubbles.ts | 142 +++ app/src/utils/toolTimelineFormatting.ts | 123 ++ docs/agent-subagent-tool-flow.md | 657 +++++++++++ docs/install.md | 8 +- scripts/ensure-tauri-cli.sh | 11 + scripts/release/sign-and-notarize-macos.sh | 69 +- .../agent/agents/integrations_agent/prompt.md | 11 +- .../agent/agents/orchestrator/agent.toml | 29 +- .../agent/agents/orchestrator/prompt.md | 70 +- .../harness/subagent_runner/extract_tool.rs | 497 ++++++++ .../agent/harness/subagent_runner/handoff.rs | 230 ++++ .../agent/harness/subagent_runner/mod.rs | 46 + .../ops.rs} | 963 ++------------- .../harness/subagent_runner/tool_prep.rs | 234 ++++ .../agent/harness/subagent_runner/types.rs | 99 ++ src/openhuman/composio/ops.rs | 30 +- src/openhuman/composio/providers/gmail/mod.rs | 1 + .../composio/providers/gmail/post_process.rs | 1045 +++++++++++++++++ .../composio/providers/gmail/provider.rs | 11 +- src/openhuman/composio/providers/mod.rs | 1 - .../composio/providers/notion/provider.rs | 2 +- .../composio/providers/post_process.rs | 188 --- src/openhuman/composio/providers/traits.rs | 23 + src/openhuman/composio/tools.rs | 17 +- src/openhuman/config/schema/context.rs | 10 +- src/openhuman/config/schema/tools.rs | 8 +- src/openhuman/context/tool_result_budget.rs | 11 +- src/openhuman/learning/linkedin_enrichment.rs | 2 +- src/openhuman/memory/conversations/mod.rs | 2 +- src/openhuman/memory/conversations/store.rs | 72 ++ src/openhuman/memory/rpc_models.rs | 9 + src/openhuman/providers/compatible.rs | 10 +- src/openhuman/routing/provider.rs | 2 +- src/openhuman/threads/ops.rs | 270 ++++- src/openhuman/threads/schemas.rs | 43 +- src/openhuman/tokenjuice/reduce.rs | 3 +- .../tools/impl/agent/complete_onboarding.rs | 6 + src/openhuman/tools/impl/computer/mouse.rs | 2 +- .../tools/impl/system/current_time.rs | 167 +++ src/openhuman/tools/impl/system/mod.rs | 2 + src/openhuman/tools/ops.rs | 36 + src/openhuman/voice/hallucination.rs | 2 +- src/openhuman/voice/streaming.rs | 2 +- 58 files changed, 4862 insertions(+), 1292 deletions(-) create mode 100644 app/src/hooks/usageRefresh.ts create mode 100644 app/src/utils/__tests__/agentMessageBubbles.test.ts create mode 100644 app/src/utils/__tests__/toolTimelineFormatting.test.ts create mode 100644 app/src/utils/agentMessageBubbles.ts create mode 100644 app/src/utils/toolTimelineFormatting.ts create mode 100644 docs/agent-subagent-tool-flow.md create mode 100644 src/openhuman/agent/harness/subagent_runner/extract_tool.rs create mode 100644 src/openhuman/agent/harness/subagent_runner/handoff.rs create mode 100644 src/openhuman/agent/harness/subagent_runner/mod.rs rename src/openhuman/agent/harness/{subagent_runner.rs => subagent_runner/ops.rs} (63%) create mode 100644 src/openhuman/agent/harness/subagent_runner/tool_prep.rs create mode 100644 src/openhuman/agent/harness/subagent_runner/types.rs create mode 100644 src/openhuman/composio/providers/gmail/post_process.rs delete mode 100644 src/openhuman/composio/providers/post_process.rs create mode 100644 src/openhuman/tools/impl/system/current_time.rs diff --git a/app/package.json b/app/package.json index 48f6531cc..142149de3 100644 --- a/app/package.json +++ b/app/package.json @@ -5,9 +5,9 @@ "scripts": { "dev": "vite", "dev:web": "vite", - "dev:app": "yarn tauri:ensure && yarn core:stage && bash ../scripts/setup-chromium-safe-storage.sh && source ../scripts/load-dotenv.sh && APPLE_SIGNING_IDENTITY='OpenHuman Dev Signer' cargo tauri dev", + "dev:app": "yarn tauri:ensure && export CEF_PATH=\"$HOME/Library/Caches/tauri-cef\" && yarn core:stage && bash ../scripts/setup-chromium-safe-storage.sh && source ../scripts/load-dotenv.sh && APPLE_SIGNING_IDENTITY='OpenHuman Dev Signer' cargo tauri dev", "dev:cef": "yarn dev:app", - "dev:wry": "yarn tauri:ensure && yarn core:stage && source ../scripts/load-dotenv.sh && cargo tauri dev --no-default-features --features wry", + "dev:wry": "yarn tauri:ensure && export CEF_PATH=\"$HOME/Library/Caches/tauri-cef\" && yarn core:stage && source ../scripts/load-dotenv.sh && cargo tauri dev --no-default-features --features wry", "core:stage": "node ../scripts/stage-core-sidecar.mjs", "tauri:ensure": "bash ../scripts/ensure-tauri-cli.sh", "build": "tsc && vite build", @@ -15,12 +15,12 @@ "compile": "tsc --noEmit", "preview": "vite preview", "tauri": "tauri", - "tauri:build:ui": "yarn tauri:ensure && cargo tauri build -- --bin OpenHuman", - "macos:build:intel": "yarn tauri:ensure && source ../scripts/load-dotenv.sh && cargo tauri build --bundles app dmg --target x86_64-apple-darwin -- --bin OpenHuman", - "macos:build:intel:debug": "yarn tauri:ensure && source ../scripts/load-dotenv.sh && cargo tauri build --debug --bundles app dmg --target x86_64-apple-darwin -- --bin OpenHuman", - "macos:build:debug": "yarn tauri:ensure && source ../scripts/load-dotenv.sh && cargo tauri build --debug --bundles app dmg -- --bin OpenHuman", - "macos:build:release": "yarn tauri:ensure && source ../scripts/load-dotenv.sh && cargo tauri build --bundles app dmg -- --bin OpenHuman", - "macos:build:release:signed": "yarn tauri:ensure && source ../scripts/load-env.sh && cargo tauri build --bundles app dmg -- --bin OpenHuman", + "tauri:build:ui": "yarn tauri:ensure && export CEF_PATH=\"$HOME/Library/Caches/tauri-cef\" && cargo tauri build -- --bin OpenHuman", + "macos:build:intel": "yarn tauri:ensure && export CEF_PATH=\"$HOME/Library/Caches/tauri-cef\" && source ../scripts/load-dotenv.sh && cargo tauri build --bundles app dmg --target x86_64-apple-darwin -- --bin OpenHuman", + "macos:build:intel:debug": "yarn tauri:ensure && export CEF_PATH=\"$HOME/Library/Caches/tauri-cef\" && source ../scripts/load-dotenv.sh && cargo tauri build --debug --bundles app dmg --target x86_64-apple-darwin -- --bin OpenHuman", + "macos:build:debug": "yarn tauri:ensure && export CEF_PATH=\"$HOME/Library/Caches/tauri-cef\" && source ../scripts/load-dotenv.sh && cargo tauri build --debug --bundles app dmg -- --bin OpenHuman", + "macos:build:release": "yarn tauri:ensure && export CEF_PATH=\"$HOME/Library/Caches/tauri-cef\" && source ../scripts/load-dotenv.sh && cargo tauri build --bundles app dmg -- --bin OpenHuman", + "macos:build:release:signed": "yarn tauri:ensure && export CEF_PATH=\"$HOME/Library/Caches/tauri-cef\" && source ../scripts/load-env.sh && cargo tauri build --bundles app dmg -- --bin OpenHuman", "macos:build:sign:release": "yarn macos:build:release:signed", "macos:run": "open '../target/debug/bundle/macos/OpenHuman.app'", "macos:dev": "yarn macos:build:debug && open '../target/debug/bundle/macos/OpenHuman.app'", diff --git a/app/src-tauri/src/lib.rs b/app/src-tauri/src/lib.rs index 1d0d52d16..1a6d8e5f3 100644 --- a/app/src-tauri/src/lib.rs +++ b/app/src-tauri/src/lib.rs @@ -89,6 +89,7 @@ fn overlay_parent_rpc_url() -> Option { Some(trimmed.to_string()) } +#[allow(dead_code)] // Overlay disabled in tauri.conf.json; helper kept for future re-enable. fn pin_overlay_bottom_right(window: &WebviewWindow) { let Ok(Some(monitor)) = window.current_monitor() else { log::warn!("[overlay] could not resolve current monitor for positioning"); @@ -111,6 +112,7 @@ fn pin_overlay_bottom_right(window: &WebviewWindow) { } #[cfg(target_os = "macos")] +#[allow(dead_code)] // Overlay disabled in tauri.conf.json; helper kept for future re-enable. fn configure_overlay_window_macos(window: &WebviewWindow) { // Standard NSWindow cannot float above fullscreen apps on macOS because // fullscreen apps run in a separate Space. Only NSPanel can do this. @@ -612,23 +614,23 @@ pub fn run() { } } - #[cfg(target_os = "macos")] - { - if let Some(window) = app.get_webview_window("overlay") { - configure_overlay_window_macos(&window); - } else { - log::warn!("[overlay] overlay window not found during setup"); - } - } - - if let Some(window) = app.get_webview_window("overlay") { - pin_overlay_bottom_right(&window); - if let Err(err) = window.show() { - log::warn!("[overlay] failed to show overlay on startup: {err}"); - } else { - log::info!("[overlay] overlay shown on startup"); - } - } + // Overlay window is currently disabled in `tauri.conf.json` (the + // `overlay` entry under `app.windows` was removed), so we skip + // the macOS NSPanel reclass + bottom-right pin + initial show + // here. The helpers (`configure_overlay_window_macos`, + // `pin_overlay_bottom_right`) and the React entry point + // (`src/overlay/OverlayApp.tsx`) are kept intact so the overlay + // can be re-enabled by restoring the config entry and the two + // setup blocks below. + // + // #[cfg(target_os = "macos")] + // if let Some(window) = app.get_webview_window("overlay") { + // configure_overlay_window_macos(&window); + // } + // if let Some(window) = app.get_webview_window("overlay") { + // pin_overlay_bottom_right(&window); + // let _ = window.show(); + // } if let Err(err) = setup_tray(app.handle()) { log::error!("[tray] failed to setup tray icon: {err}"); diff --git a/app/src-tauri/tauri.conf.json b/app/src-tauri/tauri.conf.json index 9ae19b16e..3b1b66d89 100644 --- a/app/src-tauri/tauri.conf.json +++ b/app/src-tauri/tauri.conf.json @@ -20,20 +20,6 @@ "decorations": true, "resizable": true, "center": true - }, - { - "label": "overlay", - "title": "OpenHuman Overlay", - "width": 50, - "height": 50, - "transparent": true, - "decorations": false, - "alwaysOnTop": true, - "skipTaskbar": true, - "resizable": false, - "visible": false, - "create": false, - "center": false } ], "security": { diff --git a/app/src-tauri/vendor/tauri-cef b/app/src-tauri/vendor/tauri-cef index c64bea697..7a5f42099 160000 --- a/app/src-tauri/vendor/tauri-cef +++ b/app/src-tauri/vendor/tauri-cef @@ -1 +1 @@ -Subproject commit c64bea6977c6cb922dc9ee269d10367baf2fd4f6 +Subproject commit 7a5f4209955265abbc73f99e9cec14d59667ab48 diff --git a/app/src/hooks/usageRefresh.ts b/app/src/hooks/usageRefresh.ts new file mode 100644 index 000000000..e2abba3fe --- /dev/null +++ b/app/src/hooks/usageRefresh.ts @@ -0,0 +1,28 @@ +import debug from 'debug'; + +type UsageRefreshListener = () => void; + +const listeners = new Set(); +const usageRefreshLog = debug('usage-refresh'); +let dispatchCount = 0; + +export function subscribeUsageRefresh(listener: UsageRefreshListener): () => void { + listeners.add(listener); + usageRefreshLog('[usage-refresh] subscribe listeners=%d', listeners.size); + return () => { + listeners.delete(listener); + usageRefreshLog('[usage-refresh] unsubscribe listeners=%d', listeners.size); + }; +} + +export function requestUsageRefresh(): void { + dispatchCount += 1; + usageRefreshLog('[usage-refresh] dispatch count=%d listeners=%d', dispatchCount, listeners.size); + for (const listener of listeners) { + try { + listener(); + } catch (error) { + usageRefreshLog('[usage-refresh] listener_error count=%d error=%O', dispatchCount, error); + } + } +} diff --git a/app/src/hooks/useUsageState.test.ts b/app/src/hooks/useUsageState.test.ts index fcf2dffc5..080fba929 100644 --- a/app/src/hooks/useUsageState.test.ts +++ b/app/src/hooks/useUsageState.test.ts @@ -1,4 +1,4 @@ -import { renderHook, waitFor } from '@testing-library/react'; +import { act, renderHook, waitFor } from '@testing-library/react'; import { beforeEach, describe, expect, it, vi } from 'vitest'; const mockGetCurrentPlan = vi.fn(); @@ -131,4 +131,62 @@ describe('useUsageState', () => { expect(result.current.isBudgetExhausted).toBe(false); expect(result.current.shouldShowBudgetCompletedMessage).toBe(false); }); + + it('refetches when a global usage refresh is requested', async () => { + const { useUsageState } = await import('./useUsageState'); + const { requestUsageRefresh } = await import('./usageRefresh'); + + mockGetCurrentPlan.mockResolvedValue({ + plan: 'BASIC', + hasActiveSubscription: true, + planExpiry: '2026-05-01T00:00:00.000Z', + subscription: { + id: 'sub_123', + status: 'active', + currentPeriodEnd: '2026-05-01T00:00:00.000Z', + quantity: 1, + }, + monthlyBudgetUsd: 20, + weeklyBudgetUsd: 10, + fiveHourCapUsd: 3, + }); + mockGetTeamUsage + .mockResolvedValueOnce({ + remainingUsd: 9, + cycleBudgetUsd: 10, + cycleLimit5hr: 1, + cycleLimit7day: 1, + fiveHourCapUsd: 3, + fiveHourResetsAt: null, + cycleStartDate: '2026-04-09T00:00:00.000Z', + cycleEndsAt: '2026-04-16T00:00:00.000Z', + bypassCycleLimit: false, + }) + .mockResolvedValueOnce({ + remainingUsd: 7, + cycleBudgetUsd: 10, + cycleLimit5hr: 2, + cycleLimit7day: 3, + fiveHourCapUsd: 3, + fiveHourResetsAt: null, + cycleStartDate: '2026-04-09T00:00:00.000Z', + cycleEndsAt: '2026-04-16T00:00:00.000Z', + bypassCycleLimit: false, + }); + + const { result } = renderHook(() => useUsageState()); + + await waitFor(() => { + expect(result.current.isLoading).toBe(false); + }); + expect(result.current.teamUsage?.remainingUsd).toBe(9); + + act(() => { + requestUsageRefresh(); + }); + + await waitFor(() => { + expect(result.current.teamUsage?.remainingUsd).toBe(7); + }); + }); }); diff --git a/app/src/hooks/useUsageState.ts b/app/src/hooks/useUsageState.ts index 90d2dc5ec..0b9cb0137 100644 --- a/app/src/hooks/useUsageState.ts +++ b/app/src/hooks/useUsageState.ts @@ -3,6 +3,7 @@ import { useCallback, useEffect, useState } from 'react'; import { billingApi } from '../services/api/billingApi'; import { creditsApi, type TeamUsage } from '../services/api/creditsApi'; import type { CurrentPlanData, PlanTier } from '../types/api'; +import { subscribeUsageRefresh } from './usageRefresh'; export interface UsageState { teamUsage: TeamUsage | null; @@ -50,6 +51,8 @@ export function useUsageState(): UsageState { setFetchCount(c => c + 1); }, []); + useEffect(() => subscribeUsageRefresh(refresh), [refresh]); + useEffect(() => { let cancelled = false; setIsLoading(true); diff --git a/app/src/pages/Conversations.tsx b/app/src/pages/Conversations.tsx index 38e8522e4..c11062714 100644 --- a/app/src/pages/Conversations.tsx +++ b/app/src/pages/Conversations.tsx @@ -13,6 +13,7 @@ import { beginInferenceTurn, clearRuntimeForThread, setToolTimelineForThread, + type ToolTimelineEntry, } from '../store/chatRuntimeSlice'; import { useAppDispatch, useAppSelector } from '../store/hooks'; import { selectSocketStatus } from '../store/socketSelectors'; @@ -28,6 +29,7 @@ import { setSelectedThread, } from '../store/threadSlice'; import type { ThreadMessage } from '../types/thread'; +import { parseMarkdownTable, splitAgentMessageIntoBubbles } from '../utils/agentMessageBubbles'; import { openUrl } from '../utils/openUrl'; import { isTauri, @@ -38,6 +40,7 @@ import { openhumanVoiceTranscribeBytes, openhumanVoiceTts, } from '../utils/tauriCommands'; +import { formatTimelineEntry } from '../utils/toolTimelineFormatting'; // Chat uses the reasoning model; `agentic-v1` is reserved for sub-agents // that execute tool calls, not the primary user-facing conversation. @@ -126,6 +129,211 @@ function isAllowedExternalHref(rawHref: string): boolean { } } +type AgentBubblePosition = 'single' | 'first' | 'middle' | 'last'; + +function getAgentBubbleChrome(position: AgentBubblePosition): string { + if (position === 'single') return 'rounded-2xl rounded-bl-md'; + if (position === 'first') return 'rounded-2xl rounded-bl-lg'; + if (position === 'middle') return 'rounded-2xl rounded-tl-md rounded-bl-lg'; + return 'rounded-2xl rounded-tl-md rounded-bl-md'; +} + +function BubbleMarkdown({ content, tone = 'agent' }: { content: string; tone?: 'agent' | 'user' }) { + const proseTone = + tone === 'user' + ? 'prose-invert prose-p:text-white prose-li:text-white prose-a:text-white prose-code:text-white prose-strong:text-white prose-headings:text-white [&_li::marker]:text-white/85' + : 'prose-a:text-primary-500 prose-code:text-primary-700 prose-headings:text-sm [&_li::marker]:text-stone-700'; + + return ( + + ); +} + +function TableCellMarkdown({ content }: { content: string }) { + return ( + + ); +} + +function ToolTimelineBlock({ entries }: { entries: ToolTimelineEntry[] }) { + const latestRunningEntryId = [...entries].reverse().find(entry => entry.status === 'running')?.id; + + const normalizeToolBody = (value?: string): string | undefined => { + if (!value) return undefined; + const trimmed = value.trim(); + if (trimmed.length === 0) return undefined; + if (trimmed === '{}' || trimmed === '[]' || trimmed === 'null') return undefined; + return value; + }; + + return ( +
+ {entries.map(entry => { + const formatted = formatTimelineEntry(entry); + const detailContent = + normalizeToolBody(formatted.detail) ?? normalizeToolBody(entry.argsBuffer); + const shouldAutoExpand = latestRunningEntryId != null && latestRunningEntryId === entry.id; + const statusTone = + entry.status === 'running' + ? { + pill: 'bg-amber-100 text-amber-600', + bubble: 'bg-amber-50 text-amber-900', + code: 'text-amber-800', + chevron: 'text-amber-500', + } + : entry.status === 'success' + ? { + pill: 'bg-sage-100 text-sage-600', + bubble: 'bg-sage-50 text-sage-900', + code: 'text-sage-800', + chevron: 'text-sage-500', + } + : { + pill: 'bg-coral-100 text-coral-600', + bubble: 'bg-coral-50 text-coral-900', + code: 'text-coral-800', + chevron: 'text-coral-500', + }; + + return ( +
+ {detailContent ? ( +
+ + + ▶ + + {formatted.title} + + {entry.status} + + + {formatted.detail ? ( +
+ {formatted.detail} +
+ ) : ( +
+                    {detailContent}
+                  
+ )} +
+ ) : ( +
+ {formatted.title} + + {entry.status} + +
+ )} +
+ ); + })} +
+ ); +} + +function AgentMessageBubble({ + content, + position = 'single', +}: { + content: string; + position?: AgentBubblePosition; +}) { + const table = parseMarkdownTable(content); + const bubbleChrome = getAgentBubbleChrome(position); + + if (table) { + return ( +
+
+ + + + {table.headers.map(header => ( + + ))} + + + + {table.rows.map((row, rowIndex) => ( + + {row.map((cell, cellIndex) => ( + + ))} + + ))} + +
+ {header} +
+ +
+
+
+ ); + } + + return ( +
+ +
+ ); +} + function formatResetTime(isoStr: string): string { const ms = new Date(isoStr).getTime() - Date.now(); if (ms <= 0) return 'now'; @@ -489,7 +697,6 @@ const Conversations = ({ variant = 'page' }: ConversationsProps = {}) => { if (composerBlocked) return; const sendingThreadId = selectedThreadId; - const userMessage: ThreadMessage = { id: `msg_${Date.now()}_${Math.random()}`, content: trimmed, @@ -499,7 +706,13 @@ const Conversations = ({ variant = 'page' }: ConversationsProps = {}) => { createdAt: new Date().toISOString(), }; - void dispatch(addMessageLocal({ threadId: sendingThreadId, message: userMessage })); + try { + await dispatch(addMessageLocal({ threadId: sendingThreadId, message: userMessage })).unwrap(); + } catch (error) { + const msg = error instanceof Error ? error.message : String(error); + setSendError(chatSendError('cloud_send_failed', msg)); + return; + } setInputValue(''); setSendError(null); // Silence timer: fires only if 120s pass without ANY inference progress @@ -751,6 +964,15 @@ const Conversations = ({ variant = 'page' }: ConversationsProps = {}) => { const selectedThreadToolTimeline = selectedThreadId ? (toolTimelineByThread[selectedThreadId] ?? []) : []; + const visibleMessages = messages.filter(msg => !msg.extraMetadata?.hidden); + const hasVisibleMessages = visibleMessages.length > 0; + const latestVisibleMessage = visibleMessages[visibleMessages.length - 1] ?? null; + const latestVisibleAgentMessage = [...visibleMessages] + .reverse() + .find(msg => msg.sender === 'agent'); + const activeSubagentTimelineEntry = selectedThreadToolTimeline.find( + entry => entry.status === 'running' && entry.name.startsWith('subagent:') + ); const selectedInferenceStatus = selectedThreadId ? (inferenceStatusByThread[selectedThreadId] ?? null) : null; @@ -766,6 +988,8 @@ const Conversations = ({ variant = 'page' }: ConversationsProps = {}) => { (inferenceTurnLifecycleByThread[selectedThreadId] === 'started' || inferenceTurnLifecycleByThread[selectedThreadId] === 'streaming') ); + const shouldRenderTimelineBeforeLatestAgentMessage = + selectedThreadToolTimeline.length > 0 && !isSending && Boolean(latestVisibleAgentMessage); const sortedThreads = [...threads].sort( (a, b) => new Date(b.lastMessageAt).getTime() - new Date(a.lastMessageAt).getTime() @@ -846,7 +1070,7 @@ const Conversations = ({ variant = 'page' }: ConversationsProps = {}) => { -
+ {/*
{formatRelativeTime(thread.lastMessageAt)} @@ -855,7 +1079,7 @@ const Conversations = ({ variant = 'page' }: ConversationsProps = {}) => { {thread.messageCount} msg{thread.messageCount !== 1 ? 's' : ''} )} -
+
*/} )) )} @@ -898,7 +1122,7 @@ const Conversations = ({ variant = 'page' }: ConversationsProps = {}) => { )} -
+
{isLoadingMessages ? (
{Array.from({ length: 4 }).map((_, i) => ( @@ -933,53 +1157,55 @@ const Conversations = ({ variant = 'page' }: ConversationsProps = {}) => { Reload
- ) : messages.length > 0 ? ( + ) : hasVisibleMessages ? (
- {messages - .filter(msg => !msg.extraMetadata?.hidden) - .map(msg => ( + {visibleMessages.map(msg => ( +
+ {shouldRenderTimelineBeforeLatestAgentMessage && + latestVisibleAgentMessage?.id === msg.id && ( + + )}
-
- +
+ {msg.sender === 'agent' ? ( +
+ {splitAgentMessageIntoBubbles(msg.content).map( + (segment, index, parts) => { + const position: AgentBubblePosition = + parts.length === 1 + ? 'single' + : index === 0 + ? 'first' + : index === parts.length - 1 + ? 'last' + : 'middle'; + + return ( + + ); + } + )} + {latestVisibleMessage?.id === msg.id && ( +

+ {formatRelativeTime(msg.createdAt)} +

+ )} +
+ ) : ( +
+ + {latestVisibleMessage?.id === msg.id && ( +

+ {formatRelativeTime(msg.createdAt)} +

+ )} +
+ )} {(() => { + if (latestVisibleMessage?.id !== msg.id) return null; const myReactions = (msg.extraMetadata?.myReactions as string[] | undefined) ?? []; const hasReactions = myReactions.length > 0; - // Show reaction row if there are existing reactions (any sender) - // or if this is an agent message (manual picker available) + // Show reaction row only for the most recent visible message. if (!hasReactions && msg.sender !== 'agent') return null; return (
@@ -1081,7 +1307,8 @@ const Conversations = ({ variant = 'page' }: ConversationsProps = {}) => { })()}
- ))} +
+ ))} {isSending && // Suppress the legacy 3-dot placeholder once streaming // output (visible text or thinking) has started — the @@ -1109,7 +1336,7 @@ const Conversations = ({ variant = 'page' }: ConversationsProps = {}) => { (selectedStreamingAssistant.content.length > 0 || selectedStreamingAssistant.thinking.length > 0) && (
-
+
{selectedStreamingAssistant.thinking.length > 0 && (
@@ -1146,39 +1373,24 @@ const Conversations = ({ variant = 'page' }: ConversationsProps = {}) => { {selectedInferenceStatus.phase === 'tool_use' && `Running ${selectedInferenceStatus.activeTool ?? 'tool'}...`} {selectedInferenceStatus.phase === 'subagent' && - `Sub-agent ${selectedInferenceStatus.activeSubagent ?? ''} working...`} + `${ + formatTimelineEntry( + activeSubagentTimelineEntry ?? { + id: 'active-subagent', + name: `subagent:${selectedInferenceStatus.activeSubagent ?? ''}`, + round: selectedInferenceStatus.iteration, + status: 'running', + } + ).title + }...`}
)} {/* Tool call timeline */} - {selectedThreadToolTimeline.length > 0 && ( -
- {selectedThreadToolTimeline.map(entry => ( -
-
- {entry.name} - - {entry.status} - -
- {entry.status === 'running' && - entry.argsBuffer && - entry.argsBuffer.length > 0 && ( -
-                            {entry.argsBuffer}
-                          
- )} -
- ))} -
- )} + {selectedThreadToolTimeline.length > 0 && + !shouldRenderTimelineBeforeLatestAgentMessage && ( + + )} {isSending && rustChat && (
- {messages.length === 0 && suggestedQuestions.length > 0 && !isLoadingSuggestions && ( + {!hasVisibleMessages && suggestedQuestions.length > 0 && !isLoadingSuggestions && (
{suggestedQuestions.map((s, i) => ( diff --git a/app/src/providers/ChatRuntimeProvider.tsx b/app/src/providers/ChatRuntimeProvider.tsx index 9b7e7e24c..769f9fef9 100644 --- a/app/src/providers/ChatRuntimeProvider.tsx +++ b/app/src/providers/ChatRuntimeProvider.tsx @@ -1,6 +1,7 @@ import debug from 'debug'; -import { useEffect, useRef } from 'react'; +import { useCallback, useEffect, useRef } from 'react'; +import { requestUsageRefresh } from '../hooks/usageRefresh'; import { type ChatInferenceStartEvent, type ChatIterationStartEvent, @@ -31,9 +32,11 @@ import { selectSocketStatus } from '../store/socketSelectors'; import { addInferenceResponse, createNewThread, + generateThreadTitleIfNeeded, setActiveThread, setSelectedThread, } from '../store/threadSlice'; +import { formatTimelineEntry, promptFromArgsBuffer } from '../utils/toolTimelineFormatting'; const logChatRuntime = debug('openhuman:chat-runtime'); @@ -122,50 +125,73 @@ const ChatRuntimeProvider = ({ children }: { children: React.ReactNode }) => { return (hash >>> 0).toString(36); }; - const resolveVisibleThreadForProactive = async ( - incomingThreadId: string - ): Promise => { - if (!incomingThreadId.startsWith('proactive:')) { - return incomingThreadId; - } - - const state = store.getState().thread; - const targetFromState = - state.selectedThreadId ?? state.activeThreadId ?? state.threads[0]?.id ?? null; - if (targetFromState) { - return targetFromState; - } - - if (proactiveThreadCreationPromiseRef.current) { - return proactiveThreadCreationPromiseRef.current; - } - - const createPromise: Promise = (async () => { - try { - const newThread = await dispatch(createNewThread()).unwrap(); - dispatch(setSelectedThread(newThread.id)); - return newThread.id; - } catch (error) { - rtLog('proactive_thread_create_failed', { - err: error instanceof Error ? error.message : String(error), - }); - return null; - } finally { - proactiveThreadCreationPromiseRef.current = null; + const resolveVisibleThreadForProactive = useCallback( + async (incomingThreadId: string): Promise => { + if (!incomingThreadId.startsWith('proactive:')) { + return incomingThreadId; } - })(); - proactiveThreadCreationPromiseRef.current = createPromise; - try { - return await createPromise; - } finally { - // no-op: cleared in createPromise.finally - } - }; + const state = store.getState().thread; + const targetFromState = + state.selectedThreadId ?? state.activeThreadId ?? state.threads[0]?.id ?? null; + if (targetFromState) { + return targetFromState; + } + + if (proactiveThreadCreationPromiseRef.current) { + return proactiveThreadCreationPromiseRef.current; + } + + const createPromise: Promise = (async () => { + try { + const newThread = await dispatch(createNewThread()).unwrap(); + dispatch(setSelectedThread(newThread.id)); + return newThread.id; + } catch (error) { + rtLog('proactive_thread_create_failed', { + err: error instanceof Error ? error.message : String(error), + }); + return null; + } finally { + proactiveThreadCreationPromiseRef.current = null; + } + })(); + proactiveThreadCreationPromiseRef.current = createPromise; + + try { + return await createPromise; + } finally { + // no-op: cleared in createPromise.finally + } + }, + [dispatch] + ); useEffect(() => { if (socketStatus !== 'connected') return; + const decorateEntry = (entry: ToolTimelineEntry): ToolTimelineEntry => { + const formatted = formatTimelineEntry(entry); + return { ...entry, displayName: formatted.title, detail: formatted.detail }; + }; + + const findPendingDelegationContext = ( + entries: ToolTimelineEntry[], + round: number + ): { sourceToolName?: string; prompt?: string } => { + for (let i = entries.length - 1; i >= 0; i -= 1) { + const entry = entries[i]; + if (entry.status !== 'running' || entry.round !== round) continue; + if (entry.name === 'spawn_subagent' || entry.name.startsWith('delegate_')) { + return { + sourceToolName: entry.name, + prompt: entry.detail ?? promptFromArgsBuffer(entry.argsBuffer), + }; + } + } + return {}; + }; + rtLog('subscribe_chat_events', { socket: socketStatus }); const cleanup = subscribeChatEvents({ onInferenceStart: (event: ChatInferenceStartEvent) => { @@ -223,23 +249,23 @@ const ChatRuntimeProvider = ({ children }: { children: React.ReactNode }) => { let entries: ToolTimelineEntry[]; if (existingIdx >= 0) { entries = [...existing]; - entries[existingIdx] = { + entries[existingIdx] = decorateEntry({ ...entries[existingIdx], name: event.tool_name, round: event.round, status: 'running', - }; + }); } else { entries = [ ...existing, - { + decorateEntry({ id: event.tool_call_id ?? `${event.thread_id}:${event.round}:${existing.length}:${event.tool_name}`, name: event.tool_name, round: event.round, status: 'running', - }, + }), ]; } dispatch(setToolTimelineForThread({ threadId: event.thread_id, entries })); @@ -310,17 +336,20 @@ const ChatRuntimeProvider = ({ children }: { children: React.ReactNode }) => { ); const existing = store.getState().chatRuntime.toolTimelineByThread[event.thread_id] ?? []; + const pendingContext = findPendingDelegationContext(existing, event.round); dispatch( setToolTimelineForThread({ threadId: event.thread_id, entries: [ ...existing, - { + decorateEntry({ id: `${event.thread_id}:subagent:${event.skill_id}:${event.tool_name}`, name: `subagent:${event.tool_name}`, round: event.round, status: 'running', - }, + detail: pendingContext.prompt, + sourceToolName: pendingContext.sourceToolName, + }), ], }) ); @@ -331,10 +360,10 @@ const ChatRuntimeProvider = ({ children }: { children: React.ReactNode }) => { if (existing.length > 0) { const entries = existing.map(entry => entry.id === subagentRowId && entry.status === 'running' - ? { + ? decorateEntry({ ...entry, status: (event.success ? 'success' : 'error') as ToolTimelineEntryStatus, - } + }) : entry ); dispatch(setToolTimelineForThread({ threadId: event.thread_id, entries })); @@ -408,24 +437,24 @@ const ChatRuntimeProvider = ({ children }: { children: React.ReactNode }) => { let entries: ToolTimelineEntry[]; if (matchIdx >= 0) { entries = [...existing]; - entries[matchIdx] = { + entries[matchIdx] = decorateEntry({ ...entries[matchIdx], argsBuffer: `${entries[matchIdx].argsBuffer ?? ''}${event.delta}`, name: entries[matchIdx].name.length === 0 && event.tool_name ? event.tool_name : entries[matchIdx].name, - }; + }); } else { entries = [ ...existing, - { + decorateEntry({ id: event.tool_call_id, name: event.tool_name ?? '', round: event.round, status: 'running', argsBuffer: event.delta, - }, + }), ]; } dispatch(setToolTimelineForThread({ threadId: event.thread_id, entries })); @@ -482,12 +511,49 @@ const ChatRuntimeProvider = ({ children }: { children: React.ReactNode }) => { ); dispatch(setToolTimelineForThread({ threadId: event.thread_id, entries })); } - if (!event.segment_total) { - void dispatch( - addInferenceResponse({ content: event.full_response, threadId: event.thread_id }) - ); + void (async () => { + try { + await dispatch( + addInferenceResponse({ content: event.full_response, threadId: event.thread_id }) + ).unwrap(); + void dispatch( + generateThreadTitleIfNeeded({ + threadId: event.thread_id, + assistantMessage: event.full_response, + }) + ); + } catch (error) { + rtLog('chat_done_append_failed', { + thread: event.thread_id, + request: event.request_id, + error: error instanceof Error ? error.message : String(error), + }); + } + rtLog('refresh_usage_counter', { + thread: event.thread_id, + request: event.request_id, + reason: 'chat_done', + }); + requestUsageRefresh(); + dispatch(endInferenceTurn({ threadId: event.thread_id })); + dispatch(setActiveThread(null)); + })(); + return; } + + void dispatch( + generateThreadTitleIfNeeded({ + threadId: event.thread_id, + assistantMessage: event.full_response, + }) + ); + rtLog('refresh_usage_counter', { + thread: event.thread_id, + request: event.request_id, + reason: 'chat_done', + }); + requestUsageRefresh(); dispatch(endInferenceTurn({ threadId: event.thread_id })); dispatch(setActiveThread(null)); }, @@ -532,6 +598,13 @@ const ChatRuntimeProvider = ({ children }: { children: React.ReactNode }) => { }) ); } + + rtLog('refresh_usage_counter', { + thread: event.thread_id, + request: event.request_id, + reason: 'chat_error', + }); + requestUsageRefresh(); } dispatch(endInferenceTurn({ threadId: event.thread_id })); @@ -543,7 +616,7 @@ const ChatRuntimeProvider = ({ children }: { children: React.ReactNode }) => { rtLog('unsubscribe_chat_events'); cleanup(); }; - }, [dispatch, socketStatus]); + }, [dispatch, resolveVisibleThreadForProactive, socketStatus]); return <>{children}; }; diff --git a/app/src/services/api/threadApi.test.ts b/app/src/services/api/threadApi.test.ts index 635c5208d..7daf460d2 100644 --- a/app/src/services/api/threadApi.test.ts +++ b/app/src/services/api/threadApi.test.ts @@ -57,4 +57,32 @@ describe('threadApi', () => { }); expect(result).toEqual(message); }); + + it('generates a thread title via threads RPC', async () => { + const thread = { + id: 'default-thread', + title: 'Invoice follow-up', + chatId: null, + isActive: true, + messageCount: 2, + lastMessageAt: '2026-04-10T12:01:00Z', + createdAt: '2026-04-10T12:00:00Z', + }; + mockCallCoreRpc.mockResolvedValueOnce({ data: thread }); + + const { threadApi } = await import('./threadApi'); + const result = await threadApi.generateTitleIfNeeded( + 'default-thread', + 'I can draft the invoice follow-up note for you.' + ); + + expect(mockCallCoreRpc).toHaveBeenCalledWith({ + method: 'openhuman.threads_generate_title', + params: { + thread_id: 'default-thread', + assistant_message: 'I can draft the invoice follow-up note for you.', + }, + }); + expect(result).toEqual(thread); + }); }); diff --git a/app/src/services/api/threadApi.ts b/app/src/services/api/threadApi.ts index d1fdceb41..4ae208d5e 100644 --- a/app/src/services/api/threadApi.ts +++ b/app/src/services/api/threadApi.ts @@ -1,3 +1,5 @@ +import debug from 'debug'; + import type { PurgeResultData, Thread, @@ -19,6 +21,8 @@ function unwrapEnvelope(response: Envelope | T): T { return response as T; } +const generateTitleLog = debug('threadApi.generateTitleIfNeeded'); + export const threadApi = { createNewThread: async (): Promise => { const response = await callCoreRpc>({ @@ -50,6 +54,27 @@ export const threadApi = { return unwrapEnvelope(response); }, + generateTitleIfNeeded: async (threadId: string, assistantMessage?: string): Promise => { + generateTitleLog('enter threadId=%s assistantMessage=%o', threadId, assistantMessage); + try { + const response = await callCoreRpc>({ + method: 'openhuman.threads_generate_title', + params: { thread_id: threadId, assistant_message: assistantMessage }, + }); + const thread = unwrapEnvelope(response); + generateTitleLog('success threadId=%s response=%o thread=%o', threadId, response, thread); + return thread; + } catch (error) { + generateTitleLog( + 'error threadId=%s assistantMessage=%o error=%O', + threadId, + assistantMessage, + error + ); + throw error; + } + }, + updateMessage: async ( threadId: string, messageId: string, diff --git a/app/src/store/chatRuntimeSlice.ts b/app/src/store/chatRuntimeSlice.ts index c13e8fc53..312d364f3 100644 --- a/app/src/store/chatRuntimeSlice.ts +++ b/app/src/store/chatRuntimeSlice.ts @@ -16,6 +16,9 @@ export interface ToolTimelineEntry { round: number; status: ToolTimelineEntryStatus; argsBuffer?: string; + displayName?: string; + detail?: string; + sourceToolName?: string; } export interface StreamingAssistantState { diff --git a/app/src/store/threadSlice.ts b/app/src/store/threadSlice.ts index 2b52395e4..0a72c8058 100644 --- a/app/src/store/threadSlice.ts +++ b/app/src/store/threadSlice.ts @@ -148,6 +148,36 @@ export const addInferenceResponse = createAsyncThunk( } ); +export const generateThreadTitleIfNeeded = createAsyncThunk( + 'thread/generateThreadTitleIfNeeded', + async ( + payload: { threadId: string; assistantMessage?: string }, + { dispatch, rejectWithValue } + ) => { + let thread: Thread; + try { + thread = await threadApi.generateTitleIfNeeded(payload.threadId, payload.assistantMessage); + } catch (error) { + return rejectWithValue( + error instanceof Error ? error.message : 'Failed to generate thread title' + ); + } + + try { + await dispatch(loadThreads()).unwrap(); + } catch (error) { + if (import.meta.env.DEV) { + console.debug('[threadSlice] generateThreadTitleIfNeeded refresh failed', { + threadId: payload.threadId, + error, + }); + } + } + + return thread; + } +); + export const persistReaction = createAsyncThunk( 'thread/persistReaction', async ( @@ -273,6 +303,14 @@ const threadSlice = createSlice({ .addCase(addMessageLocal.fulfilled, (state, action) => { appendMessageToCache(state, action.payload.threadId, action.payload.message); }) + .addCase(generateThreadTitleIfNeeded.fulfilled, (state, action) => { + const idx = state.threads.findIndex(thread => thread.id === action.payload.id); + if (idx >= 0) { + state.threads[idx] = action.payload; + } else { + state.threads = [action.payload, ...state.threads]; + } + }) .addCase(addInferenceResponse.fulfilled, (state, action) => { appendMessageToCache(state, action.payload.threadId, action.payload.message); // Do not clear activeThreadId here: streaming sends many segment append diff --git a/app/src/utils/__tests__/agentMessageBubbles.test.ts b/app/src/utils/__tests__/agentMessageBubbles.test.ts new file mode 100644 index 000000000..58b2788bb --- /dev/null +++ b/app/src/utils/__tests__/agentMessageBubbles.test.ts @@ -0,0 +1,105 @@ +import { describe, expect, it } from 'vitest'; + +import { parseMarkdownTable, splitAgentMessageIntoBubbles } from '../agentMessageBubbles'; + +describe('splitAgentMessageIntoBubbles', () => { + it('returns a single bubble when there are no newlines', () => { + expect(splitAgentMessageIntoBubbles('One line only.')).toEqual(['One line only.']); + }); + + it('returns no bubbles for empty or whitespace-only content', () => { + expect(splitAgentMessageIntoBubbles('')).toEqual([]); + expect(splitAgentMessageIntoBubbles(' \n\n ')).toEqual([]); + }); + + it('keeps single-paragraph newline-separated lines in one bubble', () => { + expect(splitAgentMessageIntoBubbles('First line\nSecond line\nThird line')).toEqual([ + 'First line\nSecond line\nThird line', + ]); + }); + + it('ignores empty lines between bubbles', () => { + expect(splitAgentMessageIntoBubbles('First line\n\nSecond line\n\n\nThird line')).toEqual([ + 'First line', + 'Second line', + 'Third line', + ]); + }); + + it('keeps fenced code blocks together as one bubble', () => { + const content = 'Here is code\n```ts\nconst x = 1;\nconst y = 2;\n```\nDone'; + expect(splitAgentMessageIntoBubbles(content)).toEqual([ + 'Here is code', + '```ts\nconst x = 1;\nconst y = 2;\n```', + 'Done', + ]); + }); + + it('normalizes windows newlines', () => { + expect(splitAgentMessageIntoBubbles('First\r\nSecond\r\nThird')).toEqual([ + 'First\nSecond\nThird', + ]); + }); + + it('keeps markdown tables together as one segment', () => { + const content = + 'Summary\n| Name | Value |\n| --- | --- |\n| Alpha | 1 |\n| Beta | 2 |\nNext step'; + expect(splitAgentMessageIntoBubbles(content)).toEqual([ + 'Summary', + '| Name | Value |\n| --- | --- |\n| Alpha | 1 |\n| Beta | 2 |', + 'Next step', + ]); + }); + + it('keeps double-newline paragraphs in the same bubble', () => { + const content = 'First line\nSecond line\n\nThird paragraph\nFourth line'; + expect(splitAgentMessageIntoBubbles(content)).toEqual([ + 'First line\nSecond line', + 'Third paragraph\nFourth line', + ]); + }); + + it('normalizes 3+ newlines down to double before splitting', () => { + const content = 'One\n\n\n\nTwo\n\n\nThree'; + expect(splitAgentMessageIntoBubbles(content)).toEqual(['One', 'Two', 'Three']); + }); + + it('treats
as a bubble break', () => { + const content = 'First section\n
\nSecond section\n\n
\nThird section'; + expect(splitAgentMessageIntoBubbles(content)).toEqual([ + 'First section', + 'Second section', + 'Third section', + ]); + }); + + it('never returns an hr-only bubble', () => { + expect(splitAgentMessageIntoBubbles('
')).toEqual([]); + expect(splitAgentMessageIntoBubbles('Before\n\n
\n\nAfter')).toEqual(['Before', 'After']); + }); + + it('never returns a markdown thematic-break-only bubble', () => { + expect(splitAgentMessageIntoBubbles('---')).toEqual([]); + expect(splitAgentMessageIntoBubbles('***')).toEqual([]); + expect(splitAgentMessageIntoBubbles('___')).toEqual([]); + expect(splitAgentMessageIntoBubbles('Before\n\n---\n\nAfter')).toEqual(['Before', 'After']); + }); +}); + +describe('parseMarkdownTable', () => { + it('parses a markdown table into headers and rows', () => { + expect( + parseMarkdownTable('| Name | Value |\n| --- | --- |\n| Alpha | 1 |\n| Beta | 2 |') + ).toEqual({ + headers: ['Name', 'Value'], + rows: [ + ['Alpha', '1'], + ['Beta', '2'], + ], + }); + }); + + it('returns null for non-table content', () => { + expect(parseMarkdownTable('Just a normal message')).toBeNull(); + }); +}); diff --git a/app/src/utils/__tests__/toolTimelineFormatting.test.ts b/app/src/utils/__tests__/toolTimelineFormatting.test.ts new file mode 100644 index 000000000..2fcfe3a52 --- /dev/null +++ b/app/src/utils/__tests__/toolTimelineFormatting.test.ts @@ -0,0 +1,67 @@ +import { describe, expect, it } from 'vitest'; + +import type { ToolTimelineEntry } from '../../store/chatRuntimeSlice'; +import { formatTimelineEntry } from '../toolTimelineFormatting'; + +function entry(overrides: Partial): ToolTimelineEntry { + return { id: 'x', name: 'delegate_notion', round: 1, status: 'running', ...overrides }; +} + +describe('formatTimelineEntry', () => { + it('formats integration delegation tools with a user-facing provider label', () => { + expect( + formatTimelineEntry( + entry({ + name: 'delegate_notion', + argsBuffer: JSON.stringify({ prompt: 'Find the project brief in Notion.' }), + }) + ) + ).toEqual({ title: 'Checking your Notion', detail: 'Find the project brief in Notion.' }); + }); + + it('formats spawn_subagent for integrations_agent from toolkit args', () => { + expect( + formatTimelineEntry( + entry({ + name: 'spawn_subagent', + argsBuffer: JSON.stringify({ + agent_id: 'integrations_agent', + prompt: + 'Get my 5 most recent emails. Show subject, sender, date, and a short preview for each.', + toolkit: 'gmail', + }), + }) + ) + ).toEqual({ + title: 'Checking your Gmail', + detail: + 'Get my 5 most recent emails. Show subject, sender, date, and a short preview for each.', + }); + }); + + it('formats spawned integration agents with the inherited prompt', () => { + expect( + formatTimelineEntry( + entry({ + name: 'subagent:integrations_agent', + sourceToolName: 'delegate_notion', + detail: 'Search Notion for the latest roadmap.', + }) + ) + ).toEqual({ title: 'Checking your Notion', detail: 'Search Notion for the latest roadmap.' }); + }); + + it('falls back to humanized generic labels for non-integration subagents', () => { + expect(formatTimelineEntry(entry({ name: 'subagent:researcher' }))).toEqual({ + title: 'Researching', + detail: undefined, + }); + }); + + it('formats composio_list_connections with user-facing copy', () => { + expect(formatTimelineEntry(entry({ name: 'composio_list_connections' }))).toEqual({ + title: 'Viewing your Integrations', + detail: undefined, + }); + }); +}); diff --git a/app/src/utils/agentMessageBubbles.ts b/app/src/utils/agentMessageBubbles.ts new file mode 100644 index 000000000..e6ff2dba7 --- /dev/null +++ b/app/src/utils/agentMessageBubbles.ts @@ -0,0 +1,142 @@ +/** + * Split an agent message into render-time bubble segments. + * + * Normalize excessive vertical whitespace first, then split only on double + * newlines. Fenced code blocks stay intact as a single segment so + * Markdown/code rendering does not fragment unexpectedly. + * Markdown tables also stay grouped so they can render as dedicated table UI. + */ +export function splitAgentMessageIntoBubbles(content: string): string[] { + const normalized = content + .replace(/\r\n/g, '\n') + .replace(/\n{3,}/g, '\n\n') + .replace(/\n?\s*\s*\n?/gi, '\n\n'); + const trimmedContent = normalized.trim(); + if (trimmedContent.length === 0) return []; + if (!normalized.includes('\n')) { + return isVisualSeparatorOnly(trimmedContent) ? [] : [trimmedContent]; + } + + const lines = normalized.split('\n'); + const segments: string[] = []; + let currentLines: string[] = []; + let inFence = false; + + const flushCurrent = () => { + const segment = currentLines.join('\n').trim(); + if (segment.length > 0) { + segments.push(segment); + } + currentLines = []; + }; + + for (let index = 0; index < lines.length; index += 1) { + const line = lines[index]; + const trimmedLine = line.trim(); + + if (!inFence && isMarkdownTableStart(lines, index)) { + if (currentLines.length > 0) { + flushCurrent(); + } + const tableLines = [line, lines[index + 1]]; + index += 2; + while (index < lines.length && looksLikeMarkdownTableRow(lines[index])) { + tableLines.push(lines[index]); + index += 1; + } + index -= 1; + segments.push(tableLines.join('\n').trim()); + continue; + } + + if (trimmedLine.startsWith('```')) { + if (!inFence && currentLines.length > 0) { + flushCurrent(); + } + currentLines.push(line); + inFence = !inFence; + if (!inFence) { + flushCurrent(); + } + continue; + } + + if (inFence) { + currentLines.push(line); + continue; + } + + if (trimmedLine.length === 0) { + if (currentLines.length > 0) { + flushCurrent(); + } + continue; + } + + currentLines.push(line); + } + + flushCurrent(); + return segments.filter(segment => !isVisualSeparatorOnly(segment)); +} + +export interface ParsedMarkdownTable { + headers: string[]; + rows: string[][]; +} + +export function parseMarkdownTable(content: string): ParsedMarkdownTable | null { + const normalized = content.replace(/\r\n/g, '\n').trim(); + const lines = normalized + .split('\n') + .map(line => line.trim()) + .filter(line => line.length > 0); + + if (lines.length < 2) return null; + if (!isMarkdownTableStart(lines, 0)) return null; + + const headers = splitMarkdownTableCells(lines[0]); + const rows = lines.slice(2).map(splitMarkdownTableCells); + + if (headers.length === 0 || rows.some(row => row.length !== headers.length)) { + return null; + } + + return { headers, rows }; +} + +function isMarkdownTableStart(lines: string[], index: number): boolean { + const header = lines[index]; + const separator = lines[index + 1]; + if (!header || !separator) return false; + return looksLikeMarkdownTableRow(header) && looksLikeMarkdownTableSeparator(separator); +} + +function looksLikeMarkdownTableRow(line: string): boolean { + const trimmed = line.trim(); + if (!trimmed.includes('|')) return false; + const cells = splitMarkdownTableCells(trimmed); + return cells.length >= 2; +} + +function looksLikeMarkdownTableSeparator(line: string): boolean { + const cells = splitMarkdownTableCells(line); + if (cells.length < 2) return false; + return cells.every(cell => /^:?-{3,}:?$/.test(cell)); +} + +function splitMarkdownTableCells(line: string): string[] { + return line + .trim() + .replace(/^\|/, '') + .replace(/\|$/, '') + .split('|') + .map(cell => cell.trim()); +} + +function isVisualSeparatorOnly(segment: string): boolean { + const trimmed = segment.trim(); + if (trimmed.length === 0) return true; + if (/^$/i.test(trimmed)) return true; + return /^(?:-{3,}|\*{3,}|_{3,})$/.test(trimmed.replace(/\s+/g, '')); +} diff --git a/app/src/utils/toolTimelineFormatting.ts b/app/src/utils/toolTimelineFormatting.ts new file mode 100644 index 000000000..16478241c --- /dev/null +++ b/app/src/utils/toolTimelineFormatting.ts @@ -0,0 +1,123 @@ +import type { ToolTimelineEntry } from '../store/chatRuntimeSlice'; + +interface ParsedToolArgs { + agent_id?: string; + prompt?: string; + toolkit?: string; +} + +export function formatTimelineEntry(entry: ToolTimelineEntry): { title: string; detail?: string } { + const parsedArgs = parseToolArgs(entry.argsBuffer); + + if (entry.name === 'spawn_subagent' && parsedArgs?.agent_id === 'integrations_agent') { + const provider = + inferIntegrationName(parsedArgs.toolkit) ?? inferIntegrationNameFromPrompt(parsedArgs.prompt); + return { + title: provider ? `Checking your ${provider}` : 'Checking your connected app', + detail: parsedArgs.prompt?.trim() || entry.detail, + }; + } + + if (entry.name === 'integrations_agent' || entry.name === 'subagent:integrations_agent') { + const provider = + inferIntegrationName(entry.sourceToolName) ?? + inferIntegrationName(parsedArgs?.toolkit) ?? + inferIntegrationNameFromPrompt(entry.detail) ?? + inferIntegrationNameFromPrompt(parsedArgs?.prompt); + + return { + title: provider ? `Checking your ${provider}` : 'Checking your connected app', + detail: entry.detail, + }; + } + + if (entry.name === 'subagent:researcher' || entry.name === 'researcher') { + return { title: 'Researching', detail: entry.detail }; + } + if (entry.name === 'composio_list_connections') { + return { title: 'Viewing your Integrations', detail: entry.detail }; + } + if (entry.name === 'subagent:orchestrator' || entry.name === 'orchestrator') { + return { title: 'Planning next steps', detail: entry.detail }; + } + if (entry.name === 'subagent:critic' || entry.name === 'critic') { + return { title: 'Reviewing the work', detail: entry.detail }; + } + if (entry.name === 'subagent:tools_agent' || entry.name === 'tools_agent') { + return { title: 'Using tools', detail: entry.detail }; + } + if (entry.name === 'subagent:code_executor' || entry.name === 'code_executor') { + return { title: 'Running code', detail: entry.detail }; + } + + if (entry.name.startsWith('delegate_')) { + const provider = inferIntegrationName(entry.name); + return { + title: provider ? `Checking your ${provider}` : humanizeIdentifier(entry.name), + detail: entry.detail ?? parsedArgs?.prompt, + }; + } + + return { + title: entry.displayName ?? humanizeIdentifier(entry.name), + detail: entry.detail ?? parsedArgs?.prompt, + }; +} + +export function promptFromArgsBuffer(argsBuffer?: string): string | undefined { + return parseToolArgs(argsBuffer)?.prompt?.trim() || undefined; +} + +export function inferIntegrationName(input?: string): string | undefined { + if (!input) return undefined; + + const delegateMatch = input.match(/^delegate_(.+)$/); + if (delegateMatch) { + return humanizeIdentifier(delegateMatch[1]); + } + + const toolkitMatch = input.match( + /^(gmail|notion|github|slack|discord|linear|jira|google_calendar|google_drive|calendar)$/i + ); + if (toolkitMatch) { + return humanizeIdentifier(toolkitMatch[1]); + } + + return undefined; +} + +function inferIntegrationNameFromPrompt(prompt?: string): string | undefined { + if (!prompt) return undefined; + const known = [ + 'Notion', + 'Gmail', + 'GitHub', + 'Slack', + 'Discord', + 'Linear', + 'Jira', + 'Google Calendar', + 'Google Drive', + ]; + + const lower = prompt.toLowerCase(); + return known.find(name => lower.includes(name.toLowerCase())); +} + +function parseToolArgs(argsBuffer?: string): ParsedToolArgs | null { + if (!argsBuffer) return null; + try { + const parsed = JSON.parse(argsBuffer) as ParsedToolArgs; + return parsed && typeof parsed === 'object' ? parsed : null; + } catch { + return null; + } +} + +function humanizeIdentifier(value: string): string { + return value + .replace(/^subagent:/, '') + .replace(/^delegate_/, '') + .replace(/_/g, ' ') + .replace(/\b\w/g, char => char.toUpperCase()); +} diff --git a/docs/agent-subagent-tool-flow.md b/docs/agent-subagent-tool-flow.md new file mode 100644 index 000000000..51392de48 --- /dev/null +++ b/docs/agent-subagent-tool-flow.md @@ -0,0 +1,657 @@ +# Agent / Subagent / Tool Flow + +This document explains the current runtime flow around the agent harness, with emphasis on: + +- how the main agent turn executes +- how tools are exposed and executed +- how `spawn_subagent` works +- how typed vs fork subagents differ +- where to look when debugging harness and delegation issues + +Scope: current Rust implementation under `src/openhuman/agent/` and `src/openhuman/tools/`. + +## Why This Exists + +The code path is split across several layers: + +- built-in agent definitions in `src/openhuman/agent/agents/` +- harness data + task-local plumbing in `src/openhuman/agent/harness/` +- main session lifecycle in `src/openhuman/agent/harness/session/` +- delegation tools in `src/openhuman/tools/impl/agent/` +- synthesised `delegate_*` tools in `src/openhuman/tools/orchestrator_tools.rs` + +If you only read one file, the system looks simpler than it is. The actual runtime path crosses all of them. + +## File Map + +### Registry and definitions + +- `src/openhuman/agent/agents/loader.rs` + Loads built-in agents from `agent.toml` plus dynamic `prompt.rs` builders. +- `src/openhuman/agent/harness/definition.rs` + Defines `AgentDefinition`, `ToolScope`, `SubagentEntry`, `PromptSource`, and registry-facing data. +- `src/openhuman/agent/harness/mod.rs` + Re-exports the harness entrypoints. + +### Main agent session + +- `src/openhuman/agent/harness/session/builder.rs` + Builds an `Agent`, chooses dispatcher, applies visible-tool filtering, synthesises delegation tools. +- `src/openhuman/agent/harness/session/turn.rs` + Main turn lifecycle, tool execution, parent/fork context setup, transcript persistence, post-turn hooks. + +### Subagent path + +- `src/openhuman/tools/impl/agent/spawn_subagent.rs` + Runtime tool entrypoint for explicit subagent spawns. +- `src/openhuman/agent/harness/fork_context.rs` + Task-local parent and fork context. +- `src/openhuman/agent/harness/subagent_runner.rs` + Typed/fork subagent execution, inner loop, tool filtering, transcript writes, large-result handoff. + +### Generic tool loop / bus path + +- `src/openhuman/agent/harness/tool_loop.rs` + Shared LLM -> tool -> tool result -> LLM loop used by the bus handler and legacy call sites. +- `src/openhuman/agent/bus.rs` + Native event-bus entrypoint `agent.run_turn`. + +## High-Level Model + +There are two related but distinct execution tiers: + +1. `Agent::turn` + This is the stateful session runtime. It owns conversation history, system prompt reuse, memory loading, hooks, transcript resume, and the parent context needed for subagents. + +2. `run_subagent` + This is an isolated delegated run. It does not become a nested full `Agent` session. It runs a smaller inner loop and returns a single compact text result to the parent as a normal tool result. + +That distinction matters when debugging. A subagent is not a second copy of the full session runtime. + +## Flow Diagram + +### Full parent -> tool -> subagent flow + +```text +User message + | + v ++---------------------------+ +| Agent::turn | +| session/turn.rs | ++---------------------------+ + | + | 1. resume transcript if present + | 2. build/reuse system prompt + | 3. load memory context + | 4. install ParentExecutionContext task-local + v ++---------------------------+ +| Parent iteration loop | +| provider call | ++---------------------------+ + | + | provider response + v ++---------------------------+ +| Parse tool calls | +| dispatcher + parser | ++---------------------------+ + | + +-------------------------------+ + | no tool calls | + | | + v | ++---------------------------+ | +| Final assistant text | | +| appended to parent history| | ++---------------------------+ | + | | + v | +Return to caller | + | + | has tool calls + v + +---------------------------+ + | Execute tool calls | + | parent tool runtime | + +---------------------------+ + | + +-------------------+-------------------+ + | | + | regular tool | spawn_subagent + v v + +---------------------------+ +---------------------------+ + | Tool::execute(...) | | SpawnSubagentTool | + +---------------------------+ | impl/agent/ | + | | spawn_subagent.rs | + | result +---------------------------+ + v | + +---------------------------+ | validate args + | append tool result | | lookup AgentDefinition + | to parent history | | publish spawn event + +---------------------------+ v + | +---------------------------+ + +-------------------------->| run_subagent(...) | + | subagent_runner.rs | + +---------------------------+ + | + +-------------------------+-------------------------+ + | | + | typed mode | fork mode + v v + +---------------------------+ +---------------------------+ + | run_typed_mode | | run_fork_mode | + | - resolve model | | - require ForkContext | + | - filter tools | | - replay parent prefix | + | - build narrow prompt | | - reuse parent tool specs | + +---------------------------+ +---------------------------+ + | | + +-------------------------+-------------------------+ + | + v + +---------------------------+ + | run_inner_loop | + | subagent private loop | + +---------------------------+ + | + +---------------------------+---------------------------+ + | | + | no tool calls | tool calls + v v + +---------------------------+ +---------------------------+ + | final child text | | child executes allowed | + | returned to parent tool | | tools, appends results, | + +---------------------------+ | loops again | + +---------------------------+ + | + v + +---------------------------+ + | SpawnSubagentTool returns | + | ToolResult(output) | + +---------------------------+ + | + v + +---------------------------+ + | parent appends tool | + | result to history | + +---------------------------+ + | + v + +---------------------------+ + | next parent iteration | + | synthesizes final answer | + +---------------------------+ +``` + +### Context wiring for subagents + +```text +Agent::turn + | + +--> build ParentExecutionContext + | - provider + | - all_tools / all_tool_specs + | - model / temperature + | - memory / memory_context + | - connected_integrations + | - composio_client + | - tool_call_format + | - session lineage + | + +--> with_parent_context(...) + | + +--> any tool call inside this turn can read current_parent() + | + +--> SpawnSubagentTool + | + +--> run_subagent(...) + | + +--> typed mode uses ParentExecutionContext directly + | + +--> fork mode also requires current_fork() + | + +--> exact parent prompt + prefix replay +``` + +## Startup and Registry Loading + +Built-in agents live under `src/openhuman/agent/agents/*/` as: + +- `agent.toml` +- `prompt.rs` +- optional `prompt.md` kept as nearby reference material + +`loader.rs` parses each `agent.toml`, stamps the source as builtin, and installs the `prompt.rs` builder as `PromptSource::Dynamic`. + +The global `AgentDefinitionRegistry` is initialized at startup. `spawn_subagent` depends on it. If the registry is missing, the tool returns a clear error instead of trying to run. + +Important consequence: agent delegation is data-driven. The runtime does not hardcode an enum of built-in agents. + +## How a Main Agent Session Is Built + +`AgentBuilder::build` in `session/builder.rs` assembles: + +- provider +- full tool registry +- visible tool specs +- memory backend +- prompt builder +- dispatcher +- context manager + +Two tool sets exist at build time: + +- full tool registry: what the runtime can execute +- visible tool set: what the model can see in its schema/prompt + +That split is intentional. The parent may have access to more runtime tools than it exposes directly to the model. + +### Synthesised delegation tools + +For agents with `subagents = [...]` in their definition, the builder synthesises `delegate_*` tools using `collect_orchestrator_tools()`: + +- `SubagentEntry::AgentId("researcher")` becomes an `ArchetypeDelegationTool` +- `SubagentEntry::Skills({ skills = "*" })` expands to one `SkillDelegationTool` per connected integration + +These tools are added to the model-visible surface at build time. They are wrappers around delegation, not standalone business logic. + +## Main Turn Flow + +`Agent::turn` in `session/turn.rs` is the main harness path. + +### 1. Transcript resume and prompt bootstrap + +On a fresh session: + +- it tries to resume a previous transcript for KV-cache reuse +- fetches connected integrations +- fetches learned context +- builds the system prompt once +- stores that system prompt as the first message + +On later turns it deliberately does not rebuild the system prompt. Byte stability is treated as a runtime invariant for backend prefix caching. + +### 2. Memory context injection + +Per turn, it asks the memory loader for relevant context and prepends that context to the user message. This is parent-session behavior. Subagents do not run the same memory lookup path. + +### 3. Parent execution context is captured + +Before the loop starts, `Agent::turn` snapshots a `ParentExecutionContext` and installs it on the task-local via `with_parent_context(...)`. + +That context carries the data subagents need: + +- provider +- all tools and tool specs +- model / temperature +- memory handle +- loaded memory context +- connected integrations +- composio client +- tool call format +- session / transcript lineage + +Without this task-local, `spawn_subagent` cannot work. + +### 4. Iterative provider loop + +For each iteration: + +- context reduction runs first +- the dispatcher converts history into provider messages +- the provider is called +- response text and tool calls are parsed +- tool calls are executed +- tool results are appended to history +- the loop repeats until no tool calls remain + +This is the full parent loop. It also emits progress events and drives post-turn hooks. + +## Tool Execution in the Parent Loop + +The parent loop special-cases delegation but otherwise treats tools generically. + +Core behaviors: + +- unknown or filtered-out tools become structured error results +- `CliRpcOnly` tools are blocked in the autonomous loop +- approval-gated tools can be denied before execution +- successful outputs may be scrubbed / compacted / summarized + +The parent’s history preserves: + +- assistant tool call intent +- tool results +- final assistant response + +That history format is what the next iteration reasons from. + +## Where `spawn_subagent` Enters + +The explicit delegation tool lives in `src/openhuman/tools/impl/agent/spawn_subagent.rs`. + +Its flow is: + +1. parse `agent_id`, `prompt`, optional `context`, optional `toolkit`, optional `mode` +2. require the global `AgentDefinitionRegistry` +3. resolve the target definition +4. run pre-flight validation for `integrations_agent` +5. publish `DomainEvent::SubagentSpawned` +6. call `run_subagent(...)` +7. publish completed or failed event +8. return the subagent’s final text as a normal `ToolResult` + +Important: the parent model never sees the subagent’s internal transcript. It only sees the final tool result string returned by `spawn_subagent`. + +## Typed vs Fork Subagents + +`run_subagent` chooses one of two modes. + +### Typed mode + +Default path. Implemented by `run_typed_mode(...)`. + +Behavior: + +- resolves model from the definition +- filters the parent’s tools down to what the child is allowed to use +- builds a fresh narrow system prompt +- optionally injects inherited memory context +- runs an isolated inner tool loop + +This is the normal specialist-agent path. + +### Fork mode + +Optimization path. Implemented by `run_fork_mode(...)`. + +Behavior: + +- requires a `ForkContext` task-local +- replays the parent’s exact rendered prompt and exact message prefix +- reuses the parent’s tool schema snapshot +- appends only the new fork task prompt +- runs the same inner loop + +This is for prefix-cache reuse, not for stricter isolation. It is deliberately byte-stable and closely coupled to the parent request shape. + +## How Tool Filtering Works for Subagents + +Typed subagents do not get a cloned tool registry. Instead the runner filters the parent’s tool list by index. + +Filtering inputs: + +- `definition.tools` +- `definition.disallowed_tools` +- `definition.skill_filter` +- `SubagentRunOptions.skill_filter_override` +- `definition.extra_tools` + +Additional runtime rules: + +- non-`welcome` subagents lose `complete_onboarding` +- `tools_agent` strips Composio skill tools +- `integrations_agent` with a bound toolkit may inject dynamic per-action Composio tools + +The allowed tool names become both: + +- the execution allowlist +- the prompt-visible tool catalog + +If the model emits a tool call outside that allowlist, the runner feeds back an error result and continues. + +## Prompt Construction for Typed Subagents + +Typed mode creates a `PromptContext` and then does one of: + +- `PromptSource::Dynamic`: call the Rust prompt builder directly +- `PromptSource::Inline` or `PromptSource::File`: load raw body, then wrap it with `render_subagent_system_prompt(...)` + +Definition flags control which standard sections are omitted: + +- `omit_identity` +- `omit_memory_context` +- `omit_safety_preamble` +- `omit_skills_catalog` +- `omit_profile` +- `omit_memory_md` + +This is one of the main token-saving levers in the harness. + +## The Subagent Inner Loop + +The actual delegated execution happens in `run_inner_loop(...)`. + +It is a slimmed-down tool loop: + +- call provider +- parse tool calls +- persist transcript after provider response +- execute tools +- append results +- persist transcript again +- stop on final text or max iterations + +It returns: + +- final output text +- iteration count +- aggregated usage + +Unlike the parent `Agent::turn`, it does not own the broader session lifecycle. + +## Integrations Agent Special Cases + +`integrations_agent` is the trickiest subagent path. + +### Toolkit gate in `spawn_subagent` + +If `agent_id == "integrations_agent"`: + +- `toolkit` is mandatory +- the toolkit must exist in the allowlist +- if it exists but is not connected, the tool returns a success message explaining that authorization is required + +This is intentionally not always treated as a hard tool failure, because disconnected integrations are a user-facing state, not necessarily a runtime error. + +### Text-mode override + +In `run_inner_loop`, `integrations_agent` with tool specs forces text mode instead of native tool calling. + +Why: + +- large Composio JSON schemas can blow provider grammar/context limits + +What changes: + +- tool specs are omitted from the API payload +- XML-style tool instructions are injected into the system prompt +- the runner parses `...` blocks out of plain text +- tool results in text mode are fed back as a user message containing `` tags + +If a delegated integration run looks different from native-tool runs, this is usually why. + +### Large result handoff cache + +For toolkit-scoped `integrations_agent` runs, oversized tool results may be replaced by placeholders and stashed in an in-memory `ResultHandoffCache`. + +The child can then call `extract_from_result(result_id, query)` to ask targeted follow-up questions against the cached payload. + +This is not the same as generic payload summarization. It is a progressive-disclosure path specific to oversized delegated tool outputs. + +## Parent -> Subagent -> Parent Result Shape + +Conceptually the data flow is: + +1. parent model emits `spawn_subagent(...)` +2. tool runtime executes the delegated subagent loop +3. subagent finishes with one final text output +4. `spawn_subagent` returns that text as its tool result +5. parent history receives the tool result +6. parent model gets another iteration and synthesizes the user-facing answer + +The parent does not absorb the child’s internal reasoning trace or full message history. Only the compact final output crosses the boundary. + +## Bus Path vs Session Path + +There are two outer entrypoints to keep straight. + +### `Agent::turn` + +Used for full stateful sessions. This is the richer harness. + +### `agent.run_turn` via `src/openhuman/agent/bus.rs` + +This native event-bus handler calls `run_tool_call_loop(...)` directly using owned Rust payloads. + +It supports: + +- provider reuse +- tool filtering +- per-turn extra tools +- progress streaming + +But it does not create a full `Agent` session object. If you are debugging channel-dispatch behavior, this distinction matters. + +## Debugging Checklist + +### 1. Confirm which execution tier you are in + +Ask first: + +- full `Agent::turn` session? +- bus `agent.run_turn` path? +- explicit `spawn_subagent` tool? +- synthesised `delegate_*` tool leading into `spawn_subagent`? + +If you confuse these, logs will look contradictory. + +### 2. Check registry state + +If delegation fails very early, confirm: + +- `AgentDefinitionRegistry::init_global(...)` ran at startup +- the target agent id exists +- workspace overrides did not shadow the expected built-in definition + +### 3. Check task-local availability + +If `run_subagent` errors with missing context: + +- `NoParentContext` means the tool ran outside a parent turn +- `NoForkContext` means fork mode was requested but the fork snapshot was never installed + +These are wiring issues, not prompt issues. + +### 4. Check tool visibility vs tool execution + +A tool can exist in the parent registry but still be invisible to a child due to: + +- named `ToolScope` +- `disallowed_tools` +- `skill_filter` +- welcome-only stripping +- toolkit narrowing + +If the model says “Unknown tool” or “not available to this sub-agent”, inspect filtering first. + +### 5. Check transcript artifacts + +Subagents persist transcripts per iteration using the parent session lineage plus a child session key. This is useful for debugging partial runs and crashes during tool execution. + +Parent sessions and subagents do not write identical transcript shapes, so compare like with like. + +### 6. Check the provider mode + +If tool calling is malformed, verify whether the run used: + +- native tools +- p-format / xml instructions +- integrations-agent text mode + +The parser and message shape differ. + +## Useful Log Prefixes + +These prefixes are the most useful grep anchors: + +- `[agent_loop]` +- `[agent]` +- `[tool-loop]` +- `[spawn_subagent]` +- `[subagent_runner]` +- `[subagent_runner:typed]` +- `[subagent_runner:fork]` +- `[subagent_runner:text-mode]` +- `[subagent_runner:handoff]` +- `[orchestrator_tools]` +- `[agent::bus]` +- `[transcript]` + +## Best Existing Tests to Read First + +For end-to-end harness behavior: + +- `src/openhuman/agent/harness/session/tests.rs` + - `turn_dispatches_spawn_subagent_through_full_path` + - `turn_dispatches_spawn_subagent_in_fork_mode` + +For runner behavior in isolation: + +- `src/openhuman/agent/harness/subagent_runner.rs` tests + - typed mode returns text + - memory-context inclusion/omission + - tool filtering + - one-tool execution + - blocked tool recovery + - fork prefix replay + - missing parent/fork context errors + +For orchestration-tool synthesis: + +- `src/openhuman/tools/orchestrator_tools.rs` tests + +For generic parent loop behavior: + +- `src/openhuman/agent/tests.rs` + +## Common Failure Modes + +### Subagent never starts + +Usually one of: + +- registry not initialized +- invalid `agent_id` +- missing parent context +- missing fork context + +### Subagent starts but cannot call expected tools + +Usually one of: + +- tool filtered out by definition scope +- `skill_filter` or toolkit override narrowed too aggressively +- tool is `CliRpcOnly` +- dynamic integration tools were not injected because the toolkit/client state was missing + +### Integrations agent behaves unlike other agents + +Usually expected. It may be in text mode and may be using the oversized-result handoff cache. + +### Parent seems to “lose” child reasoning + +Expected. Only the child’s final output is returned to the parent. Internal child history stays isolated. + +## Practical Mental Model + +The safest mental model is: + +- the parent session is the durable conversation runtime +- tools are the execution boundary +- subagents are tool implementations that happen to run their own mini LLM loop +- fork mode is a cache-optimization path, not a different product feature +- `integrations_agent` is a special delegated runtime with extra provider and payload safeguards + +If you debug from that model, the current codebase makes much more sense. diff --git a/docs/install.md b/docs/install.md index e1b570604..3d1e6adc5 100644 --- a/docs/install.md +++ b/docs/install.md @@ -185,10 +185,14 @@ echo "$(cat openhuman-core-${VERSION}-${TARGET}.tar.gz.sha256) openhuman-core-$ The default runtime is **CEF** (bundled Chromium), which requires the **vendored CEF-aware `tauri-cli`** at `app/src-tauri/vendor/tauri-cef/crates/tauri-cli`. The stock `@tauri-apps/cli` does **not** know how to bundle the Chromium Embedded Framework into `OpenHuman.app/Contents/Frameworks/`, so a bundle produced by it panics at startup inside `cef::library_loader::LibraryLoader::new` with `No such file or directory`. -All `cargo tauri` scripts in `app/package.json` (`yarn dev:app`, `yarn macos:build:*`, etc.) run [`scripts/ensure-tauri-cli.sh`](../scripts/ensure-tauri-cli.sh) first, which installs the vendored CLI into `~/.cargo/bin/cargo-tauri` on first use. If you ever overwrite it (e.g. `npm i -g @tauri-apps/cli` or `cargo install tauri-cli`), re-run: +All `cargo tauri` scripts in `app/package.json` (`yarn dev:app`, `yarn macos:build:*`, etc.) run [`scripts/ensure-tauri-cli.sh`](../scripts/ensure-tauri-cli.sh) first, which installs the vendored CLI into `~/.cargo/bin/cargo-tauri` on first use. Those scripts also `export CEF_PATH="$HOME/Library/Caches/tauri-cef"` so that **every** `cef-dll-sys` invocation — the main app's and the inner `cargo build` that `tauri-bundler`'s `build.rs` runs to produce the embedded `cef-helper` — resolves to the same CEF binary distribution. Without this, the embedded helper ends up with bindings from a *different* downloaded CEF than the framework loaded at runtime, and helper processes abort with `FATAL: CefApp_0_CToCpp called with invalid version -1`. + +If you ever overwrite `cargo-tauri` (e.g. `npm i -g @tauri-apps/cli` or `cargo install tauri-cli`), or switch CEF versions, reinstall with `CEF_PATH` set and force a bundler rebuild (touch forces `tauri-bundler/build.rs` to recompile the embedded cef-helper): ```bash -cargo install --locked --path app/src-tauri/vendor/tauri-cef/crates/tauri-cli +export CEF_PATH="$HOME/Library/Caches/tauri-cef" +touch app/src-tauri/vendor/tauri-cef/cef-helper/src/*.rs +cargo install --force --locked --path app/src-tauri/vendor/tauri-cef/crates/tauri-cli ``` --- diff --git a/scripts/ensure-tauri-cli.sh b/scripts/ensure-tauri-cli.sh index c971cd835..45c066e61 100755 --- a/scripts/ensure-tauri-cli.sh +++ b/scripts/ensure-tauri-cli.sh @@ -25,6 +25,16 @@ if [[ ! -f "$VENDOR_CARGO_TOML" ]]; then exit 1 fi +# Pin a single CEF binary distribution location for *every* cef-dll-sys build: +# - the main app's cef-dll-sys (linked into OpenHuman / openhuman_lib) +# - the inner `cargo build` that tauri-bundler's build.rs runs to produce +# the embedded cef-helper that becomes OpenHuman Helper.app/*. +# If these disagree on which CEF dist to use, the helper processes will abort +# with `CefApp_0_CToCpp called with invalid version -1` because the helper's +# bindings and the loaded framework are out of sync. +export CEF_PATH="${CEF_PATH:-$HOME/Library/Caches/tauri-cef}" +mkdir -p "$CEF_PATH" + # Detect whether the currently installed cargo-tauri came from our vendored path. CRATES_TOML="${CARGO_HOME:-$HOME/.cargo}/.crates.toml" if [[ -f "$CRATES_TOML" ]] && grep -q "tauri-cli.*$VENDOR_CLI" "$CRATES_TOML" 2>/dev/null; then @@ -33,5 +43,6 @@ if [[ -f "$CRATES_TOML" ]] && grep -q "tauri-cli.*$VENDOR_CLI" "$CRATES_TOML" 2> fi echo "[ensure-tauri-cli] installing vendored CEF-aware tauri-cli from $VENDOR_CLI" +echo "[ensure-tauri-cli] CEF_PATH=$CEF_PATH" echo "[ensure-tauri-cli] (first install only — takes a few minutes; subsequent runs are instant)" cargo install --locked --path "$VENDOR_CLI" diff --git a/scripts/release/sign-and-notarize-macos.sh b/scripts/release/sign-and-notarize-macos.sh index 0e18ef91c..1618393be 100755 --- a/scripts/release/sign-and-notarize-macos.sh +++ b/scripts/release/sign-and-notarize-macos.sh @@ -43,43 +43,76 @@ echo "[sign] Signing identity imported into $KEYCHAIN" # ── Sign .app contents ────────────────────────────────────────────────────── echo "[sign] Signing .app contents and bundle" -echo "[sign] Bundle contents:" +echo "[sign] Bundle contents (MacOS/):" ls -la "$APP_PATH/Contents/MacOS/" +if [ -d "$APP_PATH/Contents/Frameworks" ]; then + echo "[sign] Bundle contents (Frameworks/):" + ls -la "$APP_PATH/Contents/Frameworks/" +fi MAIN_EXE="$(defaults read "$APP_PATH/Contents/Info.plist" CFBundleExecutable 2>/dev/null || echo "OpenHuman")" echo "[sign] Main executable (from plist): $MAIN_EXE" -# Sign all non-main binaries (sidecars) in MacOS/ +codesign_hardened() { + codesign --force --options runtime \ + --entitlements "$ENTITLEMENTS" \ + --sign "$APPLE_SIGNING_IDENTITY" \ + --timestamp \ + "$@" +} + +# ── Nested Frameworks/ (CEF + Helper apps) ────────────────────────────────── +# Must be signed from the inside out, before the outer .app bundle. +if [ -d "$APP_PATH/Contents/Frameworks" ]; then + # 1. Sign loose dylibs / binaries inside any *.framework + while IFS= read -r -d '' fw; do + echo "[sign] Scanning framework: $(basename "$fw")" + while IFS= read -r -d '' item; do + # Skip symlinks; codesign will sign the real file via Versions/Current + [ -L "$item" ] && continue + case "$item" in + *.dylib|*.so) + echo "[sign] Signing lib: \"${item#${APP_PATH}/}\"" + codesign_hardened "$item" + ;; + esac + done < <(find "$fw" -type f -print0) + # Sign the framework bundle itself + echo "[sign] Signing framework bundle: $(basename "$fw")" + codesign_hardened "$fw" + done < <(find "$APP_PATH/Contents/Frameworks" -maxdepth 1 -type d -name '*.framework' -print0) + + # 2. Sign each nested Helper.app (inner binary first, then the bundle) + while IFS= read -r -d '' helper; do + HELPER_EXE="$(defaults read "$helper/Contents/Info.plist" CFBundleExecutable 2>/dev/null || true)" + if [ -n "$HELPER_EXE" ] && [ -f "$helper/Contents/MacOS/$HELPER_EXE" ]; then + echo "[sign] Signing helper binary: $(basename "$helper")/$HELPER_EXE" + codesign_hardened "$helper/Contents/MacOS/$HELPER_EXE" + fi + echo "[sign] Signing helper bundle: $(basename "$helper")" + codesign_hardened "$helper" + done < <(find "$APP_PATH/Contents/Frameworks" -maxdepth 1 -type d -name '*.app' -print0) +fi + +# ── Sidecars and loose binaries in MacOS/ ─────────────────────────────────── for bin in "$APP_PATH/Contents/MacOS/"*; do [ -f "$bin" ] && [ -x "$bin" ] || continue BASENAME="$(basename "$bin")" [ "$BASENAME" = "$MAIN_EXE" ] && continue echo "[sign] Signing sidecar: $BASENAME" - codesign --force --options runtime \ - --entitlements "$ENTITLEMENTS" \ - --sign "$APPLE_SIGNING_IDENTITY" \ - --timestamp \ - "$bin" + codesign_hardened "$bin" done # Sign sidecars in Resources/ if any for bin in "$APP_PATH/Contents/Resources/"openhuman-core-*; do [ -f "$bin" ] || continue echo "[sign] Signing resource sidecar: $(basename "$bin")" - codesign --force --options runtime \ - --entitlements "$ENTITLEMENTS" \ - --sign "$APPLE_SIGNING_IDENTITY" \ - --timestamp \ - "$bin" + codesign_hardened "$bin" done -# Sign the .app bundle itself +# ── Outer .app bundle ─────────────────────────────────────────────────────── echo "[sign] Signing .app bundle..." -codesign --force --options runtime \ - --entitlements "$ENTITLEMENTS" \ - --sign "$APPLE_SIGNING_IDENTITY" \ - --timestamp \ - "$APP_PATH" +codesign_hardened "$APP_PATH" # ── Verify ─────────────────────────────────────────────────────────────────── echo "[sign] Verifying signatures" diff --git a/src/openhuman/agent/agents/integrations_agent/prompt.md b/src/openhuman/agent/agents/integrations_agent/prompt.md index 3d04374dc..c36221fe5 100644 --- a/src/openhuman/agent/agents/integrations_agent/prompt.md +++ b/src/openhuman/agent/agents/integrations_agent/prompt.md @@ -6,9 +6,10 @@ You are the **Integrations Agent**. You interact with one connected external ser - **`composio_list_tools`** — inspect the action catalogue for your bound toolkit. Returns the `function.name` slug + JSON schema for each action. - **`composio_execute`** — run a Composio action: `{ tool: "", arguments: {...} }`. +- **`extract_from_result`** — runtime-provided system tool for oversized-result runs. Use it when a tool returned too much data to inspect directly: pass the prior `result_id` plus a narrow `query`, and it will return only the requested slice from that oversized result. - **Per-action tools** — the toolkit's individual action tools are already registered in your tool list with typed schemas (e.g. `GMAIL_SEND_EMAIL`, `NOTION_CREATE_PAGE`). Prefer calling these directly over the generic `composio_execute`. -You do **not** have `composio_list_toolkits`, `composio_list_connections`, `composio_authorize`, shell, file I/O, or any other capability. Stay inside this surface. +You do **not** have shell, file I/O, or any other capability beyond these permitted system / Composio tools. Stay inside this surface. ## Typical flow @@ -24,9 +25,11 @@ You do **not** have `composio_list_toolkits`, `composio_list_connections`, `comp - **Be precise** — every action expects a specific argument shape. Validate against the schema before calling. - **Report results** — state what action was taken and the outcome, including any cost reported by Composio. -## Handling oversized tool results +## Handling large tool results -When an action returns a very large payload (~100 KB or more), decide based on what the caller asked for. +Action payloads can be chunky. Work from what the caller asked for. + +If a tool returns a `result_id` placeholder, your next step is `extract_from_result({ result_id, query })` with a narrowly scoped query that targets only the caller's requested information. ### Path A — caller wants an answer, not the raw data @@ -38,7 +41,7 @@ Scan the result for the specific facts that answer the question, then synthesise Examples: "show me all open issues", "export my contacts", "give me the full thread". -You cannot write files from this agent. Return a concise summary inline (count, key highlights, representative identifiers) and tell the caller you are returning the structured data so the orchestrator can persist it — the orchestrator, not you, owns file I/O. +You cannot write files from this agent. Return a concise inline structured payload instead: count, key highlights, and representative identifiers. Do **not** claim you exported, saved, persisted, or handed off files, and do **not** imply the orchestrator performed file I/O on your behalf. ### Hard cap diff --git a/src/openhuman/agent/agents/orchestrator/agent.toml b/src/openhuman/agent/agents/orchestrator/agent.toml index 1c70affb0..4bd03514a 100644 --- a/src/openhuman/agent/agents/orchestrator/agent.toml +++ b/src/openhuman/agent/agents/orchestrator/agent.toml @@ -44,15 +44,13 @@ subagents = [ "tools_agent", "critic", "archivist", - # Runtime-dispatched only — the runtime calls the summarizer sub-agent - # directly when a tool returns more than - # `context.summarizer_payload_threshold_tokens` (default 500000). The LLM must NOT be - # able to call this sub-agent itself, so `collect_orchestrator_tools` - # filters out `summarizer` and never synthesises a `delegate_summarizer` - # tool. Listing it here keeps the registration explicit (so it shows - # up in the orchestrator's subagent inventory) while the filter - # enforces the runtime-only contract. - "summarizer", + # NOTE: `summarizer` used to be listed here for the runtime-only + # oversized-tool-result hook. That path is currently disabled + # (`context.summarizer_payload_threshold_tokens = 0`) after recursive + # dispatch was observed. The agent definition is still registered via + # `agents::loader` so the payload_summarizer machinery can resolve it + # if the threshold is ever raised back above zero — it just isn't + # exposed to the orchestrator's subagent inventory right now. { skills = "*" }, ] @@ -74,8 +72,21 @@ hint = "reasoning" # / composio_execute directly. named = [ "query_memory", + "memory_store", + "memory_forget", "read_workspace_state", "ask_user_clarification", "spawn_subagent", "composio_list_connections", + # Time + scheduling — lets the orchestrator answer "what time is it", + # "remind me in 10 minutes", "every morning at 8" directly rather than + # delegating or telling the user it can't. `current_time` grounds + # relative-time parsing; `cron_add` / `cron_list` / `cron_remove` + # manage recurring + one-shot agent/shell jobs; `schedule` is the + # simpler shell-only alias for one-shot reminders. + "current_time", + "cron_add", + "cron_list", + "cron_remove", + "schedule", ] diff --git a/src/openhuman/agent/agents/orchestrator/prompt.md b/src/openhuman/agent/agents/orchestrator/prompt.md index 2865a453b..c9be8ca04 100644 --- a/src/openhuman/agent/agents/orchestrator/prompt.md +++ b/src/openhuman/agent/agents/orchestrator/prompt.md @@ -21,11 +21,79 @@ You are the **Orchestrator**, the senior agent in a multi-agent system. Your rol | **Researcher** | Finding information in docs, web, or files. Compresses to dense markdown. | | **Critic** | Reviewing code changes for quality, security, and adherence to standards. | +## Direct Tools (call these yourself — no delegation needed) + +Some capabilities are cheap, read-only, or purely declarative — delegating them +to a sub-agent wastes a turn. Use these directly: + +| Tool | When to use | +| --------------------------- | --------------------------------------------------------------------------------------------------------- | +| `current_time` | Any time the user refers to "now", "in 10 minutes", "tomorrow", "tonight", or before scheduling anything. | +| `cron_add` / `cron_list` / `cron_remove` | Reminders, recurring tasks, follow-ups. Use `job_type: "agent"` with a `prompt` to have a future agent run fire (e.g. send a pushover reminder). Use cron expressions for recurring, `at` for one-shot absolute times, `every` for fixed intervals. | +| `schedule` | Lightweight alias for one-shot shell reminders. Prefer `cron_add` with `job_type: "agent"` for anything that should produce a user-visible message. | +| `query_memory` | Pull long-term user context (preferences, past conversations, saved notes) before answering personal questions. | +| `memory_store` / `memory_forget` | Persist a fact the user asked you to remember, or drop one they asked you to forget. | +| `read_workspace_state` | Get git status + file tree before planning a code task. | +| `composio_list_connections` | Check which external integrations (Gmail, Notion, GitHub, …) the user has authorised *right now*. Session-start list may be stale. | +| `ask_user_clarification` | Ask one focused question when the request is ambiguous — don't guess. | +| `spawn_subagent` | Escape hatch for agent ids not listed in the delegation table above. | + +**Scheduling rule of thumb.** To "remind me in 10 minutes", call `current_time` +first. If `cron_add` is available and enabled for this runtime, then call +`cron_add` with `schedule = {kind:"at", at:""}`, `job_type:"agent"`, +and a `prompt` that tells a future agent what to deliver (e.g. "Send pushover: +'stand up and stretch'"). If `cron_add` is disabled by config, absent from your +tool list, or returns an error, do not promise the reminder: tell the user you +can't schedule it in this environment and, if helpful, provide the computed time +or a manual fallback. + ## Rules - **Never spawn yourself** — You cannot delegate to another Orchestrator. - **Minimise sub-agents** — Use the fewest agents necessary. Simple questions don't need a DAG. - **Context is expensive** — Pass only relevant context to sub-agents, not everything. - **Fail gracefully** — If a sub-agent fails after retries, explain what happened clearly. -- **Stay concise** — Your final response should be direct and actionable. - **Escalate when appropriate** — If orchestration is the wrong mode or a specialist cannot make progress, hand control back to OpenHuman Core with a concise explanation and let Core handle general interactions. + +## Response Style + +Reply like you're texting a friend: casual, lowercase-ok, as few words as possible without losing meaning. No preamble, no recap, no "I'll now…". + +**Avoid em dashes (—).** Use a comma, period, colon, or just a new bubble instead. + +**Go easy on emojis.** Default to none. At most one, only when it genuinely adds something (e.g. a quick reaction). Never decorate every bubble. + +Split thoughts into separate chat bubbles using a **blank line** (double newline) between them. One idea per bubble. + +When the user asks for something that'll take a moment, first bubble should acknowledge (e.g. "on it", "gotcha", "k checking"), then the next bubble has the result or next step. + +Examples: + +User: remind me to stretch in 10 min +→ +```text +got it + +reminder set for 7:42pm +``` + +User: what's on my calendar tomorrow? +→ +```text +one sec + +nothing on the books — you're free +``` + +User: summarise the last notion doc I edited +→ +```text +checking notion + +"Q2 roadmap" — 3 bullets: ship auth, cut v0.4, hire designer +``` + +Short answers can skip the ack: + +User: what time is it? +→ `7:31pm` diff --git a/src/openhuman/agent/harness/subagent_runner/extract_tool.rs b/src/openhuman/agent/harness/subagent_runner/extract_tool.rs new file mode 100644 index 000000000..fdfa7b6f7 --- /dev/null +++ b/src/openhuman/agent/harness/subagent_runner/extract_tool.rs @@ -0,0 +1,497 @@ +//! `extract_from_result` — a sub-agent-side tool that answers a targeted +//! query against a payload previously stashed by the handoff cache (see +//! [`super::handoff`]). +//! +//! This used to dispatch the `summarizer` archetype as a full sub-agent. +//! That dragged along system-prompt scaffolding, a tool-loop, and an +//! extra inference round for a workload that really only needs one +//! completion call. So the tool now drives `provider.chat_with_system` +//! directly against the extraction model (`"summarization-v1"` — same +//! string [`super::definition::ModelSpec::Hint("summarization").resolve`] +//! would have produced, so router entries keyed on it still apply). +//! +//! Transcript discipline: the LLM call still costs tokens, so every +//! extraction round-trip is persisted as its own `session_raw/` JSONL (+ +//! companion `.md`) under the parent's session chain. Single-shot calls +//! produce one file; chunked calls produce one file per chunk sharing a +//! common `call_seq`. Transcript failures are warnings — they never +//! block the tool result. + +use std::path::{Path, PathBuf}; +use std::sync::{Arc, Mutex as StdMutex}; + +use async_trait::async_trait; +use futures::stream::StreamExt; +use serde_json::{json, Value}; + +use super::handoff::{chunk_content, ResultHandoffCache, HANDOFF_MAX_ENTRIES}; +use crate::openhuman::agent::harness::session::transcript::{ + resolve_keyed_transcript_path, write_transcript, MessageUsage, TranscriptMeta, TurnUsage, +}; +use crate::openhuman::providers::{ChatMessage, Provider}; +use crate::openhuman::tools::{Tool, ToolCategory, ToolResult}; + +// ── Tunables ────────────────────────────────────────────────────────── + +/// Model id used for `extract_from_result` LLM calls. Mirrors the +/// resolution `ModelSpec::Hint("summarization").resolve(...)` would have +/// produced for the retired summarizer sub-agent so routing table +/// entries that targeted the summarizer continue to apply. +const EXTRACT_MODEL_ID: &str = "summarization-v1"; + +/// Temperature for extraction calls. Low but non-zero so the model can +/// pick reasonable phrasings when rewriting identifiers into a compact +/// answer, without straying into creative territory. +const EXTRACT_TEMPERATURE: f64 = 0.2; + +/// Char budget per extraction call. Chosen so a single chunk + prompt +/// scaffolding + output stays well below the extraction model's context +/// window (~196k tokens) — at ~4 chars/token that leaves comfortable +/// headroom for the extraction contract and response. +const EXTRACT_CHUNK_CHAR_BUDGET: usize = 60_000; + +/// System prompt fed to the provider on every `extract_from_result` +/// call. Lifted in spirit from the old `summarizer` agent's prompt but +/// trimmed to the core extraction contract — no fluff about iteration +/// budgets or sub-agent roles because this is a pure tool call. +const EXTRACT_SYSTEM_PROMPT: &str = "\ +You are an extraction assistant. A larger tool output is provided below. \ +Return ONLY the specific facts the user's query asks for. \ +Preserve identifiers verbatim (ids, urls, emails, timestamps, prices). \ +Be compact: no preamble, no commentary, no apologies, no meta-statements. \ +If the payload contains nothing relevant to the query, reply with an \ +empty string — do not invent information."; + +// ── Tool impl ───────────────────────────────────────────────────────── + +/// The `extract_from_result` tool registered into the sub-agent's tool +/// surface when a handoff cache is active (currently: integrations_agent +/// with a toolkit scope). +pub(super) struct ExtractFromResultTool { + cache: Arc, + provider: Arc, + /// Workspace root for transcript writes. + workspace_dir: PathBuf, + /// Parent session chain joined with `__`, e.g. + /// `"1700000000_orchestrator__1700000005_1234_integrations_agent_abc"`. + /// Extract-call transcripts append a unique per-call suffix to this. + parent_chain: String, + /// Logical agent id that owns the calls (e.g. `"integrations_agent"`). + /// Only used to compose a descriptive `agent_name` in transcript meta. + owner_agent_id: String, + /// Monotonic counter so repeated calls within the same millisecond + /// still land on distinct transcript files. + call_seq: StdMutex, +} + +impl ExtractFromResultTool { + pub(super) fn new( + cache: Arc, + provider: Arc, + workspace_dir: PathBuf, + parent_chain: String, + owner_agent_id: String, + ) -> Self { + Self { + cache, + provider, + workspace_dir, + parent_chain, + owner_agent_id, + call_seq: StdMutex::new(0), + } + } + + fn next_call_seq(&self) -> u64 { + let mut guard = self + .call_seq + .lock() + .expect("extract_from_result call_seq mutex poisoned"); + *guard = guard.saturating_add(1); + *guard + } +} + +#[async_trait] +impl Tool for ExtractFromResultTool { + fn name(&self) -> &str { + "extract_from_result" + } + + fn description(&self) -> &str { + "Answer a targeted question against an oversized tool output that was \ + stashed under a `result_id` handle. Use this when a previous tool call \ + returned a placeholder like `result_id=\"res_1\"` because its raw output \ + was too large to show inline. Pass the handle plus a natural-language \ + `query` naming the exact facts/identifiers you need; returns only the \ + extracted answer, not the full payload. Multiple queries against the \ + same `result_id` are allowed — each one is independent." + } + + fn parameters_schema(&self) -> Value { + json!({ + "type": "object", + "properties": { + "result_id": { + "type": "string", + "description": "The handle emitted in the oversized tool output placeholder (e.g. `res_1`)." + }, + "query": { + "type": "string", + "description": "Natural-language question naming the exact facts or identifiers to extract. Be specific." + } + }, + "required": ["result_id", "query"] + }) + } + + fn category(&self) -> ToolCategory { + ToolCategory::System + } + + async fn execute(&self, args: Value) -> anyhow::Result { + let result_id = args.get("result_id").and_then(|v| v.as_str()).unwrap_or(""); + let query = args.get("query").and_then(|v| v.as_str()).unwrap_or(""); + + if result_id.is_empty() || query.is_empty() { + return Ok(ToolResult::error( + "extract_from_result requires non-empty `result_id` and `query`.", + )); + } + + let cached = match self.cache.get(result_id) { + Some(c) => c, + None => { + return Ok(ToolResult::error(format!( + "No cached result found for id '{result_id}'. The handle may have been evicted (cache holds the {HANDOFF_MAX_ENTRIES} most recent entries). Re-run the original tool to get a fresh handle." + ))); + } + }; + + // Fast path: payload fits in a single provider turn. + if cached.content.len() <= EXTRACT_CHUNK_CHAR_BUDGET { + tracing::debug!( + tool = %cached.tool_name, + bytes = cached.content.len(), + "[extract_from_result] single-shot extraction" + ); + return self + .extract_single_shot(&cached.tool_name, &cached.content, query) + .await; + } + + // Slow path: chunk + parallel map. A single call on a payload + // large enough to need the handoff (hundreds of KB common for + // Gmail / Notion list operations) risks either (a) overflowing + // the extraction model's context window, or (b) a low-quality + // single-pass answer that misses facts near the tail. Splitting + // into budgeted chunks and running them in parallel keeps each + // call under its context budget and usually finishes faster + // than a sequential single-shot call on the whole blob. + // + // No reduce stage: per-chunk extracts are concatenated in + // original chunk order. A reduce LLM call adds latency (often + // the slowest single turn) and becomes a single point of + // failure when the upstream provider stalls. For + // listing/extraction queries concatenation is equivalent; for + // top-N / global-ordering queries the caller can post-process. + let chunks = chunk_content(&cached.content, EXTRACT_CHUNK_CHAR_BUDGET); + tracing::info!( + tool = %cached.tool_name, + total_bytes = cached.content.len(), + chunk_count = chunks.len(), + chunk_budget = EXTRACT_CHUNK_CHAR_BUDGET, + "[extract_from_result] chunked extraction" + ); + + // Map stage: each chunk extracts items matching `query` from + // ITS OWN slice only. Dispatched with bounded concurrency — + // `buffer_unordered(MAP_CONCURRENCY)` keeps at most N calls in + // flight at any time. Fully parallel `join_all` was generating + // 504-gateway-timeout storms from the staging proxy when 7+ + // concurrent calls piled onto the upstream; batching at 3 + // trades some wall-clock time for reliability. + const MAP_CONCURRENCY: usize = 3; + let total_chunks = chunks.len(); + + // Each chunk gets its own monotonic call_seq so sibling + // transcripts written in parallel still land on distinct files. + let call_seq_base = self.next_call_seq(); + let workspace_dir = self.workspace_dir.clone(); + let parent_chain = self.parent_chain.clone(); + let owner_agent_id = self.owner_agent_id.clone(); + + // Consume `chunks` with `into_iter` so each async block owns + // its `String` — `buffer_unordered` polls the stream lazily + // and needs futures with no borrows into the enclosing scope. + let map_futures = chunks.into_iter().enumerate().map(|(i, chunk)| { + let provider = self.provider.clone(); + let tool_name = cached.tool_name.clone(); + let query = query.to_string(); + let workspace_dir = workspace_dir.clone(); + let parent_chain = parent_chain.clone(); + let owner_agent_id = owner_agent_id.clone(); + async move { + let user_prompt = format!( + "Tool name: {tool_name}\nChunk {idx} of {total}\n\n\ + Query: {query}\n\n\ + This is one slice of a larger tool output. Extract ONLY \ + items in THIS slice that match the query. Preserve \ + identifiers verbatim. Return an empty string if nothing \ + in this slice is relevant.\n\n\ + --- BEGIN SLICE ---\n{chunk}\n--- END SLICE ---", + idx = i + 1, + total = total_chunks, + ); + let result = provider + .chat_with_system( + Some(EXTRACT_SYSTEM_PROMPT), + &user_prompt, + EXTRACT_MODEL_ID, + EXTRACT_TEMPERATURE, + ) + .await; + + // Persist this chunk's transcript before returning, so + // a partial failure higher up the stream still leaves + // an auditable record on disk. + let transcript_input: Result<&str, String> = match &result { + Ok(text) => Ok(text.as_str()), + Err(e) => Err(e.to_string()), + }; + let chunk_label = format!("chunk{:03}of{:03}", i + 1, total_chunks); + write_extract_transcript( + &workspace_dir, + &parent_chain, + &owner_agent_id, + call_seq_base, + Some(&chunk_label), + EXTRACT_SYSTEM_PROMPT, + &user_prompt, + match &transcript_input { + Ok(s) => Ok(*s), + Err(s) => Err(s.as_str()), + }, + EXTRACT_MODEL_ID, + ); + + (i, result) + } + }); + + let mut map_results: Vec<(usize, _)> = futures::stream::iter(map_futures) + .buffer_unordered(MAP_CONCURRENCY) + .collect() + .await; + // `buffer_unordered` yields futures in completion order; restore + // original chunk order so the concatenated output matches the + // natural ordering of the underlying tool result (e.g. Notion's + // reverse-chrono page list). + map_results.sort_by_key(|(i, _)| *i); + + let partials: Vec = map_results + .into_iter() + .filter_map(|(i, r)| match r { + Ok(text) => { + let trimmed = text.trim(); + if trimmed.is_empty() { + None + } else { + Some(trimmed.to_string()) + } + } + Err(e) => { + tracing::warn!( + chunk_idx = i, + error = %e, + "[extract_from_result] map-stage provider call failed; dropping partial" + ); + None + } + }) + .collect(); + + if partials.is_empty() { + tracing::debug!( + "[extract_from_result] no matching content found across any chunk; returning empty extraction" + ); + return Ok(ToolResult::success(String::new())); + } + + // Concatenate per-chunk summaries in original chunk order. + // `join` with a single partial yields it unchanged (no trailing + // separator), so no special-case is needed. + Ok(ToolResult::success(partials.join("\n\n---\n\n"))) + } +} + +impl ExtractFromResultTool { + async fn extract_single_shot( + &self, + tool_name: &str, + content: &str, + query: &str, + ) -> anyhow::Result { + let user_prompt = format!( + "Tool name: {tool_name}\n\nQuery: {query}\n\n\ + Raw tool output follows. Extract ONLY the information the query \ + asks for.\n\n\ + --- BEGIN ---\n{content}\n--- END ---", + ); + + let call_seq = self.next_call_seq(); + let provider_result = self + .provider + .chat_with_system( + Some(EXTRACT_SYSTEM_PROMPT), + &user_prompt, + EXTRACT_MODEL_ID, + EXTRACT_TEMPERATURE, + ) + .await; + + // Persist the transcript before returning — the LLM call cost + // tokens regardless of whether we ultimately return success. + let transcript_input: Result<&str, String> = match &provider_result { + Ok(text) => Ok(text.as_str()), + Err(e) => Err(e.to_string()), + }; + write_extract_transcript( + &self.workspace_dir, + &self.parent_chain, + &self.owner_agent_id, + call_seq, + None, + EXTRACT_SYSTEM_PROMPT, + &user_prompt, + match &transcript_input { + Ok(s) => Ok(*s), + Err(s) => Err(s.as_str()), + }, + EXTRACT_MODEL_ID, + ); + + match provider_result { + Ok(text) => { + let trimmed = text.trim(); + if trimmed.is_empty() { + tracing::debug!( + "[extract_from_result] provider returned an empty response; returning empty extraction" + ); + Ok(ToolResult::success(String::new())) + } else { + Ok(ToolResult::success(trimmed.to_string())) + } + } + Err(e) => Ok(ToolResult::error(format!( + "extract_from_result: provider call failed: {e}" + ))), + } + } +} + +// ── Transcript writer ───────────────────────────────────────────────── + +/// Persist a single extract-from-result LLM round-trip as its own +/// transcript file under `session_raw/DDMMYYYY/{stem}.jsonl` (+ `.md`). +/// +/// Best-effort: transcript failures are logged and swallowed so a +/// readable-log hiccup never blocks the extraction itself. Appends a +/// short suffix to the parent chain so every call lands on a distinct +/// file (sibling extract calls within the same tool invocation still +/// get unique stems). +fn write_extract_transcript( + workspace_dir: &Path, + parent_chain: &str, + owner_agent_id: &str, + call_seq: u64, + chunk_label: Option<&str>, + system_prompt: &str, + user_prompt: &str, + assistant_output: Result<&str, &str>, + model: &str, +) { + let now = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .unwrap_or_default(); + let unix_ts = now.as_secs(); + let nanos = now.subsec_nanos(); + let chunk_tag = match chunk_label { + Some(label) => format!("_{label}"), + None => String::new(), + }; + let stem = format!("{parent_chain}__extract_{unix_ts}_{nanos:09}_{call_seq:04}{chunk_tag}"); + + let path = match resolve_keyed_transcript_path(workspace_dir, &stem) { + Ok(p) => p, + Err(e) => { + tracing::warn!( + error = %e, + stem = %stem, + "[extract_from_result] could not resolve transcript path; skipping transcript" + ); + return; + } + }; + + let (assistant_text, is_error) = match assistant_output { + Ok(text) => (text.to_string(), false), + Err(err) => (format!("[error] {err}"), true), + }; + + let messages = vec![ + ChatMessage { + role: "system".into(), + content: system_prompt.to_string(), + }, + ChatMessage { + role: "user".into(), + content: user_prompt.to_string(), + }, + ChatMessage { + role: "assistant".into(), + content: assistant_text, + }, + ]; + + // Token counts aren't surfaced by `chat_with_system`; leave cost / + // usage fields zeroed and let the backend's own telemetry fill in + // the blanks when we wire richer accounting later. + let ts_rfc3339 = chrono::Utc::now().to_rfc3339(); + let turn_usage = TurnUsage { + model: model.to_string(), + usage: MessageUsage { + input: 0, + output: 0, + cached_input: 0, + cost_usd: 0.0, + }, + ts: ts_rfc3339.clone(), + }; + + let meta = TranscriptMeta { + agent_name: format!("{owner_agent_id}::extract_from_result"), + dispatcher: "native".into(), + created: ts_rfc3339.clone(), + updated: ts_rfc3339, + turn_count: 1, + input_tokens: 0, + output_tokens: 0, + cached_input_tokens: 0, + charged_amount_usd: 0.0, + }; + + if let Err(e) = write_transcript(&path, &messages, &meta, Some(&turn_usage)) { + tracing::warn!( + error = %e, + path = %path.display(), + "[extract_from_result] transcript write failed" + ); + } else { + tracing::debug!( + path = %path.display(), + is_error, + "[extract_from_result] transcript written" + ); + } +} diff --git a/src/openhuman/agent/harness/subagent_runner/handoff.rs b/src/openhuman/agent/harness/subagent_runner/handoff.rs new file mode 100644 index 000000000..e6955e071 --- /dev/null +++ b/src/openhuman/agent/harness/subagent_runner/handoff.rs @@ -0,0 +1,230 @@ +//! Progressive-disclosure handoff cache for oversized tool results. +//! +//! Typed sub-agents (integrations_agent in particular) regularly call tools +//! that return megabyte-scale payloads — `GMAIL_LIST_MESSAGES`, +//! `NOTION_GET_PAGE`, `GOOGLEDRIVE_LIST_FILES`. The default behaviour pushes +//! that raw blob into the sub-agent's history as a tool-result message, and +//! the NEXT iteration ships the bloated history back to the provider where +//! it hits the model's context-length ceiling. +//! +//! Progressive disclosure fixes this: when a tool returns too much data we +//! stash the full payload here, replace it in history with a short +//! placeholder (size + preview + `result_id` + how to query it), and expose +//! an `extract_from_result` tool (see [`super::extract_tool`]) that the +//! sub-agent can call with a targeted query. The extractor only runs when +//! the sub-agent actually asks for a narrower view. +//! +//! This module owns: +//! * the thresholds and limits (token cut-off, preview size, max entries); +//! * the [`ResultHandoffCache`] store itself (FIFO-evicting, `Arc`-shared); +//! * the [`build_handoff_placeholder`] renderer used when rewriting tool +//! results into history. + +use std::collections::HashMap; +use std::sync::Mutex as StdMutex; + +// ── Tunables ─────────────────────────────────────────────────────────────── + +/// Token threshold above which a tool result is routed to the handoff +/// cache instead of being pushed into history raw. Token count is +/// estimated at ~4 chars/token (mirrors +/// `crate::openhuman::agent::harness::payload_summarizer` and +/// `crate::openhuman::tree_summarizer::types::estimate_tokens`). +/// +/// Set at `50_000` so the clean Gmail / Notion envelopes emitted by provider +/// post-processing fit through unchanged for normal workloads — only +/// genuinely oversized results (bulk fetches, raw thread dumps) are routed +/// through the `extract_from_result` path. +pub(super) const HANDOFF_OVERSIZE_THRESHOLD_TOKENS: usize = 50_000; + +/// Characters of the raw payload to surface in the placeholder preview. +/// Enough for the sub-agent to recognise the shape (JSON keys, first +/// record) and often small enough to answer trivial questions without a +/// follow-up `extract_from_result` call. +pub(super) const HANDOFF_PREVIEW_CHARS: usize = 1500; + +/// Maximum entries per session. Bounded to keep memory use predictable on +/// long-running sub-agents that might call many large tools. When over +/// capacity we evict the oldest entry (FIFO); callers see "no cached +/// result" for evicted ids and can either re-run the tool or ask the +/// user/orchestrator to narrow the request. +pub(super) const HANDOFF_MAX_ENTRIES: usize = 8; + +// ── Store ────────────────────────────────────────────────────────────────── + +/// Per-spawn cache of oversized tool payloads. One instance is built at +/// the top of `run_typed_mode` and shared (via `Arc`) with both the inner +/// tool-call loop (writes) and the `extract_from_result` tool (reads). +#[derive(Default)] +pub(super) struct ResultHandoffCache { + inner: StdMutex, +} + +#[derive(Default)] +struct HandoffInner { + /// FIFO of inserted ids, used for eviction. + order: Vec, + /// Content by id. + entries: HashMap, + /// Monotonic counter for id generation within the session. + next_id: u64, +} + +pub(super) struct CachedResult { + pub(super) tool_name: String, + pub(super) content: String, +} + +impl ResultHandoffCache { + pub(super) fn new() -> Self { + Self::default() + } + + /// Stash a payload and return a stable, short, grep-friendly id. + pub(super) fn store(&self, tool_name: String, content: String) -> String { + let mut g = match self.inner.lock() { + Ok(g) => g, + Err(poisoned) => poisoned.into_inner(), + }; + g.next_id = g.next_id.saturating_add(1); + let id = format!("res_{:x}", g.next_id); + g.order.push(id.clone()); + g.entries + .insert(id.clone(), CachedResult { tool_name, content }); + while g.order.len() > HANDOFF_MAX_ENTRIES { + let evicted = g.order.remove(0); + g.entries.remove(&evicted); + } + id + } + + pub(super) fn get(&self, result_id: &str) -> Option { + let g = self.inner.lock().ok()?; + g.entries.get(result_id).map(|r| CachedResult { + tool_name: r.tool_name.clone(), + content: r.content.clone(), + }) + } +} + +// ── Placeholder renderer ─────────────────────────────────────────────────── + +/// Build the placeholder text that replaces an oversized tool result in +/// the sub-agent's history. Shows the payload size (estimated tokens and +/// raw bytes), a preview, and a call shape for the `extract_from_result` +/// tool. The sub-agent decides whether to answer from the preview or +/// dispatch the extractor. +/// +/// Token count is estimated at ~4 chars/token (same heuristic as the +/// trigger threshold in [`HANDOFF_OVERSIZE_THRESHOLD_TOKENS`]), so the +/// unit the sub-agent sees matches the unit the runtime used to decide +/// to hand off in the first place. +pub(super) fn build_handoff_placeholder(tool_name: &str, result_id: &str, raw: &str) -> String { + let preview: String = raw.chars().take(HANDOFF_PREVIEW_CHARS).collect(); + let raw_tokens = raw.len().div_ceil(4); + format!( + "[oversized tool output: {raw_tokens} tokens ({raw_bytes} bytes) — stashed as result_id=\"{result_id}\"]\n\ + Preview (first {preview_chars} chars):\n{preview}\n\n\ + If the preview does not answer your task, call:\n\ + extract_from_result(result_id=\"{result_id}\", query=\"\")\n\ + Good queries name the exact fields/identifiers you need \ + (e.g. \"subject and sender of the 5 most recent messages\"). \ + Tool: {tool_name}", + raw_bytes = raw.len(), + preview_chars = preview.chars().count(), + ) +} + +// ── Content hygiene helpers (used by the extract path) ───────────────────── + +use once_cell::sync::Lazy; +use regex::Regex; + +/// Strip common noise from tool outputs before they're stashed or chunked. +/// +/// Agent tools frequently return raw HTML email bodies, inline SVG, base64 +/// data URIs, CSS/JS blocks, and collapsed whitespace — all of which bloat +/// the handoff cache and waste summarizer context on tokens that carry +/// zero semantic value for most extraction queries. Cleaning before the +/// oversize check means (a) some payloads drop below threshold entirely +/// and skip the extract pipeline, (b) chunked payloads fit more real +/// content per chunk, and (c) summarizers see clean text instead of +/// parsing around markup. +pub(super) fn clean_tool_output(content: &str) -> String { + static SCRIPT_RE: Lazy = + Lazy::new(|| Regex::new(r"(?is)]*>.*?").unwrap()); + static STYLE_RE: Lazy = + Lazy::new(|| Regex::new(r"(?is)]*>.*?").unwrap()); + static SVG_RE: Lazy = + Lazy::new(|| Regex::new(r"(?is)]*>.*?").unwrap()); + static HTML_COMMENT_RE: Lazy = Lazy::new(|| Regex::new(r"(?s)").unwrap()); + static DATA_URI_RE: Lazy = + Lazy::new(|| Regex::new(r"(?i)data:[a-z0-9.+\-/]+;base64,[A-Za-z0-9+/=]+").unwrap()); + static HTML_TAG_RE: Lazy = Lazy::new(|| Regex::new(r"<[^>]+>").unwrap()); + static WS_RE: Lazy = Lazy::new(|| Regex::new(r"[ \t\f\v]+").unwrap()); + static BLANK_LINE_RE: Lazy = Lazy::new(|| Regex::new(r"\n{3,}").unwrap()); + + let cleaned = SCRIPT_RE.replace_all(content, ""); + let cleaned = STYLE_RE.replace_all(&cleaned, ""); + let cleaned = SVG_RE.replace_all(&cleaned, "[svg]"); + let cleaned = HTML_COMMENT_RE.replace_all(&cleaned, ""); + let cleaned = DATA_URI_RE.replace_all(&cleaned, "[data-uri]"); + let cleaned = HTML_TAG_RE.replace_all(&cleaned, ""); + let cleaned = WS_RE.replace_all(&cleaned, " "); + let cleaned = BLANK_LINE_RE.replace_all(&cleaned, "\n\n"); + cleaned.trim().to_string() +} + +/// Split `content` into chunks no larger than `budget` bytes, breaking +/// at natural boundaries (blank lines, then single newlines) so the +/// extraction LLM rarely sees a structure torn mid-record. Falls back to +/// char-safe slicing for pathological single-line inputs. +pub(super) fn chunk_content(content: &str, budget: usize) -> Vec { + if content.len() <= budget { + return vec![content.to_string()]; + } + + let mut chunks: Vec = Vec::new(); + let mut current = String::with_capacity(budget.min(content.len())); + + let flush = |current: &mut String, chunks: &mut Vec| { + if !current.is_empty() { + chunks.push(std::mem::take(current)); + } + }; + + for line in content.lines() { + let projected = current.len() + line.len() + 1; + if projected > budget && !current.is_empty() { + flush(&mut current, &mut chunks); + } + if line.len() > budget { + // Single line exceeds budget (e.g. JSON with no formatting). + // Emit any pending content, then slice the line at char + // boundaries so we don't panic on multi-byte chars. + flush(&mut current, &mut chunks); + let mut remaining = line; + while !remaining.is_empty() { + let mut cut = budget.min(remaining.len()); + while cut > 0 && !remaining.is_char_boundary(cut) { + cut -= 1; + } + if cut == 0 { + // Degenerate case — shouldn't happen for normal + // text. Take the entire remaining line to avoid + // an infinite loop. + chunks.push(remaining.to_string()); + break; + } + chunks.push(remaining[..cut].to_string()); + remaining = &remaining[cut..]; + } + } else { + current.push_str(line); + current.push('\n'); + } + } + flush(&mut current, &mut chunks); + + chunks +} diff --git a/src/openhuman/agent/harness/subagent_runner/mod.rs b/src/openhuman/agent/harness/subagent_runner/mod.rs new file mode 100644 index 000000000..92ba0fcb9 --- /dev/null +++ b/src/openhuman/agent/harness/subagent_runner/mod.rs @@ -0,0 +1,46 @@ +//! Sub-agent execution runner. +//! +//! Given an [`super::definition::AgentDefinition`] and a task prompt, the +//! runner: +//! +//! 1. Reads the [`super::fork_context::ParentExecutionContext`] task-local +//! set by the parent [`crate::openhuman::agent::Agent::turn`]. +//! 2. Resolves the sub-agent's model name (inherit / hint / exact). +//! 3. Filters the parent's tool registry per `definition.tools`, +//! `disallowed_tools`, and `skill_filter` (or, in `fork` mode, +//! inherits the parent's tools verbatim). +//! 4. Builds a narrow system prompt that strips the sections the +//! definition asks to omit (`omit_identity`, `omit_memory_context`, +//! `omit_safety_preamble`, `omit_skills_catalog`). +//! 5. Runs a slim inner tool-call loop using the parent's +//! [`crate::openhuman::providers::Provider`] and returns a single +//! text result. The intra-sub-agent history never leaks back to the +//! parent — the parent only sees one compact tool result. +//! +//! ## Layout +//! +//! This is a light `mod.rs`: every item below is declared in a sibling +//! file and re-exported here. +//! +//! | File | Contents | +//! | ----------------- | ----------------------------------------------------------- | +//! | `types.rs` | `SubagentRun{Options,Outcome,Error}`, `SubagentMode` | +//! | `ops.rs` | `run_subagent`, typed + fork mode, inner tool-call loop | +//! | `handoff.rs` | Oversized-tool-result cache + hygiene helpers | +//! | `extract_tool.rs` | `extract_from_result` tool (direct provider extraction) | +//! | `tool_prep.rs` | Tool filtering + prompt loading + text-mode protocol block | + +mod extract_tool; +mod handoff; +mod ops; +mod tool_prep; +mod types; + +// Public API — the entry point and the shapes it returns. +pub use ops::run_subagent; +pub use types::{SubagentMode, SubagentRunError, SubagentRunOptions, SubagentRunOutcome}; + +// Crate-internal re-exports: `debug_dump` calls the text-mode protocol +// renderer, and `session::builder` reuses the welcome-only guard. The +// other `tool_prep` helpers are used only inside this module. +pub(crate) use tool_prep::{build_text_mode_tool_instructions, is_welcome_only_tool}; diff --git a/src/openhuman/agent/harness/subagent_runner.rs b/src/openhuman/agent/harness/subagent_runner/ops.rs similarity index 63% rename from src/openhuman/agent/harness/subagent_runner.rs rename to src/openhuman/agent/harness/subagent_runner/ops.rs index feca8dd5e..377916506 100644 --- a/src/openhuman/agent/harness/subagent_runner.rs +++ b/src/openhuman/agent/harness/subagent_runner/ops.rs @@ -1,144 +1,39 @@ -//! Sub-agent execution runner. +//! Sub-agent execution entry points and the inner tool-call loop. //! -//! Given an [`AgentDefinition`] and a task prompt, this module: +//! The public runner lives in [`run_subagent`]. It dispatches to +//! [`run_typed_mode`] (narrow prompt + filtered tools) or +//! [`run_fork_mode`] (prefix-replay) depending on the +//! [`super::types::SubagentMode`] implied by the +//! [`crate::openhuman::agent::harness::definition::AgentDefinition`]. //! -//! 1. Reads the [`ParentExecutionContext`] task-local set by the parent -//! [`crate::openhuman::agent::Agent::turn`]. -//! 2. Resolves the sub-agent's model name (inherit / hint / exact). -//! 3. Filters the parent's tool registry per `definition.tools`, -//! `disallowed_tools`, and `skill_filter` (or, in `fork` mode, -//! inherits the parent's tools verbatim). -//! 4. Builds a narrow system prompt that strips the sections the -//! definition asks to omit (`omit_identity`, `omit_memory_context`, -//! `omit_safety_preamble`, `omit_skills_catalog`). -//! 5. Runs a slim inner tool-call loop using the parent's -//! [`crate::openhuman::providers::Provider`] and returns a single -//! text result. The intra-sub-agent history never leaks back to the -//! parent — the parent only sees one compact tool result. -//! -//! Token-saving levers in this runner: -//! - **Narrow prompt** — typed sub-agents skip identity/memory/skills. -//! - **Filtered tools** — fewer schemas in the request body. -//! - **Cheaper model** — archetype hint usually selects a smaller model. -//! - **Lower max iterations** — definition-controlled per archetype. -//! - **No memory recall** — sub-agents skip per-turn memory loading entirely; -//! the parent has already injected the relevant context. -//! - **Structural compaction** — sub-agent's tool-call history collapses -//! into a single tool result block in the parent's history. -//! - **Fork-mode prefix replay** — `uses_fork_context` definitions -//! replay the parent's exact bytes for backend prefix-cache hits. +//! Both modes delegate to [`run_inner_loop`] which drives provider +//! calls and tool execution until the model returns without further +//! tool calls (or the iteration budget is exhausted). -use super::definition::{AgentDefinition, PromptSource, ToolScope}; -use super::fork_context::{current_fork, current_parent, ForkContext, ParentExecutionContext}; -use super::session::transcript; +use std::collections::HashSet; +use std::sync::Arc; +use std::time::Instant; + +use super::super::fork_context::{ + current_fork, current_parent, ForkContext, ParentExecutionContext, +}; +use super::super::session::transcript; +use super::extract_tool::ExtractFromResultTool; +use super::handoff::{ + build_handoff_placeholder, clean_tool_output, ResultHandoffCache, + HANDOFF_OVERSIZE_THRESHOLD_TOKENS, +}; +use super::tool_prep::{ + build_text_mode_tool_instructions, filter_tool_indices, is_subagent_spawn_tool, + is_welcome_only_tool, load_prompt_source, top_k_for_toolkit, +}; +use super::types::{SubagentMode, SubagentRunError, SubagentRunOptions, SubagentRunOutcome}; +use crate::openhuman::agent::harness::definition::{AgentDefinition, PromptSource}; use crate::openhuman::context::prompt::{ render_subagent_system_prompt, PromptContext, PromptTool, SubagentRenderOptions, }; use crate::openhuman::providers::{ChatMessage, ChatRequest, Provider, ToolCall}; -use crate::openhuman::tools::{Tool, ToolCategory, ToolResult, ToolSpec}; -use async_trait::async_trait; -use futures::stream::StreamExt; -use regex::Regex; -use std::collections::{HashMap, HashSet}; -use std::sync::{Arc, LazyLock, Mutex as StdMutex}; -use std::time::{Duration, Instant}; -use thiserror::Error; - -// ───────────────────────────────────────────────────────────────────────────── -// Public API -// ───────────────────────────────────────────────────────────────────────────── - -/// Per-spawn options that override or augment what the -/// [`AgentDefinition`] specifies. Built by `SpawnSubagentTool::execute` -/// from the parent model's call arguments. -#[derive(Debug, Clone, Default)] -pub struct SubagentRunOptions { - /// Optional skill-id override (e.g. `"notion"`). When set, the - /// resolved tool list is further restricted to tools whose name - /// starts with `{skill}__`. Overrides `definition.skill_filter`. - pub skill_filter_override: Option, - - /// Optional Composio toolkit scope (e.g. `"gmail"`, `"notion"`). - /// When set, skill-category tools are further restricted to those - /// whose name starts with the uppercased `{toolkit}_` prefix, and - /// the sub-agent's rendered `Connected Integrations` section is - /// narrowed to only that toolkit's entry. Used by main/orchestrator - /// when spawning `integrations_agent` for a specific platform so the - /// sub-agent only sees one integration's tool catalogue. - pub toolkit_override: Option, - - /// Optional context blob the parent wants to inject before the - /// task prompt. Rendered as a `[Context]\n…\n` prefix. - pub context: Option, - - /// Stable id for tracing / DomainEvents (defaults to a UUID). - pub task_id: Option, -} - -/// Outcome of a single sub-agent run, returned to the parent. -#[derive(Debug, Clone)] -pub struct SubagentRunOutcome { - /// Unique identifier for this sub-task run. - pub task_id: String, - /// The ID of the agent archetype used (e.g., `researcher`). - pub agent_id: String, - /// The final text response produced by the sub-agent. - pub output: String, - /// How many LLM round-trips were performed during the run. - pub iterations: usize, - /// Total wall-clock duration of the run. - pub elapsed: Duration, - /// Which execution mode was used (Typed vs. Fork). - pub mode: SubagentMode, -} - -/// Which prompt-construction path the runner took for a sub-agent. -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -pub enum SubagentMode { - /// Built a narrow, archetype-specific prompt with filtered tools. - Typed, - /// Replayed the parent's exact rendered prompt and history prefix. - Fork, -} - -impl SubagentMode { - pub fn as_str(self) -> &'static str { - match self { - Self::Typed => "typed", - Self::Fork => "fork", - } - } -} - -/// Errors the runner can surface to the parent. The parent receives a -/// stringified version inside a tool result block. -#[derive(Debug, Error)] -pub enum SubagentRunError { - #[error("spawn_subagent called outside of an agent turn — no parent context available")] - NoParentContext, - - #[error( - "fork-mode sub-agent requested but no ForkContext is set on the task-local. \ - Did the parent agent forget to call `Agent::turn` with fork support?" - )] - NoForkContext, - - #[error("agent definition '{0}' not found in registry")] - DefinitionNotFound(String), - - #[error("failed to load archetype prompt from '{path}': {source}")] - PromptLoad { - path: String, - #[source] - source: std::io::Error, - }, - - #[error("provider call failed: {0}")] - Provider(#[from] anyhow::Error), - - #[error("sub-agent exceeded maximum iterations ({0})")] - MaxIterationsExceeded(usize), -} +use crate::openhuman::tools::{Tool, ToolCategory, ToolSpec}; /// Run a sub-agent based on its definition and a task prompt. /// @@ -238,6 +133,27 @@ async fn run_typed_mode( allowed_indices.retain(|&i| !is_welcome_only_tool(parent.all_tools[i].name())); } + // Sub-agents must never spawn their own sub-agents. Nested spawns + // create a recursion tree the harness doesn't budget, observe, or + // cost-attribute — and historically produced runaway dispatch loops + // (e.g. summarizer → summarizer → …). The orchestrator is the only + // node that delegates; every archetype running here is, by + // definition, a sub-agent. Strip `spawn_subagent` and every + // synthesised `delegate_*` tool regardless of the archetype's + // declared scope. This is belt-and-braces: archetype definitions + // should not list these tools either, but we enforce it here so a + // misconfigured TOML can't bypass the rule. + let before = allowed_indices.len(); + allowed_indices.retain(|&i| !is_subagent_spawn_tool(parent.all_tools[i].name())); + let stripped = before - allowed_indices.len(); + if stripped > 0 { + tracing::debug!( + agent_id = %definition.id, + stripped, + "[subagent_runner] removed sub-agent spawn tools from sub-agent's tool surface" + ); + } + // ── Force-include extra_tools ────────────────────────────────────── // // `extra_tools` is a simple "also include these" hook that bypasses @@ -257,6 +173,10 @@ async fn run_typed_mode( if definition.extra_tools.iter().any(|n| n == name) && !allowed_indices.contains(&i) && !disallow_set.contains(name) + // `extra_tools` cannot be used to bypass the sub-agent + // spawn guard above — a stray TOML entry listing + // `spawn_subagent` there must still be dropped. + && !is_subagent_spawn_tool(name) { allowed_indices.push(i); } @@ -373,13 +293,13 @@ async fn run_typed_mode( // too-narrow filter is worse than none — it starves the // sub-agent and forces it to guess. let top_k = top_k_for_toolkit(tk); - let filter_hits = super::tool_filter::filter_actions_by_prompt( + let filter_hits = super::super::tool_filter::filter_actions_by_prompt( task_prompt, &integration.tools, top_k, ); let selected: Vec<&crate::openhuman::context::prompt::ConnectedIntegrationTool> = - if filter_hits.len() >= super::tool_filter::MIN_CONFIDENT_HITS { + if filter_hits.len() >= super::super::tool_filter::MIN_CONFIDENT_HITS { tracing::info!( agent_id = %definition.id, toolkit = %tk, @@ -441,38 +361,40 @@ async fn run_typed_mode( // When enabled, oversized tool results get stashed into this cache // and their place in history is taken by a short placeholder (see // `build_handoff_placeholder`). The sub-agent can then call the - // companion `extract_from_result` tool below to dispatch the - // summarizer sub-agent against the cached payload with a targeted - // query. Lazy / pay-per-question, so trivial asks answerable from - // the preview don't pay any extra LLM cost. + // companion `extract_from_result` tool below to run a direct + // provider call against the cached payload with a targeted query. + // Lazy / pay-per-question, so trivial asks answerable from the + // preview don't pay any extra LLM cost. let handoff_cache: Option> = if is_integrations_agent_with_toolkit { let cache = Arc::new(ResultHandoffCache::new()); - // Resolve the summarizer definition once and register the - // extract_from_result tool alongside the composio action tools. - // If the summarizer isn't in the registry (shouldn't happen in - // production but can happen in tests), skip the tool — tool - // results will still get the placeholder + preview, the - // sub-agent just won't be able to drill in. - if let Some(reg) = - crate::openhuman::agent::harness::definition::AgentDefinitionRegistry::global() - { - if let Some(summarizer_def) = reg.get("summarizer") { - dynamic_tools.push(Box::new(ExtractFromResultTool::new( - cache.clone(), - summarizer_def.clone(), - ))); - tracing::debug!( - agent_id = %definition.id, - "[subagent_runner:typed] registered extract_from_result tool + handoff cache" - ); - } else { - tracing::warn!( - agent_id = %definition.id, - "[subagent_runner:typed] summarizer definition missing from registry — extract_from_result disabled (oversized results will still be cached and previewed)" - ); - } - } + // `extract_from_result` is now a pure tool — it takes the + // parent's provider and calls `chat_with_system` directly + // against the extraction model, instead of spawning the + // `summarizer` sub-agent. Removes an entire layer of harness + // scaffolding (system prompt assembly, tool-loop, recursion + // guards) that this workload never needed. + // + // Transcript plumbing: the extraction LLM still costs tokens, + // so each call writes a self-contained transcript under + // `session_raw/DDMMYYYY/` (and its companion `.md`) keyed by + // the parent chain, to match the rest of the session tree. + let parent_chain = match parent.session_parent_prefix.as_deref() { + Some(prefix) => format!("{}__{}", prefix, parent.session_key), + None => parent.session_key.clone(), + }; + dynamic_tools.push(Box::new(ExtractFromResultTool::new( + cache.clone(), + parent.provider.clone(), + parent.workspace_dir.clone(), + parent_chain, + definition.id.clone(), + ))); + tracing::debug!( + agent_id = %definition.id, + "[subagent_runner:typed] registered extract_from_result tool + handoff cache" + ); + Some(cache) } else { None @@ -614,7 +536,7 @@ async fn run_typed_mode( // Function-driven builder returns the final prompt text. build(&prompt_ctx).map_err(|e| SubagentRunError::PromptLoad { path: format!("", definition.id), - source: std::io::Error::new(std::io::ErrorKind::Other, e.to_string()), + source: std::io::Error::other(e.to_string()), })? } PromptSource::Inline(_) | PromptSource::File { .. } => { @@ -676,7 +598,7 @@ async fn run_typed_mode( task_id, &definition.id, handoff_cache.as_deref(), - &parent, + parent, ) .await?; @@ -741,10 +663,20 @@ async fn run_fork_mode( // request body matches the prefix the backend has already cached. // Runtime execution still resolves against the parent's live tool // registry. + // + // Sub-agents (including fork-mode ones) must not spawn their own + // sub-agents — the rule that applies in `run_typed_mode`'s filter + // applies here too. We keep `spawn_subagent` / `delegate_*` in + // `fork.tool_specs` so the prefix bytes still match the parent's + // cached body (mutating the specs would defeat the whole point of + // fork mode), and instead drop them from `allowed_names` so the + // runtime rejects any attempt to call them with the usual + // "not in allowlist" path. let allowed_names: HashSet = parent .all_tools .iter() .map(|t| t.name().to_string()) + .filter(|name| !is_subagent_spawn_tool(name)) .collect(); let model = parent.model_name.clone(); @@ -770,7 +702,7 @@ async fn run_fork_mode( task_id, &definition.id, None, - &parent, + parent, ) .await?; @@ -1005,7 +937,7 @@ async fn run_inner_loop( // `ToolCall` per parsed block so the rest of the loop can stay // uniform. let native_calls: Vec = if force_text_mode { - let (_cleaned, parsed) = super::parse::parse_tool_calls(&response_text); + let (_cleaned, parsed) = super::super::parse::parse_tool_calls(&response_text); parsed .into_iter() .enumerate() @@ -1054,7 +986,7 @@ async fn run_inner_loop( history.push(ChatMessage::assistant(response_text.clone())); } else { let assistant_history_content = - super::parse::build_native_assistant_history(&response_text, &native_calls); + super::super::parse::build_native_assistant_history(&response_text, &native_calls); history.push(ChatMessage::assistant(assistant_history_content)); } @@ -1207,689 +1139,10 @@ fn parse_tool_arguments(arguments: &str) -> serde_json::Value { .unwrap_or_else(|_| serde_json::Value::Object(Default::default())) } -// ───────────────────────────────────────────────────────────────────────────── -// Oversized-tool-result handoff (progressive disclosure) -// ───────────────────────────────────────────────────────────────────────────── -// -// Typed sub-agents (integrations_agent in particular) regularly call tools that -// return megabyte-scale payloads — `GMAIL_LIST_MESSAGES`, `NOTION_GET_PAGE`, -// `GOOGLEDRIVE_LIST_FILES`. The default behaviour pushes that raw blob into -// the sub-agent's history as a tool-result message, and the NEXT iteration -// then ships the bloated history back to the provider where it hits the -// model's context-length ceiling (observed: 276k-token prompt against a -// 196 607-token window → backend 400). -// -// The summarizer dispatches to the model with the payload + an extraction -// contract, but wiring it to fire on EVERY oversized result has two costs: -// (1) we pay for a summarizer turn even when the sub-agent could answer -// from a preview, and (2) aggressive compression sometimes drops the exact -// identifier the agent needs for a follow-up call. -// -// Progressive disclosure fixes both: when a tool returns too much data, -// we stash the full payload in a session-scoped cache, replace it in -// history with a short placeholder (size + preview + result_id + how to -// query it), and expose an `extract_from_result` tool that the sub-agent -// can call with a targeted query. The summarizer only runs when the -// sub-agent actually asks for a narrower view. - -/// Token threshold above which a tool result is routed to the handoff -/// cache instead of being pushed into history raw. 20 000 tokens keeps -/// the sub-agent's per-iteration prompt well below the 196 607-token -/// model ceiling even after many iterations, and leaves comfortable -/// headroom for the system prompt + tool catalogue. Token count is -/// estimated at ~4 chars/token (mirrors -/// [`crate::openhuman::agent::harness::payload_summarizer`] and -/// [`crate::openhuman::tree_summarizer::types::estimate_tokens`]). -const HANDOFF_OVERSIZE_THRESHOLD_TOKENS: usize = 20_000; - -/// Characters of the raw payload to surface in the placeholder preview. -/// Enough for the sub-agent to recognise the shape (JSON keys, first -/// record) and often small enough to answer trivial questions without a -/// follow-up `extract_from_result` call. -const HANDOFF_PREVIEW_CHARS: usize = 1500; - -/// Maximum entries per session. Bounded to keep memory use predictable -/// on long-running sub-agents that might call many large tools. When -/// over capacity we evict the oldest entry (FIFO); callers see "no -/// cached result" for evicted ids and can either re-run the tool or -/// ask the user/orchestrator to narrow the request. -const HANDOFF_MAX_ENTRIES: usize = 8; - -/// Per-spawn cache of oversized tool payloads. One instance is built -/// at the top of [`run_typed_mode`] and shared (via `Arc`) with both -/// the inner tool-call loop (writes) and the `extract_from_result` -/// tool (reads). -#[derive(Default)] -struct ResultHandoffCache { - inner: StdMutex, -} - -#[derive(Default)] -struct HandoffInner { - /// FIFO of inserted ids, used for eviction. - order: Vec, - /// Content by id. - entries: HashMap, - /// Monotonic counter for id generation within the session. - next_id: u64, -} - -struct CachedResult { - tool_name: String, - content: String, -} - -impl ResultHandoffCache { - fn new() -> Self { - Self::default() - } - - /// Stash a payload and return a stable, short, grep-friendly id. - fn store(&self, tool_name: String, content: String) -> String { - let mut g = match self.inner.lock() { - Ok(g) => g, - Err(poisoned) => poisoned.into_inner(), - }; - g.next_id = g.next_id.saturating_add(1); - let id = format!("res_{:x}", g.next_id); - g.order.push(id.clone()); - g.entries - .insert(id.clone(), CachedResult { tool_name, content }); - while g.order.len() > HANDOFF_MAX_ENTRIES { - let evicted = g.order.remove(0); - g.entries.remove(&evicted); - } - id - } - - fn get(&self, result_id: &str) -> Option { - let g = self.inner.lock().ok()?; - g.entries.get(result_id).map(|r| CachedResult { - tool_name: r.tool_name.clone(), - content: r.content.clone(), - }) - } -} - -/// Build the placeholder text that replaces an oversized tool result in -/// the sub-agent's history. Shows the payload size (estimated tokens -/// and raw bytes), a preview, and a call shape for the -/// `extract_from_result` tool. The sub-agent decides whether to answer -/// from the preview or dispatch the extractor. -/// -/// Token count is estimated at ~4 chars/token (same heuristic as the -/// trigger threshold in `HANDOFF_OVERSIZE_THRESHOLD_TOKENS`), so the -/// unit the sub-agent sees matches the unit the runtime used to decide -/// to hand off in the first place. -fn build_handoff_placeholder(tool_name: &str, result_id: &str, raw: &str) -> String { - let preview: String = raw.chars().take(HANDOFF_PREVIEW_CHARS).collect(); - let raw_tokens = raw.len().div_ceil(4); - format!( - "[oversized tool output: {raw_tokens} tokens ({raw_bytes} bytes) — stashed as result_id=\"{result_id}\"]\n\ - Preview (first {preview_chars} chars):\n{preview}\n\n\ - If the preview does not answer your task, call:\n\ - extract_from_result(result_id=\"{result_id}\", query=\"\")\n\ - Good queries name the exact fields/identifiers you need \ - (e.g. \"subject and sender of the 5 most recent messages\"). \ - Tool: {tool_name}", - raw_bytes = raw.len(), - preview_chars = preview.chars().count(), - ) -} - -/// Sub-agent-side tool that answers a targeted query against a payload -/// previously stashed via [`build_handoff_placeholder`]. Internally -/// dispatches the `summarizer` sub-agent with the cached payload + the -/// caller's query as the extraction contract. -struct ExtractFromResultTool { - cache: Arc, - summarizer_def: AgentDefinition, -} - -impl ExtractFromResultTool { - fn new(cache: Arc, summarizer_def: AgentDefinition) -> Self { - Self { - cache, - summarizer_def, - } - } -} - -#[async_trait] -impl Tool for ExtractFromResultTool { - fn name(&self) -> &str { - "extract_from_result" - } - - fn description(&self) -> &str { - "Answer a targeted question against an oversized tool output that was \ - stashed under a `result_id` handle. Use this when a previous tool call \ - returned a placeholder like `result_id=\"res_1\"` because its raw output \ - was too large to show inline. Pass the handle plus a natural-language \ - `query` naming the exact facts/identifiers you need; returns only the \ - extracted answer, not the full payload. Multiple queries against the \ - same `result_id` are allowed — each one is independent." - } - - fn parameters_schema(&self) -> serde_json::Value { - serde_json::json!({ - "type": "object", - "properties": { - "result_id": { - "type": "string", - "description": "The handle emitted in the oversized tool output placeholder (e.g. `res_1`)." - }, - "query": { - "type": "string", - "description": "Natural-language question naming the exact facts or identifiers to extract. Be specific." - } - }, - "required": ["result_id", "query"] - }) - } - - fn category(&self) -> ToolCategory { - ToolCategory::System - } - - async fn execute(&self, args: serde_json::Value) -> anyhow::Result { - let result_id = args.get("result_id").and_then(|v| v.as_str()).unwrap_or(""); - let query = args.get("query").and_then(|v| v.as_str()).unwrap_or(""); - - if result_id.is_empty() || query.is_empty() { - return Ok(ToolResult::error( - "extract_from_result requires non-empty `result_id` and `query`.", - )); - } - - let cached = match self.cache.get(result_id) { - Some(c) => c, - None => { - return Ok(ToolResult::error(format!( - "No cached result found for id '{result_id}'. The handle may have been evicted (cache holds the {HANDOFF_MAX_ENTRIES} most recent entries). Re-run the original tool to get a fresh handle." - ))); - } - }; - - // Fast path: payload fits in a single summarizer turn. - if cached.content.len() <= SUMMARIZER_CHUNK_CHAR_BUDGET { - tracing::debug!( - tool = %cached.tool_name, - bytes = cached.content.len(), - "[extract_from_result] single-shot extraction" - ); - return self - .extract_single_shot(&cached.tool_name, &cached.content, query) - .await; - } - - // Slow path: chunk + parallel map. A single summarizer call on a - // payload large enough to need the handoff (hundreds of KB - // common for Gmail/Notion list operations) risks either (a) - // overflowing the summarizer's own context window, or (b) - // getting a low-quality single-pass summary that misses facts - // near the tail. Splitting into budgeted chunks and running - // them in parallel keeps each summarizer turn under its - // context budget and usually finishes faster than a sequential - // single-shot call on the whole blob. - // - // No reduce stage: per-chunk summaries are concatenated in - // original chunk order. The reduce LLM call previously used - // here added latency (often the slowest single turn of the - // whole pipeline) and was the single point of failure when - // the upstream provider hung — a hung reduce could burn - // minutes before giving up. For listing/extraction queries - // concatenation is equivalent; for top-N / global-ordering - // queries the caller can post-process. - let chunks = chunk_content(&cached.content, SUMMARIZER_CHUNK_CHAR_BUDGET); - tracing::info!( - tool = %cached.tool_name, - total_bytes = cached.content.len(), - chunk_count = chunks.len(), - chunk_budget = SUMMARIZER_CHUNK_CHAR_BUDGET, - "[extract_from_result] chunked extraction" - ); - - // Map stage: each chunk extracts items matching `query` from - // ITS OWN slice only. Dispatched with bounded concurrency — - // `buffer_unordered(MAP_CONCURRENCY)` keeps at most N summarizer - // calls in flight at any time. Fully parallel `join_all` was - // generating 504-gateway-timeout storms from the staging - // proxy when 7+ concurrent summarizer calls piled onto the - // upstream; batching at 3 trades some wall-clock time for - // reliability. `run_subagent` is isolated per call (fresh - // history, independent summarizer instance). Callers share - // the same parent context. - const MAP_CONCURRENCY: usize = 3; - let total_chunks = chunks.len(); - - // Consume `chunks` with `into_iter` so each async block owns - // its `String` — `buffer_unordered` polls the stream lazily - // and needs futures with no borrows into the enclosing scope. - let map_futures = chunks.into_iter().enumerate().map(|(i, chunk)| { - let summarizer_def = self.summarizer_def.clone(); - let tool_name = cached.tool_name.clone(); - let query = query.to_string(); - async move { - let prompt = format!( - "Tool name: {tool_name}\nChunk {idx} of {total}\n\n\ - Query: {query}\n\n\ - This is one slice of a larger tool output. Extract ONLY \ - items in THIS slice that match the query. Preserve \ - identifiers verbatim (ids, urls, emails, timestamps). \ - Return an empty string if nothing in this slice is \ - relevant. Be concise — no preamble, no commentary on \ - other slices.\n\n\ - --- BEGIN SLICE ---\n{chunk}\n--- END SLICE ---", - idx = i + 1, - total = total_chunks, - ); - ( - i, - run_subagent(&summarizer_def, &prompt, SubagentRunOptions::default()).await, - ) - } - }); - - let mut map_results: Vec<(usize, _)> = futures::stream::iter(map_futures) - .buffer_unordered(MAP_CONCURRENCY) - .collect() - .await; - // `buffer_unordered` yields futures in completion order; restore - // original chunk order so the concatenated output matches the - // natural ordering of the underlying tool result (e.g. Notion's - // reverse-chrono page list). - map_results.sort_by_key(|(i, _)| *i); - - let partials: Vec = map_results - .into_iter() - .filter_map(|(i, r)| match r { - Ok(outcome) => { - let trimmed = outcome.output.trim(); - if trimmed.is_empty() { - None - } else { - Some(trimmed.to_string()) - } - } - Err(e) => { - tracing::warn!( - chunk_idx = i, - error = %e, - "[extract_from_result] map-stage summarizer call failed; dropping partial" - ); - None - } - }) - .collect(); - - if partials.is_empty() { - return Ok(ToolResult::error( - "extract_from_result: no matching content found across any chunk", - )); - } - - // Concatenate per-chunk summaries in original chunk order. - // `join` with a single partial yields it unchanged (no trailing - // separator), so no special-case is needed. - Ok(ToolResult::success(partials.join("\n\n---\n\n"))) - } -} - -impl ExtractFromResultTool { - async fn extract_single_shot( - &self, - tool_name: &str, - content: &str, - query: &str, - ) -> anyhow::Result { - let prompt = format!( - "Tool name: {tool_name}\n\nQuery: {query}\n\n\ - Raw tool output — extract ONLY the information the query asks for. \ - Preserve identifiers verbatim (ids, urls, emails, timestamps). \ - Return a compact, direct answer, no preamble.\n\n\ - --- BEGIN ---\n{content}\n--- END ---", - ); - - match run_subagent(&self.summarizer_def, &prompt, SubagentRunOptions::default()).await { - Ok(run) => { - let trimmed = run.output.trim(); - if trimmed.is_empty() { - Ok(ToolResult::error( - "extract_from_result: summarizer returned an empty response", - )) - } else { - Ok(ToolResult::success(trimmed.to_string())) - } - } - Err(e) => Ok(ToolResult::error(format!( - "extract_from_result: summarizer dispatch failed: {e}" - ))), - } - } -} - -/// Char budget per summarizer call. Chosen so a single chunk + prompt -/// scaffolding + output stays well below the summarization model's -/// context window (~196k tokens) — at ~4 chars/token that leaves -/// comfortable headroom for the extraction contract and response. -const SUMMARIZER_CHUNK_CHAR_BUDGET: usize = 60_000; - -/// Split `content` into chunks no larger than `budget` bytes, breaking -/// at natural boundaries (blank lines, then single newlines) so the -/// summarizer rarely sees a structure torn mid-record. Falls back to -/// char-safe slicing for pathological single-line inputs. -/// Strip common noise from tool outputs before they're stashed or chunked. -/// -/// Agent tools frequently return raw HTML email bodies, inline SVG, base64 -/// data URIs, CSS/JS blocks, and collapsed whitespace — all of which bloat -/// the handoff cache and waste summarizer context on tokens that carry -/// zero semantic value for most extraction queries. Cleaning before the -/// oversize check means (a) some payloads drop below threshold entirely -/// and skip the extract pipeline, (b) chunked payloads fit more real -/// content per chunk, and (c) summarizers see clean text instead of -/// parsing around markup. -/// -/// Applied generically to every tool output — no per-tool gating. The -/// patterns below target universally-noisy markup; non-HTML outputs -/// (plain JSON, plain text) are left essentially untouched because none -/// of these regexes match. -fn clean_tool_output(content: &str) -> String { - // Block-level containers with entirely useless payloads — match lazily - // across lines, case-insensitive. `(?s)` enables `.` matching `\n`. - static SCRIPT_RE: LazyLock = - LazyLock::new(|| Regex::new(r"(?is)]*>.*?").unwrap()); - static STYLE_RE: LazyLock = - LazyLock::new(|| Regex::new(r"(?is)]*>.*?").unwrap()); - static SVG_RE: LazyLock = - LazyLock::new(|| Regex::new(r"(?is)]*>.*?").unwrap()); - static HTML_COMMENT_RE: LazyLock = - LazyLock::new(|| Regex::new(r"(?s)").unwrap()); - // `data:;base64,<...>` inline payloads — keep the agent aware - // an image/asset was there, but drop the bytes. - static DATA_URI_RE: LazyLock = - LazyLock::new(|| Regex::new(r"(?i)data:[a-z0-9.+\-/]+;base64,[A-Za-z0-9+/=]+").unwrap()); - // Everything remaining that still looks like an HTML tag — strip the - // tag but keep the text content. Deliberately greedy across attributes - // but not across `>` so we don't eat inter-tag content. - static HTML_TAG_RE: LazyLock = LazyLock::new(|| Regex::new(r"<[^>]+>").unwrap()); - // Runs of whitespace → single space; collapse vertical bloat. - static WS_RE: LazyLock = LazyLock::new(|| Regex::new(r"[ \t\f\v]+").unwrap()); - static BLANK_LINE_RE: LazyLock = LazyLock::new(|| Regex::new(r"\n{3,}").unwrap()); - - let cleaned = SCRIPT_RE.replace_all(content, ""); - let cleaned = STYLE_RE.replace_all(&cleaned, ""); - let cleaned = SVG_RE.replace_all(&cleaned, "[svg]"); - let cleaned = HTML_COMMENT_RE.replace_all(&cleaned, ""); - let cleaned = DATA_URI_RE.replace_all(&cleaned, "[data-uri]"); - let cleaned = HTML_TAG_RE.replace_all(&cleaned, ""); - let cleaned = WS_RE.replace_all(&cleaned, " "); - let cleaned = BLANK_LINE_RE.replace_all(&cleaned, "\n\n"); - cleaned.trim().to_string() -} - -fn chunk_content(content: &str, budget: usize) -> Vec { - if content.len() <= budget { - return vec![content.to_string()]; - } - - let mut chunks: Vec = Vec::new(); - let mut current = String::with_capacity(budget.min(content.len())); - - let flush = |current: &mut String, chunks: &mut Vec| { - if !current.is_empty() { - chunks.push(std::mem::take(current)); - } - }; - - for line in content.lines() { - let projected = current.len() + line.len() + 1; - if projected > budget && !current.is_empty() { - flush(&mut current, &mut chunks); - } - if line.len() > budget { - // Single line exceeds budget (e.g. JSON with no formatting). - // Emit any pending content, then slice the line at char - // boundaries so we don't panic on multi-byte chars. - flush(&mut current, &mut chunks); - let mut remaining = line; - while !remaining.is_empty() { - let mut cut = budget.min(remaining.len()); - while cut > 0 && !remaining.is_char_boundary(cut) { - cut -= 1; - } - if cut == 0 { - // Degenerate case — shouldn't happen for normal - // text. Take the entire remaining line to avoid - // an infinite loop. - chunks.push(remaining.to_string()); - break; - } - chunks.push(remaining[..cut].to_string()); - remaining = &remaining[cut..]; - } - } else { - current.push_str(line); - current.push('\n'); - } - } - flush(&mut current, &mut chunks); - - chunks -} - -/// Tight top-K ceiling for toolkits whose per-action JSON schemas are -/// dense enough to blow through either Fireworks' 65 535-rule grammar -/// cap (native mode) or the 196 607-token context cap (text mode) even -/// before any tool results land in history. Determined empirically from -/// the fixture dumps under `tests/fixtures/composio_*.json` and real -/// staging failures — see the trace where Gmail at top-K=25 produced -/// a 276k-token iter-1 prompt. -const HEAVY_SCHEMA_TOOLKITS: &[&str] = &[ - "gmail", - "notion", - "github", - "salesforce", - "hubspot", - "googledrive", - "googlesheets", - "googledocs", - "microsoftteams", -]; - -const TOOL_FILTER_TOP_K_DEFAULT: usize = 25; -const TOOL_FILTER_TOP_K_HEAVY: usize = 12; - -/// Pick a top-K budget for the fuzzy filter based on how dense the -/// toolkit's action schemas tend to be. Match is case-insensitive so -/// we don't care whether the caller passed `"Gmail"` or `"gmail"`. -fn top_k_for_toolkit(toolkit: &str) -> usize { - if HEAVY_SCHEMA_TOOLKITS - .iter() - .any(|t| t.eq_ignore_ascii_case(toolkit)) - { - TOOL_FILTER_TOP_K_HEAVY - } else { - TOOL_FILTER_TOP_K_DEFAULT - } -} - -/// Format a set of `ToolSpec`s as an XML tool-use protocol block -/// appended to the system prompt in text mode. Mirrors -/// [`crate::openhuman::agent::dispatcher::XmlToolDispatcher::prompt_instructions`] -/// — same `{…}` format so the existing -/// `parse_tool_calls` helper understands what the model emits. -/// -/// Per-parameter rendering is intentionally **compact**: name, type, a -/// "required" marker, and a short one-line description if present. We -/// do **not** serialise the full JSON schema. Composio/Fireworks action -/// schemas for toolkits like Gmail or Notion run multiple KB each — -/// embedding them verbatim blows up the prompt past the model's -/// context window (282k+ tokens for 26 Gmail tools vs a 196k cap). -/// The compact listing keeps the model informed enough to call tools -/// correctly while staying within budget. If the model needs deeper -/// schema detail it can surface the error and the orchestrator will -/// clarify on the next turn. -pub(crate) fn build_text_mode_tool_instructions(_specs: &[ToolSpec]) -> String { - // The tool catalog is already rendered in the prompt's `## Tools` - // section (see `prompts::ToolsSection::build`) with full - // `Call as: NAME[arg|arg]` signatures. We previously also emitted - // an `### Available Tools` subsection here with a different - // formatting (`Parameters: name:type, ...`), which doubled the - // tool list bytes for text-mode agents — especially expensive for - // the integrations_agent toolkit-scoped spawns (~50 actions × - // 2 listings). Keep only the protocol explanation; the tool - // catalog itself comes from the prompt template. - let mut out = String::new(); - out.push_str("## Tool Use Protocol\n\n"); - out.push_str( - "To use a tool, wrap a JSON object in tags. \ - Do not nest tags. Emit one tag per call; you can emit multiple tags \ - in the same response if you need to run calls in parallel.\n\n", - ); - out.push_str( - "```\n\n{\"name\": \"tool_name\", \"arguments\": {\"param\": \"value\"}}\n\n```\n", - ); - out -} - -// ───────────────────────────────────────────────────────────────────────────── -// Tool filtering -// ───────────────────────────────────────────────────────────────────────────── - -/// Returns indices into `parent_tools` for the tools the sub-agent may -/// invoke. Index-based filtering avoids cloning `Box` (which -/// isn't Clone) and lets us reuse the parent's existing instances. -/// -/// Filters are applied in this order (shorter-circuit first): -/// 1. `disallowed` — explicit deny list. -/// 2. `skill_filter` — restrict to tools named `{skill}__*`. -/// 3. `scope` — `Wildcard` (everything remaining) or `Named` allowlist. -/// -/// Exposed `pub(crate)` so the debug dump path in -/// [`crate::openhuman::context::debug_dump`] shares the exact same -/// filter logic as the live runner — previously debug_dump carried a -/// "standalone copy" which drifted over time. -/// Tools that must never be visible to any agent except `welcome`. -/// -/// `complete_onboarding` flips the onboarding-complete flag in -/// workspace config and is the terminal step of the welcome flow; -/// every other agent must route the user back to the welcome agent -/// rather than call it directly. Central list here so both the main -/// agent builder ([`crate::openhuman::agent::harness::session::builder`]) -/// and the subagent runner apply the same guard. -pub(crate) fn is_welcome_only_tool(name: &str) -> bool { - matches!(name, "complete_onboarding") -} - -pub(crate) fn filter_tool_indices( - parent_tools: &[Box], - scope: &ToolScope, - disallowed: &[String], - skill_filter: Option<&str>, -) -> Vec { - let disallow_set: HashSet<&str> = disallowed.iter().map(|s| s.as_str()).collect(); - let skill_prefix = skill_filter.map(|s| format!("{s}__")); - - parent_tools - .iter() - .enumerate() - .filter(|(_, tool)| { - let name = tool.name(); - if disallow_set.contains(name) { - return false; - } - if let Some(prefix) = skill_prefix.as_deref() { - if !name.starts_with(prefix) { - return false; - } - } - match scope { - ToolScope::Wildcard => true, - ToolScope::Named(allowed) => allowed.iter().any(|n| n == name), - } - }) - .map(|(i, _)| i) - .collect() -} - -// ───────────────────────────────────────────────────────────────────────────── -// Prompt loading + composition -// ───────────────────────────────────────────────────────────────────────────── - -/// Resolve a [`PromptSource`] to its raw markdown body. Inline sources -/// return immediately, `Dynamic` calls the builder with the supplied -/// [`PromptContext`], `File` sources are read from disk relative to the -/// workspace `prompts/` directory or the agent crate's bundled prompts. -/// -/// Exposed `pub(crate)` so the debug dump path in -/// [`crate::openhuman::context::debug_dump`] loads prompts through the -/// exact same code the runner uses — no parallel body-loading logic. -pub(crate) fn load_prompt_source( - source: &PromptSource, - ctx: &PromptContext<'_>, -) -> Result { - let workspace_dir = ctx.workspace_dir; - match source { - PromptSource::Inline(body) => Ok(body.clone()), - PromptSource::Dynamic(build) => build(ctx).map_err(|e| SubagentRunError::PromptLoad { - path: format!("", ctx.agent_id), - source: std::io::Error::new(std::io::ErrorKind::Other, e.to_string()), - }), - PromptSource::File { path } => { - // Try the workspace's `agent/prompts/` first (so users can - // override built-in prompts), then fall back to the crate's - // own bundled prompts via `include_str!`-style lookup. - let workspace_path = workspace_dir.join("agent").join("prompts").join(path); - if workspace_path.is_file() { - return std::fs::read_to_string(&workspace_path).map_err(|e| { - SubagentRunError::PromptLoad { - path: workspace_path.display().to_string(), - source: e, - } - }); - } - // Built-in prompt fallback. The agent prompts directory is - // already shipped at `src/openhuman/agent/prompts/` and - // included in the binary via the `IdentitySection` workspace - // file write — so we re-use that scaffolding by reading from - // `/` after the parent agent has - // bootstrapped its workspace files. For sub-agent - // archetype prompts (e.g. `archetypes/researcher.md`), - // we look up by basename in the workspace, then accept - // missing files as an empty body (the runner will fall - // back to a generic role hint). - let workspace_root_path = workspace_dir.join(path); - if workspace_root_path.is_file() { - return std::fs::read_to_string(&workspace_root_path).map_err(|e| { - SubagentRunError::PromptLoad { - path: workspace_root_path.display().to_string(), - source: e, - } - }); - } - tracing::warn!( - path = %path, - "[subagent_runner] archetype prompt file not found, using empty body" - ); - Ok(String::new()) - } - } -} - -// Note: the narrow sub-agent prompt renderer lives in -// `crate::openhuman::context::prompt::render_subagent_system_prompt` -// so every system-prompt-building call-site — main agents, sub-agents, -// channel runtimes — shares one module. - -// ───────────────────────────────────────────────────────────────────────────── -// Tests -// ───────────────────────────────────────────────────────────────────────────── - #[cfg(test)] mod tests { - use super::super::definition::ModelSpec; use super::*; + use crate::openhuman::agent::harness::definition::{ModelSpec, ToolScope}; fn make_def_named_tools(names: &[&str]) -> AgentDefinition { AgentDefinition { @@ -1911,12 +1164,12 @@ mod tests { extra_tools: vec![], max_iterations: 5, timeout_secs: None, - sandbox_mode: super::super::definition::SandboxMode::None, + sandbox_mode: crate::openhuman::agent::harness::definition::SandboxMode::None, background: false, uses_fork_context: false, subagents: vec![], delegate_name: None, - source: super::super::definition::DefinitionSource::Builtin, + source: crate::openhuman::agent::harness::definition::DefinitionSource::Builtin, } } @@ -2009,7 +1262,7 @@ mod tests { // ── End-to-end runner tests with mock provider ──────────────────────── - use super::super::fork_context::{with_fork_context, with_parent_context}; + use crate::openhuman::agent::harness::fork_context::{with_fork_context, with_parent_context}; use crate::openhuman::providers::{ ChatRequest as PChatRequest, ChatResponse, Provider, ToolCall, }; @@ -2411,7 +1664,7 @@ mod tests { fork_task_prompt: "ANALYSE THIS BRANCH".into(), }; - let def = super::super::builtin_definitions::fork_definition(); + let def = crate::openhuman::agent::harness::builtin_definitions::fork_definition(); let outcome = with_parent_context(parent, async move { with_fork_context(fork, async { @@ -2450,7 +1703,7 @@ mod tests { async fn fork_mode_errors_when_no_fork_context() { let provider = ScriptedProvider::new(vec![text_response("unused")]); let parent = make_parent(provider, vec![stub("file_read")]); - let def = super::super::builtin_definitions::fork_definition(); + let def = crate::openhuman::agent::harness::builtin_definitions::fork_definition(); let result = with_parent_context(parent, async { run_subagent(&def, "x", SubagentRunOptions::default()).await diff --git a/src/openhuman/agent/harness/subagent_runner/tool_prep.rs b/src/openhuman/agent/harness/subagent_runner/tool_prep.rs new file mode 100644 index 000000000..0fe336a73 --- /dev/null +++ b/src/openhuman/agent/harness/subagent_runner/tool_prep.rs @@ -0,0 +1,234 @@ +//! Helpers that prepare the sub-agent's tool surface and system prompt +//! body before [`super::run_typed_mode`] spins up its tool-loop. +//! +//! Kept together because they share a theme (what does the sub-agent +//! actually see?) and because several of them are exposed `pub(crate)` +//! so the debug-dump path in +//! [`crate::openhuman::context::debug_dump`] can mirror the live runner +//! byte-for-byte instead of carrying its own drifting copies. + +use std::collections::HashSet; + +use super::super::definition::{PromptSource, ToolScope}; +use super::types::SubagentRunError; +use crate::openhuman::context::prompt::PromptContext; +use crate::openhuman::tools::{Tool, ToolSpec}; + +// ── Heavy-schema toolkit accounting ───────────────────────────────────── + +/// Tight top-K ceiling for toolkits whose per-action JSON schemas are +/// dense enough to blow through either Fireworks' 65 535-rule grammar +/// cap (native mode) or the 196 607-token context cap (text mode) even +/// before any tool results land in history. Determined empirically from +/// the fixture dumps under `tests/fixtures/composio_*.json` and real +/// staging failures — see the trace where Gmail at top-K=25 produced +/// a 276k-token iter-1 prompt. +const HEAVY_SCHEMA_TOOLKITS: &[&str] = &[ + "gmail", + "notion", + "github", + "salesforce", + "hubspot", + "googledrive", + "googlesheets", + "googledocs", + "microsoftteams", +]; + +const TOOL_FILTER_TOP_K_DEFAULT: usize = 25; +const TOOL_FILTER_TOP_K_HEAVY: usize = 12; + +/// Pick a top-K budget for the fuzzy filter based on how dense the +/// toolkit's action schemas tend to be. Match is case-insensitive so +/// we don't care whether the caller passed `"Gmail"` or `"gmail"`. +pub(super) fn top_k_for_toolkit(toolkit: &str) -> usize { + if HEAVY_SCHEMA_TOOLKITS + .iter() + .any(|t| t.eq_ignore_ascii_case(toolkit)) + { + TOOL_FILTER_TOP_K_HEAVY + } else { + TOOL_FILTER_TOP_K_DEFAULT + } +} + +// ── Text-mode protocol block ──────────────────────────────────────────── + +/// Format a set of `ToolSpec`s as an XML tool-use protocol block +/// appended to the system prompt in text mode. Mirrors +/// [`crate::openhuman::agent::dispatcher::XmlToolDispatcher::prompt_instructions`] +/// — same `{…}` format so the existing +/// `parse_tool_calls` helper understands what the model emits. +/// +/// Per-parameter rendering is intentionally **compact**: name, type, a +/// "required" marker, and a short one-line description if present. We +/// do **not** serialise the full JSON schema. Composio/Fireworks action +/// schemas for toolkits like Gmail or Notion run multiple KB each — +/// embedding them verbatim blows up the prompt past the model's +/// context window (282k+ tokens for 26 Gmail tools vs a 196k cap). +/// The compact listing keeps the model informed enough to call tools +/// correctly while staying within budget. If the model needs deeper +/// schema detail it can surface the error and the orchestrator will +/// clarify on the next turn. +pub(crate) fn build_text_mode_tool_instructions(_specs: &[ToolSpec]) -> String { + // The tool catalog is already rendered in the prompt's `## Tools` + // section (see `prompts::ToolsSection::build`) with full + // `Call as: NAME[arg|arg]` signatures. We previously also emitted + // an `### Available Tools` subsection here with a different + // formatting (`Parameters: name:type, ...`), which doubled the + // tool list bytes for text-mode agents — especially expensive for + // the integrations_agent toolkit-scoped spawns (~50 actions × + // 2 listings). Keep only the protocol explanation; the tool + // catalog itself comes from the prompt template. + let mut out = String::new(); + out.push_str("## Tool Use Protocol\n\n"); + out.push_str( + "To use a tool, wrap a JSON object in tags. \ + Do not nest tags. Emit one tag per call; you can emit multiple tags \ + in the same response if you need to run calls in parallel.\n\n", + ); + out.push_str( + "```\n\n{\"name\": \"tool_name\", \"arguments\": {\"param\": \"value\"}}\n\n```\n", + ); + out +} + +// ── Tool filtering ────────────────────────────────────────────────────── + +/// Tools that must never be visible to any agent except `welcome`. +/// +/// `complete_onboarding` flips the onboarding-complete flag in +/// workspace config and is the terminal step of the welcome flow; +/// every other agent must route the user back to the welcome agent +/// rather than call it directly. Central list here so both the main +/// agent builder ([`crate::openhuman::agent::harness::session::builder`]) +/// and the subagent runner apply the same guard. +pub(crate) fn is_welcome_only_tool(name: &str) -> bool { + matches!(name, "complete_onboarding") +} + +/// Tools that spawn a new sub-agent turn. A sub-agent must never be +/// able to invoke any of these — only the top-level orchestrator +/// delegates. Nested spawns would create a recursion tree the harness +/// is not designed to budget, cost, or observe. +/// +/// Matches: +/// * the generic `spawn_subagent` meta-tool (arbitrary archetype by id); +/// * every synthesised per-archetype `delegate_*` tool +/// ([`crate::openhuman::tools::orchestrator_tools::collect_orchestrator_tools`] +/// emits `delegate_researcher`, `delegate_planner`, …). +/// +/// Kept as a tight prefix/exact match rather than a registry lookup so +/// the strip is cheap to run inside [`super::ops::run_typed_mode`]'s +/// filter pass. If the delegation-tool naming scheme changes, update +/// this function and the corresponding generator in +/// `orchestrator_tools.rs` together. +pub(super) fn is_subagent_spawn_tool(name: &str) -> bool { + name == "spawn_subagent" || name.starts_with("delegate_") +} + +/// Returns indices into `parent_tools` for the tools the sub-agent may +/// invoke. Index-based filtering avoids cloning `Box` (which +/// isn't Clone) and lets us reuse the parent's existing instances. +/// +/// Filters are applied in this order (shorter-circuit first): +/// 1. `disallowed` — explicit deny list. +/// 2. `skill_filter` — restrict to tools named `{skill}__*`. +/// 3. `scope` — `Wildcard` (everything remaining) or `Named` allowlist. +/// +/// Exposed `pub(crate)` so the debug dump path in +/// [`crate::openhuman::context::debug_dump`] shares the exact same +/// filter logic as the live runner — previously debug_dump carried a +/// "standalone copy" which drifted over time. +pub(crate) fn filter_tool_indices( + parent_tools: &[Box], + scope: &ToolScope, + disallowed: &[String], + skill_filter: Option<&str>, +) -> Vec { + let disallow_set: HashSet<&str> = disallowed.iter().map(|s| s.as_str()).collect(); + let skill_prefix = skill_filter.map(|s| format!("{s}__")); + + parent_tools + .iter() + .enumerate() + .filter(|(_, tool)| { + let name = tool.name(); + if disallow_set.contains(name) { + return false; + } + if let Some(prefix) = skill_prefix.as_deref() { + if !name.starts_with(prefix) { + return false; + } + } + match scope { + ToolScope::Wildcard => true, + ToolScope::Named(allowed) => allowed.iter().any(|n| n == name), + } + }) + .map(|(i, _)| i) + .collect() +} + +// ── Prompt loading ────────────────────────────────────────────────────── + +/// Resolve a [`PromptSource`] to its raw markdown body. Inline sources +/// return immediately, `Dynamic` calls the builder with the supplied +/// [`PromptContext`], `File` sources are read from disk relative to the +/// workspace `prompts/` directory or the agent crate's bundled prompts. +/// +/// Exposed `pub(crate)` so the debug dump path in +/// [`crate::openhuman::context::debug_dump`] loads prompts through the +/// exact same code the runner uses — no parallel body-loading logic. +pub(crate) fn load_prompt_source( + source: &PromptSource, + ctx: &PromptContext<'_>, +) -> Result { + let workspace_dir = ctx.workspace_dir; + match source { + PromptSource::Inline(body) => Ok(body.clone()), + PromptSource::Dynamic(build) => build(ctx).map_err(|e| SubagentRunError::PromptLoad { + path: format!("", ctx.agent_id), + source: std::io::Error::other(e.to_string()), + }), + PromptSource::File { path } => { + // Try the workspace's `agent/prompts/` first (so users can + // override built-in prompts), then fall back to the crate's + // own bundled prompts via `include_str!`-style lookup. + let workspace_path = workspace_dir.join("agent").join("prompts").join(path); + if workspace_path.is_file() { + return std::fs::read_to_string(&workspace_path).map_err(|e| { + SubagentRunError::PromptLoad { + path: workspace_path.display().to_string(), + source: e, + } + }); + } + // Built-in prompt fallback. The agent prompts directory is + // already shipped at `src/openhuman/agent/prompts/` and + // included in the binary via the `IdentitySection` workspace + // file write — so we re-use that scaffolding by reading from + // `/` after the parent agent has + // bootstrapped its workspace files. For sub-agent + // archetype prompts (e.g. `archetypes/researcher.md`), + // we look up by basename in the workspace, then accept + // missing files as an empty body (the runner will fall + // back to a generic role hint). + let workspace_root_path = workspace_dir.join(path); + if workspace_root_path.is_file() { + return std::fs::read_to_string(&workspace_root_path).map_err(|e| { + SubagentRunError::PromptLoad { + path: workspace_root_path.display().to_string(), + source: e, + } + }); + } + tracing::warn!( + path = %path, + "[subagent_runner] archetype prompt file not found, using empty body" + ); + Ok(String::new()) + } + } +} diff --git a/src/openhuman/agent/harness/subagent_runner/types.rs b/src/openhuman/agent/harness/subagent_runner/types.rs new file mode 100644 index 000000000..e33d8366e --- /dev/null +++ b/src/openhuman/agent/harness/subagent_runner/types.rs @@ -0,0 +1,99 @@ +//! Public types for the sub-agent runner: spawn options, outcome, +//! execution mode, and error taxonomy. Pulled out of `ops.rs` so +//! external callers importing these shapes don't drag in the full +//! orchestration machinery. + +use std::time::Duration; +use thiserror::Error; + +/// Per-spawn options that override or augment what the +/// [`AgentDefinition`] specifies. Built by `SpawnSubagentTool::execute` +/// from the parent model's call arguments. +#[derive(Debug, Clone, Default)] +pub struct SubagentRunOptions { + /// Optional skill-id override (e.g. `"notion"`). When set, the + /// resolved tool list is further restricted to tools whose name + /// starts with `{skill}__`. Overrides `definition.skill_filter`. + pub skill_filter_override: Option, + + /// Optional Composio toolkit scope (e.g. `"gmail"`, `"notion"`). + /// When set, skill-category tools are further restricted to those + /// whose name starts with the uppercased `{toolkit}_` prefix, and + /// the sub-agent's rendered `Connected Integrations` section is + /// narrowed to only that toolkit's entry. Used by main/orchestrator + /// when spawning `integrations_agent` for a specific platform so the + /// sub-agent only sees one integration's tool catalogue. + pub toolkit_override: Option, + + /// Optional context blob the parent wants to inject before the + /// task prompt. Rendered as a `[Context]\n…\n` prefix. + pub context: Option, + + /// Stable id for tracing / DomainEvents (defaults to a UUID). + pub task_id: Option, +} + +/// Outcome of a single sub-agent run, returned to the parent. +#[derive(Debug, Clone)] +pub struct SubagentRunOutcome { + /// Unique identifier for this sub-task run. + pub task_id: String, + /// The ID of the agent archetype used (e.g., `researcher`). + pub agent_id: String, + /// The final text response produced by the sub-agent. + pub output: String, + /// How many LLM round-trips were performed during the run. + pub iterations: usize, + /// Total wall-clock duration of the run. + pub elapsed: Duration, + /// Which execution mode was used (Typed vs. Fork). + pub mode: SubagentMode, +} + +/// Which prompt-construction path the runner took for a sub-agent. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum SubagentMode { + /// Built a narrow, archetype-specific prompt with filtered tools. + Typed, + /// Replayed the parent's exact rendered prompt and history prefix. + Fork, +} + +impl SubagentMode { + pub fn as_str(self) -> &'static str { + match self { + Self::Typed => "typed", + Self::Fork => "fork", + } + } +} + +/// Errors the runner can surface to the parent. The parent receives a +/// stringified version inside a tool result block. +#[derive(Debug, Error)] +pub enum SubagentRunError { + #[error("spawn_subagent called outside of an agent turn — no parent context available")] + NoParentContext, + + #[error( + "fork-mode sub-agent requested but no ForkContext is set on the task-local. \ + Did the parent agent forget to call `Agent::turn` with fork support?" + )] + NoForkContext, + + #[error("agent definition '{0}' not found in registry")] + DefinitionNotFound(String), + + #[error("failed to load archetype prompt from '{path}': {source}")] + PromptLoad { + path: String, + #[source] + source: std::io::Error, + }, + + #[error("provider call failed: {0}")] + Provider(#[from] anyhow::Error), + + #[error("sub-agent exceeded maximum iterations ({0})")] + MaxIterationsExceeded(usize), +} diff --git a/src/openhuman/composio/ops.rs b/src/openhuman/composio/ops.rs index c573bf278..48a8a1c94 100644 --- a/src/openhuman/composio/ops.rs +++ b/src/openhuman/composio/ops.rs @@ -167,7 +167,7 @@ pub async fn composio_execute( let elapsed_ms = started.elapsed().as_millis() as u64; match result { - Ok(resp) => { + Ok(mut resp) => { crate::core::event_bus::publish_global( crate::core::event_bus::DomainEvent::ComposioActionExecuted { tool: tool.to_string(), @@ -177,6 +177,34 @@ pub async fn composio_execute( elapsed_ms, }, ); + // Mirror the agent-tool path (see `tools::ComposioExecuteTool::execute`): + // route through the toolkit's native provider so CLI and JSON-RPC + // callers see the same envelope the agent sees (e.g. Gmail HTML → + // markdown). `raw_html: true` in `arguments` opts out for + // `GMAIL_FETCH_EMAILS`. + // + // Provider registry is populated by `bus::start_composio_bus` on + // the server path; the CLI/RPC one-shot path never boots the bus, + // so ensure the built-ins are registered before we look up. The + // init fn is idempotent. + if resp.successful { + super::providers::init_default_providers(); + if let Some(toolkit) = super::providers::toolkit_from_slug(tool) { + if let Some(provider) = super::providers::get_provider(&toolkit) { + tracing::trace!( + toolkit = toolkit.as_str(), + tool = tool, + has_args = arguments.is_some(), + "[composio] post-processing action result" + ); + provider.post_process_action_result( + tool, + arguments.as_ref(), + &mut resp.data, + ); + } + } + } Ok(RpcOutcome::new( resp, vec![format!("composio: executed {tool} ({elapsed_ms}ms)")], diff --git a/src/openhuman/composio/providers/gmail/mod.rs b/src/openhuman/composio/providers/gmail/mod.rs index 1fc96a33f..bdc05ebe2 100644 --- a/src/openhuman/composio/providers/gmail/mod.rs +++ b/src/openhuman/composio/providers/gmail/mod.rs @@ -1,3 +1,4 @@ +mod post_process; mod provider; mod sync; #[cfg(test)] diff --git a/src/openhuman/composio/providers/gmail/post_process.rs b/src/openhuman/composio/providers/gmail/post_process.rs new file mode 100644 index 000000000..2fbe5cad2 --- /dev/null +++ b/src/openhuman/composio/providers/gmail/post_process.rs @@ -0,0 +1,1045 @@ +//! Gmail-specific post-processing of Composio action responses. +//! +//! The upstream `GMAIL_FETCH_EMAILS` payload is extremely verbose: +//! +//! * the full MIME tree under `payload.parts[]`, with base64url-encoded +//! bodies — HTML parts alone are routinely 30–100 KB per message; +//! * duplicate text in `preview.{body,subject}` and `snippet`; +//! * internal header arrays (50+ `Received:` / DKIM lines) that carry +//! no semantic value for the agent; +//! * display-layer fields (`display_url`, `internalDate`, part `mimeType` / +//! `partId` / `filename`) the model never uses. +//! +//! Feeding all of that back to the LLM burns context on presentational +//! markup. By default this module rewrites the payload into a slim +//! envelope per message: +//! +//! ```json +//! { +//! "messages": [ +//! { +//! "id": "…", +//! "threadId": "…", +//! "subject": "…", +//! "from": "…", +//! "to": "…", +//! "date": "…", +//! "labels": ["INBOX", "UNREAD"], +//! "markdown": "…converted body…", +//! "attachments": [ { "filename": "...", "mimeType": "..." } ] +//! } +//! ], +//! "nextPageToken": "…", +//! "resultSizeEstimate": 201 +//! } +//! ``` +//! +//! Callers that need the raw Composio shape can pass `raw_html: true` +//! (or `rawHtml: true`) in the action arguments — this short-circuits +//! the transform and returns the upstream payload untouched. +//! +//! Only `GMAIL_FETCH_EMAILS` is reshaped today; other Gmail action +//! responses are passed through unchanged. When we add envelopes for +//! more slugs they should live in this file, branched from +//! [`post_process`]. + +use base64::engine::general_purpose::URL_SAFE_NO_PAD; +use base64::Engine; +use once_cell::sync::Lazy; +use regex::Regex; +use serde_json::{json, Map, Value}; + +/// `html2md` is fine for normal transactional emails, but large marketing +/// HTML can explode CPU / latency. Above this size we switch to a bounded +/// fast-strip path that preserves readable text and link labels. +const MAX_HTML2MD_INPUT_BYTES: usize = 24_000; + +static HTML_NOISE_BLOCK_RE: Lazy = + Lazy::new(|| Regex::new(r"(?is)").expect("valid html comment regex")); + +/// Entry point called from `GmailProvider::post_process_action_result`. +/// +/// Dispatches on the Composio action slug. Unknown Gmail slugs fall +/// through to a no-op. +pub fn post_process(slug: &str, arguments: Option<&Value>, data: &mut Value) { + if is_raw_html_flag_set(arguments) { + tracing::debug!( + slug, + "[composio:gmail][post-process] raw_html flag set, passing through" + ); + return; + } + if slug == "GMAIL_FETCH_EMAILS" { + reshape_fetch_emails(data) + } +} + +/// Returns true when the caller explicitly set `raw_html: true` (or the +/// camelCase `rawHtml: true`) in the `arguments` object. +fn is_raw_html_flag_set(arguments: Option<&Value>) -> bool { + let Some(obj) = arguments.and_then(|v| v.as_object()) else { + return false; + }; + obj.get("raw_html") + .or_else(|| obj.get("rawHtml")) + .and_then(|v| v.as_bool()) + .unwrap_or(false) +} + +/// Rewrite a `GMAIL_FETCH_EMAILS` `data` object in place into the slim +/// envelope documented at the module level. +/// +/// The Composio response can be shaped either as `{ messages, nextPageToken, ... }` +/// directly, or wrapped one level deeper under `{ data: { messages: … } }` +/// depending on backend version; we handle both. +fn reshape_fetch_emails(data: &mut Value) { + // Unwrap an optional `data:` envelope so downstream logic only has + // to deal with one shape. + let container = match data.get_mut("messages") { + Some(_) => data, + None => match data.get_mut("data").and_then(|v| v.as_object_mut()) { + Some(_) => data.get_mut("data").unwrap(), + None => return, + }, + }; + + let Some(obj) = container.as_object_mut() else { + return; + }; + + let raw_messages = obj + .remove("messages") + .and_then(|v| match v { + Value::Array(arr) => Some(arr), + _ => None, + }) + .unwrap_or_default(); + let next_page_token = obj.remove("nextPageToken").unwrap_or(Value::Null); + let result_size_estimate = obj.remove("resultSizeEstimate").unwrap_or(Value::Null); + + let messages: Vec = raw_messages.into_iter().map(reshape_message).collect(); + + let mut envelope = Map::new(); + envelope.insert("messages".into(), Value::Array(messages)); + if !next_page_token.is_null() { + envelope.insert("nextPageToken".into(), next_page_token); + } + if !result_size_estimate.is_null() { + envelope.insert("resultSizeEstimate".into(), result_size_estimate); + } + + *container = Value::Object(envelope); +} + +/// Map one raw Composio message object to its slim counterpart. +/// +/// Preference order for the body: +/// 1. A `text/html` MIME part's base64url-decoded body → html2md. +/// 2. A `text/plain` MIME part's base64url-decoded body. +/// 3. The top-level `messageText` (Composio's decoded plain text). +/// 4. Empty string. +fn reshape_message(raw: Value) -> Value { + let Value::Object(obj) = raw else { + return raw; + }; + + let id = obj.get("messageId").cloned().unwrap_or(Value::Null); + let thread_id = obj.get("threadId").cloned().unwrap_or(Value::Null); + let subject = obj.get("subject").cloned().unwrap_or(Value::Null); + let sender = obj.get("sender").cloned().unwrap_or(Value::Null); + let to = obj.get("to").cloned().unwrap_or(Value::Null); + let date = obj + .get("messageTimestamp") + .cloned() + .or_else(|| pick_header(&obj, "Date")) + .unwrap_or(Value::Null); + let labels = obj + .get("labelIds") + .cloned() + .unwrap_or_else(|| Value::Array(Vec::new())); + + let markdown = extract_markdown_body(&obj); + let attachments = extract_attachments(&obj); + + let mut out = Map::new(); + out.insert("id".into(), id); + out.insert("threadId".into(), thread_id); + out.insert("subject".into(), subject); + out.insert("from".into(), sender); + out.insert("to".into(), to); + out.insert("date".into(), date); + out.insert("labels".into(), labels); + out.insert("markdown".into(), Value::String(markdown)); + if !attachments.is_empty() { + out.insert("attachments".into(), Value::Array(attachments)); + } + Value::Object(out) +} + +/// Find a header value by (case-insensitive) name in the Composio +/// `payload.headers[]` array. Returns `Some(Value::String)` on hit. +fn pick_header(msg: &Map, name: &str) -> Option { + let headers = msg.get("payload")?.get("headers")?.as_array()?; + for h in headers { + let hn = h.get("name").and_then(|v| v.as_str()).unwrap_or(""); + if hn.eq_ignore_ascii_case(name) { + if let Some(v) = h.get("value").and_then(|v| v.as_str()) { + return Some(Value::String(v.to_string())); + } + } + } + None +} + +/// Extract the best body representation and return it as markdown. +/// Walks `payload.parts[]` recursively — Gmail nests multipart/alternative +/// inside multipart/mixed when attachments are present. +fn extract_markdown_body(msg: &Map) -> String { + if let Some(parts) = msg.get("payload").and_then(|p| p.get("parts")) { + if let Some(html) = find_decoded_part(parts, "text/html") { + return html_email_to_markdown(&html); + } + if let Some(text) = find_decoded_part(parts, "text/plain") { + return normalize_markdownish_text(&text); + } + } + // Fallback: top-level decoded plain text (Composio convenience field). + if let Some(text) = msg.get("messageText").and_then(|v| v.as_str()) { + if looks_like_raw_html(text) { + tracing::debug!( + text_bytes = text.len(), + "[composio:gmail][post-process] messageText looked like html, using fast html strip" + ); + return fast_html_email_to_markdown(text); + } + return normalize_markdownish_text(text); + } + String::new() +} + +/// Convert raw HTML email into markdown-ish text that is safe and cheap for +/// LLM consumption. Small / normal HTML uses `html2md`; oversized HTML falls +/// back to a linear-time stripper so one pathological newsletter cannot stall +/// the whole tool call. +fn html_email_to_markdown(html: &str) -> String { + let cleaned = strip_html_noise_blocks(html); + let cleaned = cleaned.trim(); + if cleaned.is_empty() { + return String::new(); + } + + if cleaned.len() > MAX_HTML2MD_INPUT_BYTES { + tracing::debug!( + html_bytes = cleaned.len(), + threshold = MAX_HTML2MD_INPUT_BYTES, + "[composio:gmail][post-process] large html body, using fast strip fallback" + ); + return normalize_markdownish_text(&fast_html_to_text(cleaned)); + } + + let md = html2md::parse_html(cleaned); + let normalized = normalize_markdownish_text(&md); + if normalized.is_empty() + || looks_like_raw_html(&normalized) + || suspiciously_short_markdown(cleaned, &normalized) + { + tracing::debug!( + html_bytes = cleaned.len(), + "[composio:gmail][post-process] html2md output still looked like html, using fast strip fallback" + ); + return normalize_markdownish_text(&fast_html_to_text(cleaned)); + } + normalized +} + +fn fast_html_email_to_markdown(html: &str) -> String { + let cleaned = strip_html_noise_blocks(html); + normalize_markdownish_text(&fast_html_to_text(cleaned.trim())) +} + +fn strip_html_noise_blocks(html: &str) -> String { + let mut out = HTML_NOISE_BLOCK_RE.replace_all(html, "").into_owned(); + for tag in ["script", "style", "head", "title", "svg", "noscript"] { + out = strip_tag_block_case_insensitive(&out, tag); + } + out +} + +fn strip_tag_block_case_insensitive(input: &str, tag: &str) -> String { + let lower = input.to_ascii_lowercase(); + let open_pat = format!("<{tag}"); + let close_pat = format!(""); + let mut out = String::with_capacity(input.len()); + let mut cursor = 0usize; + + while let Some(rel_open) = lower[cursor..].find(&open_pat) { + let open = cursor + rel_open; + out.push_str(&input[cursor..open]); + + let Some(open_end_rel) = lower[open..].find('>') else { + cursor = open; + break; + }; + let search_from = open + open_end_rel + 1; + let Some(close_rel) = lower[search_from..].find(&close_pat) else { + cursor = open; + break; + }; + cursor = search_from + close_rel + close_pat.len(); + } + + out.push_str(&input[cursor..]); + out +} + +fn looks_like_raw_html(s: &str) -> bool { + let lower = s.to_ascii_lowercase(); + [ + " bool { + source_html.len() >= 2_000 && markdown.len().saturating_mul(20) < source_html.len() +} + +/// Recursively search a `parts` array for the first MIME part whose +/// `mimeType` starts with `prefix` (e.g. `"text/html"`), and return its +/// base64url-decoded UTF-8 body. +fn find_decoded_part(parts: &Value, prefix: &str) -> Option { + let arr = parts.as_array()?; + for part in arr { + let mime = part + .get("mimeType") + .and_then(|v| v.as_str()) + .unwrap_or_default(); + if mime.starts_with(prefix) { + if let Some(b64) = part.pointer("/body/data").and_then(|v| v.as_str()) { + if let Ok(bytes) = URL_SAFE_NO_PAD.decode(b64) { + if let Ok(s) = String::from_utf8(bytes) { + return Some(s); + } + } + } + } + // Recurse into nested `parts` (multipart/alternative inside multipart/mixed). + if let Some(inner) = part.get("parts") { + if let Some(found) = find_decoded_part(inner, prefix) { + return Some(found); + } + } + } + None +} + +/// Fast, allocation-bounded HTML to text conversion used as a safe fallback +/// when `html2md` would be too expensive on very large message bodies. +fn fast_html_to_text(html: &str) -> String { + let mut out = String::with_capacity(html.len().min(32_768)); + let mut chars = html.chars().peekable(); + + while let Some(ch) = chars.next() { + match ch { + '<' => { + let mut tag = String::new(); + let mut terminated = false; + for next in chars.by_ref() { + if next == '>' { + terminated = true; + break; + } + if tag.len() < 128 { + tag.push(next); + } + } + if !terminated { + break; + } + apply_html_tag_hint(&mut out, &tag); + } + '&' => { + let mut entity = String::new(); + while let Some(&next) = chars.peek() { + if next == ';' { + chars.next(); + break; + } + if next.is_whitespace() || entity.len() >= 16 { + break; + } + entity.push(next); + chars.next(); + } + out.push(decode_html_entity(&entity).unwrap_or('&')); + } + _ => out.push(ch), + } + } + + out +} + +fn apply_html_tag_hint(out: &mut String, raw_tag: &str) { + let mut tag = raw_tag.trim(); + if tag.is_empty() || tag.starts_with('!') || tag.starts_with('?') { + return; + } + if let Some(stripped) = tag.strip_prefix('/') { + tag = stripped.trim_start(); + } + let name = tag + .split_whitespace() + .next() + .unwrap_or_default() + .trim_matches('/') + .to_ascii_lowercase(); + + match name.as_str() { + "br" | "p" | "div" | "section" | "article" | "header" | "footer" | "table" | "tr" + | "blockquote" | "pre" => out.push('\n'), + "li" => { + if !out.ends_with('\n') { + out.push('\n'); + } + out.push_str("- "); + } + "td" | "th" => out.push(' '), + "h1" => out.push_str("\n# "), + "h2" => out.push_str("\n## "), + "h3" => out.push_str("\n### "), + "h4" => out.push_str("\n#### "), + "h5" => out.push_str("\n##### "), + "h6" => out.push_str("\n###### "), + _ => {} + } +} + +fn decode_html_entity(entity: &str) -> Option { + match entity { + "nbsp" => Some(' '), + "amp" => Some('&'), + "lt" => Some('<'), + "gt" => Some('>'), + "quot" => Some('"'), + "apos" | "#39" => Some('\''), + _ => { + if let Some(hex) = entity + .strip_prefix("#x") + .or_else(|| entity.strip_prefix("#X")) + { + u32::from_str_radix(hex, 16).ok().and_then(char::from_u32) + } else if let Some(dec) = entity.strip_prefix('#') { + dec.parse::().ok().and_then(char::from_u32) + } else { + None + } + } + } +} + +/// Pull a minimal attachments descriptor from the Composio `attachmentList` +/// (preferred) or from `payload.parts[]` entries with a non-empty filename. +fn extract_attachments(msg: &Map) -> Vec { + if let Some(list) = msg.get("attachmentList").and_then(|v| v.as_array()) { + return list + .iter() + .filter_map(|a| { + let filename = a.get("filename").and_then(|v| v.as_str())?; + if filename.is_empty() { + return None; + } + let mime = a + .get("mimeType") + .and_then(|v| v.as_str()) + .unwrap_or_default(); + Some(json!({ "filename": filename, "mimeType": mime })) + }) + .collect(); + } + Vec::new() +} + +/// Collapse runs of 3+ blank lines introduced by `html2md` on heavily +/// table-laid-out emails. Keeps single / double newlines intact. +fn strip_excess_blank_lines(s: &str) -> String { + let mut out = String::with_capacity(s.len()); + let mut blank_run = 0usize; + for line in s.lines() { + if line.trim().is_empty() { + blank_run += 1; + if blank_run <= 1 { + out.push('\n'); + } + } else { + blank_run = 0; + out.push_str(line); + out.push('\n'); + } + } + while out.ends_with('\n') { + out.pop(); + } + out +} + +/// Normalize markdown/text emitted by either `html2md` or the fast HTML strip: +/// decode leftover HTML entities, unescape html2md's markdown backslash +/// escapes, trim invisible Unicode, collapse intra-line whitespace, collapse +/// runs of noisy separator tokens (`& & & & &`), and keep only short +/// blank-line runs so the body stays compact for the model. +fn normalize_markdownish_text(s: &str) -> String { + // `html2md` leaves named entities (` `, `‌`, `​`) as + // literals and escapes markdown-significant chars with backslashes + // (`\&`, `\_`, `\.`, `\[`, …). Decode both before any further + // whitespace / entity normalization so downstream passes see plain text. + let decoded = decode_html_entities_inline(s); + let unescaped = unescape_markdown_backslashes(&decoded); + let sanitized = sanitize_llm_text(&unescaped); + let mut normalized = String::with_capacity(sanitized.len()); + + for raw_line in sanitized.lines() { + let mut line = String::with_capacity(raw_line.len()); + let mut prev_space = false; + for ch in raw_line.chars() { + let mapped = match ch { + '\u{00a0}' => ' ', + c if c.is_whitespace() => ' ', + c => c, + }; + if mapped == ' ' { + if !prev_space { + line.push(' '); + } + prev_space = true; + } else { + line.push(mapped); + prev_space = false; + } + } + let collapsed = collapse_separator_runs(line.trim()); + normalized.push_str(&collapsed); + normalized.push('\n'); + } + + strip_excess_blank_lines(normalized.trim()) +} + +/// Decode any HTML entities still present in `s`, using the same table as +/// [`decode_html_entity`] plus numeric `&#nnn;` / `&#xHH;` forms. +/// +/// Unknown entities are left as-is so we never silently swallow characters +/// that were meant to be literal ampersands. +fn decode_html_entities_inline(s: &str) -> String { + let mut out = String::with_capacity(s.len()); + let bytes = s.as_bytes(); + let mut i = 0; + while i < bytes.len() { + if bytes[i] != b'&' { + // Copy through one UTF-8 codepoint. + let ch_len = utf8_char_len(bytes[i]); + out.push_str(&s[i..i + ch_len]); + i += ch_len; + continue; + } + // Try to match an entity beginning at `i`. Entity names are ASCII + // alphanumerics, max 16 chars, terminated by `;`. + let mut j = i + 1; + let limit = (i + 1 + 16).min(bytes.len()); + while j < limit && bytes[j] != b';' { + let b = bytes[j]; + let is_name_char = b.is_ascii_alphanumeric() || b == b'#'; + if !is_name_char { + break; + } + j += 1; + } + if j < bytes.len() && bytes[j] == b';' && j > i + 1 { + let name = &s[i + 1..j]; + if let Some(ch) = decode_html_entity(name) { + out.push(ch); + i = j + 1; + continue; + } + } + // Not a recognised entity — keep the `&` and advance. + out.push('&'); + i += 1; + } + out +} + +/// UTF-8 leading-byte → codepoint length. Always returns 1..=4. +fn utf8_char_len(first: u8) -> usize { + match first { + 0x00..=0x7F => 1, + 0xC0..=0xDF => 2, + 0xE0..=0xEF => 3, + 0xF0..=0xF7 => 4, + _ => 1, + } +} + +/// Undo html2md's markdown backslash escapes for the limited set of chars +/// that routinely appear in email bodies. We only unescape where the backslash +/// is immediately followed by one of the escaped characters — any other +/// backslash usage (actual line-continuation, code fences, etc.) is preserved. +fn unescape_markdown_backslashes(s: &str) -> String { + let mut out = String::with_capacity(s.len()); + let mut chars = s.chars().peekable(); + while let Some(ch) = chars.next() { + if ch == '\\' { + if let Some(&next) = chars.peek() { + if matches!( + next, + '&' | '_' + | '*' + | '.' + | ',' + | '!' + | '(' + | ')' + | '[' + | ']' + | '<' + | '>' + | '#' + | '+' + | '-' + | '@' + | '`' + | '~' + | '=' + | '|' + | '\'' + | '"' + ) { + out.push(next); + chars.next(); + continue; + } + } + } + out.push(ch); + } + out +} + +/// Collapse runs of the same single-char separator surrounded by spaces +/// (e.g. `" & & & & Conditions"` → `" & Conditions"`). Keeps legitimate +/// uses like `"Terms & Conditions"` intact because those aren't runs. +/// Applies to `&`, `-`, `*`, `_`, `|`, `•`, `·`. +fn collapse_separator_runs(line: &str) -> String { + let mut out = String::with_capacity(line.len()); + let mut tokens = line.split(' ').peekable(); + while let Some(tok) = tokens.next() { + out.push_str(tok); + // Look ahead: if `tok` is a single separator char and the next + // token is the *same* separator, drop consecutive duplicates. + if is_collapsible_separator(tok) { + while let Some(&next) = tokens.peek() { + if next == tok { + tokens.next(); + } else { + break; + } + } + } + if tokens.peek().is_some() { + out.push(' '); + } + } + out +} + +fn is_collapsible_separator(tok: &str) -> bool { + matches!(tok, "&" | "-" | "*" | "_" | "|" | "•" | "·") +} + +/// Strip characters that carry little or no semantic value for the model but +/// inflate token count in email bodies: zero-width marks, soft hyphens, BOMs, +/// directional controls, and other control chars except newline / tab. +fn sanitize_llm_text(s: &str) -> String { + let mut out = String::with_capacity(s.len()); + for ch in s.chars() { + match ch { + // Keep structural whitespace; normalize later. + '\n' | '\r' | '\t' => out.push(ch), + // Drop ASCII / Unicode control and formatting noise commonly found + // in HTML emails and copy-pasted content. + '\u{0000}'..='\u{0008}' + | '\u{000b}' + | '\u{000c}' + | '\u{000e}'..='\u{001f}' + | '\u{007f}'..='\u{009f}' + | '\u{00ad}' + | '\u{034f}' + | '\u{061c}' + | '\u{115f}' + | '\u{1160}' + | '\u{17b4}' + | '\u{17b5}' + | '\u{180e}' + | '\u{200b}'..='\u{200f}' + | '\u{202a}'..='\u{202e}' + | '\u{2060}'..='\u{206f}' + | '\u{3164}' + | '\u{fe00}'..='\u{fe0f}' + | '\u{feff}' + | '\u{ffa0}' => {} + _ => out.push(ch), + } + } + out +} + +#[cfg(test)] +mod tests { + use super::*; + use base64::engine::general_purpose::URL_SAFE_NO_PAD; + use base64::Engine; + use serde_json::json; + + fn b64(s: &str) -> String { + URL_SAFE_NO_PAD.encode(s.as_bytes()) + } + + fn fixture() -> Value { + json!({ + "messages": [ + { + "messageId": "m1", + "threadId": "t1", + "subject": "Hello", + "sender": "a@x.com", + "to": "b@y.com", + "messageTimestamp": "2026-04-17T12:00:00Z", + "labelIds": ["INBOX", "UNREAD"], + "messageText": "Hi plain", + "display_url": "ignore-me", + "preview": { "body": "Hi plain", "subject": "Hello" }, + "attachmentList": [ + { "filename": "report.pdf", "mimeType": "application/pdf", "size": 12345 }, + { "filename": "", "mimeType": "text/html" } + ], + "payload": { + "headers": [ { "name": "Date", "value": "Fri, 17 Apr 2026 12:00:00 +0000" } ], + "parts": [ + { + "mimeType": "text/plain", + "body": { "data": b64("Hi plain text") } + }, + { + "mimeType": "text/html", + "body": { "data": b64("

Title

Hello world

") } + } + ] + } + } + ], + "nextPageToken": "tok-1", + "resultSizeEstimate": 42 + }) + } + + #[test] + fn reshape_emits_slim_envelope() { + let mut v = fixture(); + post_process("GMAIL_FETCH_EMAILS", None, &mut v); + + assert_eq!(v["nextPageToken"], "tok-1"); + assert_eq!(v["resultSizeEstimate"], 42); + + let msgs = v["messages"].as_array().unwrap(); + assert_eq!(msgs.len(), 1); + let m = &msgs[0]; + + assert_eq!(m["id"], "m1"); + assert_eq!(m["threadId"], "t1"); + assert_eq!(m["subject"], "Hello"); + assert_eq!(m["from"], "a@x.com"); + assert_eq!(m["to"], "b@y.com"); + assert_eq!(m["date"], "2026-04-17T12:00:00Z"); + assert_eq!(m["labels"], json!(["INBOX", "UNREAD"])); + + let md = m["markdown"].as_str().unwrap(); + assert!( + md.contains("Title"), + "markdown body must carry heading text: {md:?}" + ); + assert!(md.contains("Hello")); + assert!(md.contains("world")); + assert!(!md.contains("

"), "html must be stripped: {md:?}"); + + // Noise fields removed. + assert!(m.get("display_url").is_none()); + assert!(m.get("preview").is_none()); + assert!(m.get("payload").is_none()); + assert!(m.get("messageText").is_none()); + + // Attachments: empty filename entry is filtered. + let atts = m["attachments"].as_array().unwrap(); + assert_eq!(atts.len(), 1); + assert_eq!(atts[0]["filename"], "report.pdf"); + assert_eq!(atts[0]["mimeType"], "application/pdf"); + } + + #[test] + fn raw_html_flag_passes_through_unchanged() { + let mut v = fixture(); + let original = v.clone(); + let args = json!({ "raw_html": true }); + post_process("GMAIL_FETCH_EMAILS", Some(&args), &mut v); + assert_eq!( + v, original, + "raw_html=true must preserve the Composio shape" + ); + } + + #[test] + fn camel_case_raw_html_also_recognized() { + let mut v = fixture(); + let original = v.clone(); + let args = json!({ "rawHtml": true }); + post_process("GMAIL_FETCH_EMAILS", Some(&args), &mut v); + assert_eq!(v, original); + } + + #[test] + fn falls_back_to_message_text_when_no_parts() { + let mut v = json!({ + "messages": [{ + "messageId": "m1", + "threadId": "t1", + "subject": "s", + "sender": "a@x.com", + "to": "b@y.com", + "messageTimestamp": "2026-04-17", + "labelIds": [], + "messageText": " plain body text ", + "payload": {} + }], + "nextPageToken": null + }); + post_process("GMAIL_FETCH_EMAILS", None, &mut v); + let md = v["messages"][0]["markdown"].as_str().unwrap(); + assert_eq!(md, "plain body text"); + assert!(v.get("nextPageToken").is_none(), "null tokens dropped"); + } + + #[test] + fn unwraps_data_envelope() { + let mut v = json!({ + "data": { + "messages": [{ + "messageId": "m1", + "threadId": "t1", + "subject": "s", + "sender": "a@x.com", + "to": "b@y.com", + "messageTimestamp": "2026-04-17", + "labelIds": [], + "messageText": "body", + "payload": {} + }] + } + }); + post_process("GMAIL_FETCH_EMAILS", None, &mut v); + // Reshape writes into `data` in place. + let msgs = v["data"]["messages"].as_array().unwrap(); + assert_eq!(msgs.len(), 1); + assert_eq!(msgs[0]["markdown"], "body"); + } + + #[test] + fn non_fetch_slug_is_noop() { + let mut v = json!({ "messages": [{ "messageId": "m1", "messageText": "x" }] }); + let original = v.clone(); + post_process("GMAIL_SEND_EMAIL", None, &mut v); + assert_eq!(v, original); + } + + #[test] + fn nested_multipart_html_is_found() { + let html = "

Deep body

"; + let mut v = json!({ + "messages": [{ + "messageId": "m1", + "threadId": "t1", + "subject": "s", + "sender": "a@x.com", + "to": "b@y.com", + "messageTimestamp": "2026-04-17", + "labelIds": [], + "messageText": "", + "payload": { + "parts": [ + { + "mimeType": "multipart/alternative", + "parts": [ + { "mimeType": "text/plain", "body": { "data": b64("plain fallback") } }, + { "mimeType": "text/html", "body": { "data": b64(html) } } + ] + } + ] + } + }] + }); + post_process("GMAIL_FETCH_EMAILS", None, &mut v); + let md = v["messages"][0]["markdown"].as_str().unwrap(); + assert!(md.contains("Deep")); + assert!(md.contains("body")); + assert!(!md.contains("

")); + } + + #[test] + fn strip_excess_blank_lines_collapses_runs() { + let input = "a\n\n\n\nb\n\n\nc\n"; + assert_eq!(strip_excess_blank_lines(input), "a\n\nb\n\nc"); + } + + #[test] + fn large_html_uses_fast_strip_fallback() { + let html = format!( + "{}", + (0..3000) + .map(|i| format!( + "

Card {i}

Hello & welcome home

" + )) + .collect::() + ); + let md = html_email_to_markdown(&html); + assert!( + md.contains("## Card 0"), + "heading should survive fallback: {md:?}" + ); + assert!(md.contains("Hello & welcome home")); + assert!(!md.contains("
"), "html tags must be stripped: {md:?}"); + assert!( + !md.contains(".x{color:red}"), + "style blocks must be removed: {md:?}" + ); + } + + #[test] + fn normalize_markdownish_text_removes_invisible_and_extra_spaces() { + let input = " Hello\u{200b} world \n\n line\u{00a0}two "; + assert_eq!(normalize_markdownish_text(input), "Hello world\n\nline two"); + } + + #[test] + fn sanitize_llm_text_strips_weird_token_wasting_chars() { + let input = "A\u{200b}\u{200d}\u{2060}\u{feff}\u{00ad}B\u{202e}C\tD\nE"; + assert_eq!(sanitize_llm_text(input), "ABC\tD\nE"); + } + + #[test] + fn decode_entities_inline_handles_named_and_numeric() { + let s = "Terms & Conditions   and © 2026 with — dash"; + let decoded = decode_html_entities_inline(s); + assert!(decoded.contains("Terms & Conditions")); + assert!(decoded.contains("© 2026")); + assert!(decoded.contains("— dash")); + assert!(!decoded.contains("&")); + assert!(!decoded.contains("©")); + } + + #[test] + fn decode_entities_inline_preserves_unknown() { + let s = "keep ¬arealentity; and & without semi"; + assert_eq!(decode_html_entities_inline(s), s); + } + + #[test] + fn unescape_markdown_backslashes_strips_known_escapes() { + let s = r"a\&b \_ c \. d \\ e \n"; + let out = unescape_markdown_backslashes(s); + // Known escapes drop the backslash; unknown (`\\` before letter n) stays. + assert!(out.contains("a&b")); + assert!(out.contains("_")); + assert!(out.contains(". d")); + assert!(out.contains(r"\\ e")); + assert!(out.contains(r"\n")); + } + + #[test] + fn collapse_separator_runs_squashes_noise() { + assert_eq!(collapse_separator_runs("x & & & & y"), "x & y"); + assert_eq!(collapse_separator_runs("- - - header"), "- header"); + assert_eq!(collapse_separator_runs("a | | | b"), "a | b"); + // Preserves legitimate single-use separators. + assert_eq!( + collapse_separator_runs("Terms & Conditions"), + "Terms & Conditions" + ); + // Multi-char tokens are untouched. + assert_eq!(collapse_separator_runs("a -- b"), "a -- b"); + } + + #[test] + fn normalize_cleans_entity_and_separator_noise() { + let input = "Terms & & & & Conditions \ + with     spaces and\\& an escaped ampersand"; + let out = normalize_markdownish_text(input); + assert!(out.contains("Terms & Conditions"), "got: {out:?}"); + assert!(!out.contains("&")); + assert!(!out.contains(" ")); + assert!(out.contains("an escaped ampersand")); + } + + #[test] + fn detects_raw_html_like_output() { + assert!(looks_like_raw_html("hello")); + assert!(!looks_like_raw_html("# Heading\n\nBody text")); + } + + #[test] + fn html_in_message_text_is_converted() { + let mut v = json!({ + "messages": [{ + "messageId": "m1", + "threadId": "t1", + "subject": "s", + "sender": "a@x.com", + "to": "b@y.com", + "messageTimestamp": "2026-04-17", + "labelIds": [], + "messageText": "

Hello

World

", + "payload": {} + }] + }); + post_process("GMAIL_FETCH_EMAILS", None, &mut v); + let md = v["messages"][0]["markdown"].as_str().unwrap(); + assert!(md.contains("Hello")); + assert!(md.contains("World")); + assert!(!md.contains("")); + } + + #[test] + fn suspiciously_short_markdown_detects_large_collapse() { + assert!(suspiciously_short_markdown(&"x".repeat(4000), "tiny")); + assert!(!suspiciously_short_markdown( + &"x".repeat(4000), + &"y".repeat(400) + )); + } + + #[test] + fn fast_html_strip_handles_long_tags() { + let long_href = format!( + "Click me

After link

", + "x".repeat(600) + ); + let md = fast_html_email_to_markdown(&long_href); + assert!(md.contains("Click me")); + assert!(md.contains("After link")); + } +} diff --git a/src/openhuman/composio/providers/gmail/provider.rs b/src/openhuman/composio/providers/gmail/provider.rs index 10c9aa588..98476dd35 100644 --- a/src/openhuman/composio/providers/gmail/provider.rs +++ b/src/openhuman/composio/providers/gmail/provider.rs @@ -88,6 +88,15 @@ impl ComposioProvider for GmailProvider { Some(15 * 60) } + fn post_process_action_result( + &self, + slug: &str, + arguments: Option<&serde_json::Value>, + data: &mut serde_json::Value, + ) { + super::post_process::post_process(slug, arguments, data); + } + async fn fetch_user_profile( &self, ctx: &ProviderContext, @@ -283,7 +292,7 @@ impl ComposioProvider for GmailProvider { if let Some(date_val) = extract_item_id(msg, MESSAGE_DATE_PATHS) { if newest_date .as_ref() - .map_or(true, |existing| date_val > *existing) + .is_none_or(|existing| date_val > *existing) { newest_date = Some(date_val); } diff --git a/src/openhuman/composio/providers/mod.rs b/src/openhuman/composio/providers/mod.rs index 7460a7e39..72691dddb 100644 --- a/src/openhuman/composio/providers/mod.rs +++ b/src/openhuman/composio/providers/mod.rs @@ -43,7 +43,6 @@ pub mod catalogs; pub mod github; pub mod gmail; pub mod notion; -pub mod post_process; pub mod profile; pub mod registry; pub mod sync_state; diff --git a/src/openhuman/composio/providers/notion/provider.rs b/src/openhuman/composio/providers/notion/provider.rs index a545a49fe..b54b9df4e 100644 --- a/src/openhuman/composio/providers/notion/provider.rs +++ b/src/openhuman/composio/providers/notion/provider.rs @@ -252,7 +252,7 @@ impl ComposioProvider for NotionProvider { if let Some(ref et) = edited_time { if newest_edited_time .as_ref() - .map_or(true, |existing| et > existing) + .is_none_or(|existing| et > existing) { newest_edited_time = Some(et.clone()); } diff --git a/src/openhuman/composio/providers/post_process.rs b/src/openhuman/composio/providers/post_process.rs deleted file mode 100644 index e66cb5d2c..000000000 --- a/src/openhuman/composio/providers/post_process.rs +++ /dev/null @@ -1,188 +0,0 @@ -//! Per-toolkit post-processing of Composio action responses. -//! -//! Some upstream services return content in formats that are noisy for -//! the agent's context window (e.g. Gmail's full HTML message body). -//! This module gives each toolkit a chance to rewrite the response -//! before it is handed back to the LLM — for instance converting HTML -//! email bodies to markdown so the model spends fewer tokens parsing -//! presentational markup. -//! -//! The dispatch is intentionally tiny: one function per toolkit that -//! mutates a `serde_json::Value` in place. The Composio backend keeps -//! evolving its response shapes so we walk values defensively rather -//! than hard-coding field paths. - -use serde_json::Value; - -/// Apply toolkit-specific post-processing to an `composio_execute` -/// response. Mutates `value` in place. -/// -/// Calling this with an unknown toolkit slug is a no-op. -pub fn post_process(toolkit: &str, _slug: &str, value: &mut Value) { - let key = toolkit.trim().to_ascii_lowercase(); - match key.as_str() { - "gmail" => convert_html_strings(value, "gmail"), - _ => {} - } -} - -/// Walk `value` recursively. Any string field whose contents look like -/// HTML is replaced with its markdown rendering. -/// -/// We use a substring heuristic instead of a full parse for speed — -/// if the string contains both `<` and one of a few common email -/// tags it's almost certainly HTML. False positives are harmless -/// (the html2md output for a non-HTML string is essentially the -/// stripped text). -fn convert_html_strings(value: &mut Value, toolkit: &str) { - match value { - Value::String(s) => { - if looks_like_html(s) { - let md = html2md::parse_html(s); - tracing::debug!( - toolkit, - before_bytes = s.len(), - after_bytes = md.len(), - "[composio][post-process] html → markdown" - ); - *s = md; - } - } - Value::Array(items) => { - for item in items { - convert_html_strings(item, toolkit); - } - } - Value::Object(map) => { - for (_, v) in map.iter_mut() { - convert_html_strings(v, toolkit); - } - } - _ => {} - } -} - -/// Heuristic HTML detector. Returns `true` if the string contains an -/// opening `<` followed (anywhere) by one of a handful of common tags -/// found in email bodies. Tuned to avoid matching on stray angle -/// brackets in plain-text quoted replies. -fn looks_like_html(s: &str) -> bool { - if s.len() < 4 || !s.contains('<') { - return false; - } - // Cheap substring scan; case-insensitive via lowercased haystack. - // Bound the work for very large strings — we only need the first - // few KB to make the call. Walk back to a UTF-8 char boundary so we - // never slice in the middle of a multibyte sequence (4096 may land - // mid-codepoint). - let head = if s.len() > 4096 { - let mut end = 4096; - while end > 0 && !s.is_char_boundary(end) { - end -= 1; - } - &s[..end] - } else { - s - }; - let lower = head.to_ascii_lowercase(); - const MARKERS: &[&str] = &[ - "", - "", - "

", - "
", - "", - "Hi")); - assert!(looks_like_html("

Hello world

")); - assert!(looks_like_html("
Mixed Case
")); - assert!(looks_like_html("
link")); - } - - #[test] - fn looks_like_html_does_not_panic_on_multibyte_at_boundary() { - // Build a string > 4096 bytes whose 4096th byte lands inside a - // 3-byte UTF-8 codepoint. Naive `&s[..4096]` slicing panics here. - let mut s = String::with_capacity(4200); - // 4095 ASCII bytes, then a 3-byte CJK char straddling 4095..4098. - s.push_str(&"a".repeat(4095)); - s.push('日'); - s.push_str(&"b".repeat(100)); - // Should not panic; result doesn't matter, only that it returns. - let _ = looks_like_html(&s); - } - - #[test] - fn looks_like_html_rejects_plain_text() { - assert!(!looks_like_html("just a normal email body")); - assert!(!looks_like_html("")); - assert!(!looks_like_html("a < b but no tags")); - assert!(!looks_like_html("<<>>")); // not a real tag - } - - #[test] - fn convert_walks_nested_arrays_and_objects() { - let mut v = json!({ - "messages": [ - { "id": "m1", "messageText": "

hello world

" }, - { "id": "m2", "messageText": "plain text only" } - ], - "nextPageToken": null - }); - convert_html_strings(&mut v, "gmail"); - let m1 = v["messages"][0]["messageText"].as_str().unwrap(); - // html2md emits at least the text content. - assert!(m1.contains("hello")); - assert!(m1.contains("world")); - assert!(!m1.contains("

")); - // Plain text untouched. - assert_eq!(v["messages"][1]["messageText"], "plain text only"); - } - - #[test] - fn post_process_unknown_toolkit_is_noop() { - let mut v = json!({ "body": "

hi

" }); - let original = v.clone(); - post_process("notion", "NOTION_FETCH_DATA", &mut v); - assert_eq!(v, original); - } - - #[test] - fn post_process_gmail_converts_html_fields() { - let mut v = json!({ - "data": { "html": "

Hi

body

", "subject": "no tags here" } - }); - post_process("gmail", "GMAIL_FETCH_EMAILS", &mut v); - let html_field = v["data"]["html"].as_str().unwrap(); - assert!(!html_field.contains("

")); - assert!(html_field.contains("Hi")); - assert_eq!(v["data"]["subject"], "no tags here"); - } -} diff --git a/src/openhuman/composio/providers/traits.rs b/src/openhuman/composio/providers/traits.rs index f053380e8..21706fb62 100644 --- a/src/openhuman/composio/providers/traits.rs +++ b/src/openhuman/composio/providers/traits.rs @@ -116,6 +116,29 @@ pub trait ComposioProvider: Send + Sync { Ok(()) } + /// Hook fired immediately after a Composio action executed against + /// this toolkit returns a **successful** response. The provider may + /// mutate `data` in place to reshape the upstream payload before it + /// is handed back to the agent / RPC caller (e.g. convert Gmail's + /// HTML message bodies to markdown to save context tokens). + /// + /// `slug` is the full action slug (e.g. `"GMAIL_FETCH_EMAILS"`) so + /// providers can dispatch per action. `arguments` is the caller's + /// original argument object — providers can read opt-out flags from + /// it (e.g. `raw_html: true` to preserve raw HTML). + /// + /// Errors from upstream are not routed here; only `successful` + /// responses. Default impl is a no-op so providers that have nothing + /// to rewrite don't need to override. + fn post_process_action_result( + &self, + slug: &str, + arguments: Option<&serde_json::Value>, + data: &mut serde_json::Value, + ) { + let _ = (slug, arguments, data); + } + /// Hook fired when a Composio trigger webhook arrives for this /// toolkit. `payload` is the raw provider payload as forwarded by /// the backend. Implementations should be defensive — payload diff --git a/src/openhuman/composio/tools.rs b/src/openhuman/composio/tools.rs index 32ace6bd4..e632480ef 100644 --- a/src/openhuman/composio/tools.rs +++ b/src/openhuman/composio/tools.rs @@ -29,7 +29,7 @@ use crate::openhuman::tools::traits::{PermissionLevel, Tool, ToolCategory, ToolR use super::client::ComposioClient; use super::providers::{ catalog_for_toolkit, classify_unknown, find_curated, get_provider, load_user_scope_or_default, - post_process, toolkit_from_slug, ToolScope, UserScopePref, + toolkit_from_slug, ToolScope, UserScopePref, }; /// Decision returned by [`evaluate_tool_visibility`]. @@ -481,8 +481,21 @@ impl Tool for ComposioExecuteTool { // (e.g. gmail HTML → markdown). Only run on successful // responses; errors are passed through verbatim. if resp.successful { + super::providers::init_default_providers(); if let Some(toolkit) = toolkit_from_slug(&tool) { - post_process::post_process(&toolkit, &tool, &mut resp.data); + if let Some(provider) = get_provider(&toolkit) { + tracing::trace!( + toolkit = toolkit.as_str(), + tool = tool.as_str(), + has_args = arguments.is_some(), + "[composio_execute] post-processing action result" + ); + provider.post_process_action_result( + &tool, + arguments.as_ref(), + &mut resp.data, + ); + } } } Ok(ToolResult::success( diff --git a/src/openhuman/config/schema/context.rs b/src/openhuman/config/schema/context.rs index 45840df4f..d88b0c7e2 100644 --- a/src/openhuman/config/schema/context.rs +++ b/src/openhuman/config/schema/context.rs @@ -77,7 +77,9 @@ pub struct ContextConfig { /// compresses the payload into a dense note that preserves /// identifiers and key facts, and the compressed summary replaces /// the raw payload before it enters agent history. Set to `0` to - /// disable summarization entirely. Default: `500_000` tokens. + /// disable summarization entirely (the default). Set to any value + /// `> 0` to enable summarization once a payload crosses that token + /// threshold. /// /// Token count is estimated as `chars / 4` (the same heuristic used /// by `tree_summarizer::estimate_tokens`). Pairs with @@ -144,7 +146,11 @@ fn default_tool_result_budget_bytes() -> usize { } fn default_summarizer_payload_threshold_tokens() -> usize { - 500_000 + // Disabled: 0 short-circuits the payload_summarizer wiring in the + // agent builder (see session/builder.rs `> 0` guard). The summarizer + // sub-agent was being invoked recursively in some flows; keep off + // until that's root-caused. + 0 } fn default_summarizer_max_payload_tokens() -> usize { diff --git a/src/openhuman/config/schema/tools.rs b/src/openhuman/config/schema/tools.rs index 06bf58ecf..769174e39 100644 --- a/src/openhuman/config/schema/tools.rs +++ b/src/openhuman/config/schema/tools.rs @@ -234,7 +234,7 @@ impl Default for SecretsConfig { // ── Native computer control (mouse + keyboard) ───────────────────── -#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)] +#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, Default)] pub struct ComputerControlConfig { /// Master toggle for mouse and keyboard tools. Disabled by default — /// the user must explicitly opt in. @@ -242,12 +242,6 @@ pub struct ComputerControlConfig { pub enabled: bool, } -impl Default for ComputerControlConfig { - fn default() -> Self { - Self { enabled: false } - } -} - // ── Agent integration tools (backend-proxied) ─────────────────────── /// Per-integration on/off toggle. diff --git a/src/openhuman/context/tool_result_budget.rs b/src/openhuman/context/tool_result_budget.rs index 7786e73d6..5e3de4648 100644 --- a/src/openhuman/context/tool_result_budget.rs +++ b/src/openhuman/context/tool_result_budget.rs @@ -18,10 +18,13 @@ use std::fmt::Write as _; -/// Default per-tool-result budget. Chosen to keep a single oversized -/// result from blowing out the prompt while still leaving room for -/// moderately chunky outputs (directory listings, small file contents, -/// condensed HTTP bodies). +/// Default per-tool-result budget. Large raw tool payloads are trimmed +/// inline before they enter history so parent-session tool output +/// cannot grow without bound. This remains compatible with the payload +/// summarizer: when summarization is enabled it can still replace very +/// large payloads earlier in the pipeline, and when it is disabled +/// (`summarizer_payload_threshold_tokens = 0`) this budget is the +/// default safeguard. pub const DEFAULT_TOOL_RESULT_BUDGET_BYTES: usize = 16 * 1024; /// Number of trailing bytes reserved for the truncation marker. The diff --git a/src/openhuman/learning/linkedin_enrichment.rs b/src/openhuman/learning/linkedin_enrichment.rs index aa4ab0d77..5340cadff 100644 --- a/src/openhuman/learning/linkedin_enrichment.rs +++ b/src/openhuman/learning/linkedin_enrichment.rs @@ -509,7 +509,7 @@ async fn search_gmail_for_linkedin(config: &Config) -> anyhow::Result> = Lazy::new(|| Mutex::new(())); +fn redact_title_for_log(title: &str) -> String { + let mut hasher = std::collections::hash_map::DefaultHasher::new(); + title.hash(&mut hasher); + format!( + "", + title.chars().count(), + hasher.finish() + ) +} + #[derive(Debug, Clone, Copy, Default)] pub struct ConversationPurgeStats { pub thread_count: usize, @@ -111,6 +122,37 @@ impl ConversationStore { Ok(message) } + pub fn update_thread_title( + &self, + thread_id: &str, + title: &str, + updated_at: &str, + ) -> Result { + let _guard = CONVERSATION_STORE_LOCK.lock(); + let index = self.thread_index_unlocked()?; + let entry = index + .get(thread_id) + .ok_or_else(|| format!("thread {} does not exist", thread_id))?; + let threads_path = self.ensure_root()?.join(THREADS_FILENAME); + append_jsonl( + &threads_path, + &ThreadLogEntry::Upsert { + thread_id: thread_id.to_string(), + title: title.to_string(), + created_at: entry.created_at.clone(), + updated_at: updated_at.to_string(), + }, + )?; + debug!( + "{LOG_PREFIX} updated thread title id={} title={} path={}", + thread_id, + redact_title_for_log(title), + threads_path.display() + ); + self.thread_summary_unlocked(thread_id)? + .ok_or_else(|| format!("thread {} missing after title update", thread_id)) + } + pub fn update_message( &self, thread_id: &str, @@ -415,6 +457,15 @@ pub fn append_message( ConversationStore::new(workspace_dir).append_message(thread_id, message) } +pub fn update_thread_title( + workspace_dir: PathBuf, + thread_id: &str, + title: &str, + updated_at: &str, +) -> Result { + ConversationStore::new(workspace_dir).update_thread_title(thread_id, title, updated_at) +} + pub fn update_message( workspace_dir: PathBuf, thread_id: &str, @@ -684,6 +735,27 @@ mod tests { assert!(result.is_err()); } + #[test] + fn update_thread_title_persists_latest_title() { + let (_temp, store) = make_store(); + store + .ensure_thread(CreateConversationThread { + id: "t1".to_string(), + title: "Chat Apr 10 12:00 PM".to_string(), + created_at: "2026-04-10T12:00:00Z".to_string(), + }) + .unwrap(); + + let updated = store + .update_thread_title("t1", "Invoice follow-up", "2026-04-10T12:03:00Z") + .unwrap(); + + assert_eq!(updated.title, "Invoice follow-up"); + let threads = store.list_threads().unwrap(); + assert_eq!(threads[0].title, "Invoice follow-up"); + assert_eq!(threads[0].created_at, "2026-04-10T12:00:00Z"); + } + #[test] fn conversation_store_new() { let tmp = TempDir::new().unwrap(); diff --git a/src/openhuman/memory/rpc_models.rs b/src/openhuman/memory/rpc_models.rs index 2a0b18ae3..37c93a62a 100644 --- a/src/openhuman/memory/rpc_models.rs +++ b/src/openhuman/memory/rpc_models.rs @@ -158,6 +158,15 @@ pub struct AppendConversationMessageRequest { pub message: ConversationMessageRecord, } +/// Request to generate or refresh a thread title after the first exchange. +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct GenerateConversationThreadTitleRequest { + pub thread_id: String, + #[serde(default)] + pub assistant_message: Option, +} + /// Request to patch a persisted message. #[derive(Debug, Clone, Serialize, Deserialize)] #[serde(deny_unknown_fields)] diff --git a/src/openhuman/providers/compatible.rs b/src/openhuman/providers/compatible.rs index d75c87d89..bee21e0e2 100644 --- a/src/openhuman/providers/compatible.rs +++ b/src/openhuman/providers/compatible.rs @@ -1458,9 +1458,7 @@ impl OpenAiCompatibleProvider { if let Some(tc_list) = choice.delta.tool_calls.as_ref() { for tc in tc_list { let idx = tc.index.unwrap_or(0); - let entry = tool_accum - .entry(idx) - .or_insert_with(StreamingToolCall::default); + let entry = tool_accum.entry(idx).or_default(); if let Some(id) = tc.id.as_ref() { if entry.id.is_none() { @@ -1570,8 +1568,8 @@ impl OpenAiCompatibleProvider { // `ApiChatResponse` from the accumulators so downstream code // sees the same shape as the non-streaming path. let tool_calls_for_api: Vec = tool_accum - .into_iter() - .map(|(_idx, c)| ToolCall { + .into_values() + .map(|c| ToolCall { id: c.id, kind: Some("function".to_string()), function: Some(Function { @@ -1586,7 +1584,7 @@ impl OpenAiCompatibleProvider { // JSON yet. Some( serde_json::from_str(&c.arguments) - .unwrap_or_else(|_| serde_json::Value::String(c.arguments)), + .unwrap_or(serde_json::Value::String(c.arguments)), ) }, }), diff --git a/src/openhuman/routing/provider.rs b/src/openhuman/routing/provider.rs index a1c9f3ce0..d34c9ea3d 100644 --- a/src/openhuman/routing/provider.rs +++ b/src/openhuman/routing/provider.rs @@ -297,7 +297,7 @@ impl IntelligentRoutingProvider { model: &str, temperature: f64, ) -> Result { - let has_tools = request.tools.map_or(false, |t| !t.is_empty()); + let has_tools = request.tools.is_some_and(|t| !t.is_empty()); let (primary, fallback, category, local_healthy) = self.resolve(model).await; let started = Instant::now(); let mut fallback_occurred = false; diff --git a/src/openhuman/threads/ops.rs b/src/openhuman/threads/ops.rs index 823f5ae6c..85637360d 100644 --- a/src/openhuman/threads/ops.rs +++ b/src/openhuman/threads/ops.rs @@ -10,15 +10,21 @@ use crate::openhuman::memory::{ ApiEnvelope, ApiMeta, AppendConversationMessageRequest, ConversationMessageRecord, ConversationMessagesRequest, ConversationMessagesResponse, ConversationThreadSummary, ConversationThreadsListResponse, DeleteConversationThreadRequest, - DeleteConversationThreadResponse, EmptyRequest, PaginationMeta, - PurgeConversationThreadsResponse, UpdateConversationMessageRequest, + DeleteConversationThreadResponse, EmptyRequest, GenerateConversationThreadTitleRequest, + PaginationMeta, PurgeConversationThreadsResponse, UpdateConversationMessageRequest, UpsertConversationThreadRequest, }; +use crate::openhuman::providers::{self, ProviderRuntimeOptions}; use crate::rpc::RpcOutcome; use serde::Serialize; use std::collections::BTreeMap; +use std::hash::{Hash, Hasher}; use std::path::PathBuf; +const THREAD_TITLE_LOG_PREFIX: &str = "[threads:title]"; +const THREAD_TITLE_MODEL_HINT: &str = "hint:summarize"; +const THREAD_TITLE_SYSTEM_PROMPT: &str = "You generate short, specific chat thread titles from the first user message and the assistant reply. Return only the title text. Keep it under 8 words. No quotes. No markdown. No trailing punctuation unless it is part of a proper noun."; + fn request_id() -> String { uuid::Uuid::new_v4().to_string() } @@ -30,6 +36,12 @@ fn counts(entries: impl IntoIterator) -> BTreeMap< .collect() } +fn title_log_fingerprint(title: &str) -> String { + let mut hasher = std::collections::hash_map::DefaultHasher::new(); + title.hash(&mut hasher); + format!("{:016x}", hasher.finish()) +} + fn envelope( data: T, counts: Option>, @@ -92,6 +104,86 @@ fn record_to_message(record: ConversationMessageRecord) -> ConversationMessage { } } +fn is_auto_generated_thread_title(title: &str) -> bool { + let trimmed = title.trim(); + let bytes = trimmed.as_bytes(); + if bytes.len() < 16 || !trimmed.starts_with("Chat ") { + return false; + } + + let month_end = 8; + if bytes.len() <= month_end || !bytes[5..month_end].iter().all(|b| b.is_ascii_alphabetic()) { + return false; + } + if bytes.get(month_end) != Some(&b' ') { + return false; + } + + let mut idx = month_end + 1; + let day_start = idx; + while idx < bytes.len() && bytes[idx].is_ascii_digit() { + idx += 1; + } + if idx == day_start || idx - day_start > 2 { + return false; + } + if bytes.get(idx) != Some(&b' ') { + return false; + } + idx += 1; + + let hour_start = idx; + while idx < bytes.len() && bytes[idx].is_ascii_digit() { + idx += 1; + } + if idx == hour_start || idx - hour_start > 2 { + return false; + } + if bytes.get(idx) != Some(&b':') { + return false; + } + idx += 1; + + if idx + 2 >= bytes.len() + || !bytes[idx].is_ascii_digit() + || !bytes[idx + 1].is_ascii_digit() + || bytes[idx + 2] != b' ' + { + return false; + } + idx += 3; + + matches!(&trimmed[idx..], "AM" | "PM") +} + +fn collapse_whitespace(input: &str) -> String { + input.split_whitespace().collect::>().join(" ") +} + +fn sanitize_generated_title(raw: &str) -> Option { + let line = raw + .lines() + .find(|line| !line.trim().is_empty()) + .unwrap_or(raw) + .trim(); + let trimmed = line + .trim_matches(|c: char| matches!(c, '"' | '\'' | '`')) + .trim() + .trim_end_matches(['.', '!', '?', ':', ';']) + .trim(); + let collapsed = collapse_whitespace(trimmed); + if collapsed.is_empty() { + return None; + } + Some(collapsed.chars().take(80).collect()) +} + +fn build_title_prompt(user_message: &str, assistant_message: &str) -> String { + format!( + "First user message:\n{user_message}\n\nAssistant reply:\n{assistant_message}\n\nReturn the best thread title." + ) +} + /// Lists all conversation threads. pub async fn threads_list( _request: EmptyRequest, @@ -184,6 +276,180 @@ pub async fn message_append( )) } +/// Generates a durable thread title from the first user message and assistant reply. +pub async fn thread_generate_title( + request: GenerateConversationThreadTitleRequest, +) -> Result>, String> { + let config = Config::load_or_init() + .await + .map_err(|e| format!("load config: {e}"))?; + let dir = config.workspace_dir.clone(); + let Some(thread) = conversations::list_threads(dir.clone())? + .into_iter() + .find(|thread| thread.id == request.thread_id) + else { + return Err(format!("thread {} not found", request.thread_id)); + }; + + if !is_auto_generated_thread_title(&thread.title) { + tracing::debug!( + thread_id = %request.thread_id, + title_len = thread.title.chars().count(), + title_hash = %title_log_fingerprint(&thread.title), + "{THREAD_TITLE_LOG_PREFIX} skipping non-placeholder title" + ); + return Ok(envelope( + thread_to_summary(thread), + Some(counts([("num_threads", 1)])), + None, + )); + } + + let messages = conversations::get_messages(dir.clone(), &request.thread_id)?; + let Some(first_user_message) = messages + .iter() + .find(|message| message.sender == "user" && !message.content.trim().is_empty()) + .map(|message| message.content.trim().to_string()) + else { + tracing::debug!( + thread_id = %request.thread_id, + "{THREAD_TITLE_LOG_PREFIX} no user message yet; skipping" + ); + return Ok(envelope( + thread_to_summary(thread), + Some(counts([("num_threads", 1)])), + None, + )); + }; + + let assistant_message = request + .assistant_message + .as_deref() + .map(str::trim) + .filter(|message| !message.is_empty()) + .map(ToOwned::to_owned) + .or_else(|| { + messages + .iter() + .find(|message| message.sender == "agent" && !message.content.trim().is_empty()) + .map(|message| message.content.trim().to_string()) + }); + + let Some(assistant_message) = assistant_message else { + tracing::debug!( + thread_id = %request.thread_id, + "{THREAD_TITLE_LOG_PREFIX} no assistant message yet; skipping" + ); + return Ok(envelope( + thread_to_summary(thread), + Some(counts([("num_threads", 1)])), + None, + )); + }; + + let provider_runtime_options = ProviderRuntimeOptions { + auth_profile_override: None, + openhuman_dir: config.config_path.parent().map(std::path::PathBuf::from), + secrets_encrypt: config.secrets.encrypt, + reasoning_enabled: config.runtime.reasoning_enabled, + }; + + let provider = match providers::create_intelligent_routing_provider( + config.api_key.as_deref(), + config.api_url.as_deref(), + &config, + &provider_runtime_options, + ) { + Ok(provider) => provider, + Err(error) => { + tracing::warn!( + thread_id = %request.thread_id, + error = %error, + "{THREAD_TITLE_LOG_PREFIX} provider init failed; leaving placeholder title" + ); + return Ok(envelope( + thread_to_summary(thread), + Some(counts([("num_threads", 1)])), + None, + )); + } + }; + + tracing::debug!( + thread_id = %request.thread_id, + user_len = first_user_message.len(), + assistant_len = assistant_message.len(), + model = THREAD_TITLE_MODEL_HINT, + "{THREAD_TITLE_LOG_PREFIX} generating thread title" + ); + + let raw_title = match provider + .chat_with_system( + Some(THREAD_TITLE_SYSTEM_PROMPT), + &build_title_prompt(&first_user_message, &assistant_message), + THREAD_TITLE_MODEL_HINT, + 0.2, + ) + .await + { + Ok(title) => title, + Err(error) => { + tracing::warn!( + thread_id = %request.thread_id, + error = %error, + "{THREAD_TITLE_LOG_PREFIX} title generation failed; leaving placeholder title" + ); + return Ok(envelope( + thread_to_summary(thread), + Some(counts([("num_threads", 1)])), + None, + )); + } + }; + + let Some(title) = sanitize_generated_title(&raw_title) else { + tracing::warn!( + thread_id = %request.thread_id, + raw_title_len = raw_title.chars().count(), + raw_title_hash = %title_log_fingerprint(&raw_title), + "{THREAD_TITLE_LOG_PREFIX} generated empty title after sanitization" + ); + return Ok(envelope( + thread_to_summary(thread), + Some(counts([("num_threads", 1)])), + None, + )); + }; + + if title == thread.title { + return Ok(envelope( + thread_to_summary(thread), + Some(counts([("num_threads", 1)])), + None, + )); + } + + let updated = conversations::update_thread_title( + dir, + &request.thread_id, + &title, + &chrono::Utc::now().to_rfc3339(), + )?; + + tracing::debug!( + thread_id = %request.thread_id, + title_len = updated.title.chars().count(), + title_hash = %title_log_fingerprint(&updated.title), + "{THREAD_TITLE_LOG_PREFIX} updated thread title" + ); + + Ok(envelope( + thread_to_summary(updated), + Some(counts([("num_threads", 1)])), + None, + )) +} + /// Updates metadata on an existing conversation message. pub async fn message_update( request: UpdateConversationMessageRequest, diff --git a/src/openhuman/threads/schemas.rs b/src/openhuman/threads/schemas.rs index 94b1868aa..0e64acabb 100644 --- a/src/openhuman/threads/schemas.rs +++ b/src/openhuman/threads/schemas.rs @@ -7,7 +7,8 @@ use crate::core::all::{ControllerFuture, RegisteredController}; use crate::core::{ControllerSchema, FieldSchema, TypeSchema}; use crate::openhuman::memory::{ AppendConversationMessageRequest, ConversationMessagesRequest, DeleteConversationThreadRequest, - EmptyRequest, UpdateConversationMessageRequest, UpsertConversationThreadRequest, + EmptyRequest, GenerateConversationThreadTitleRequest, UpdateConversationMessageRequest, + UpsertConversationThreadRequest, }; use super::ops; @@ -19,6 +20,7 @@ pub fn all_controller_schemas() -> Vec { schemas("create_new"), schemas("messages_list"), schemas("message_append"), + schemas("generate_title"), schemas("message_update"), schemas("delete"), schemas("purge"), @@ -47,6 +49,10 @@ pub fn all_registered_controllers() -> Vec { schema: schemas("message_append"), handler: handle_message_append, }, + RegisteredController { + schema: schemas("generate_title"), + handler: handle_generate_title, + }, RegisteredController { schema: schemas("message_update"), handler: handle_message_update, @@ -192,6 +198,33 @@ pub fn schemas(function: &str) -> ControllerSchema { required: true, }], }, + "generate_title" => ControllerSchema { + namespace: "threads", + function: "generate_title", + description: + "Generate a short thread title from the first user message and assistant reply.", + inputs: vec![ + FieldSchema { + name: "thread_id", + ty: TypeSchema::String, + comment: "Thread identifier.", + required: true, + }, + FieldSchema { + name: "assistant_message", + ty: TypeSchema::Option(Box::new(TypeSchema::String)), + comment: + "Optional completed assistant reply to use instead of the stored first agent message.", + required: false, + }, + ], + outputs: vec![FieldSchema { + name: "result", + ty: TypeSchema::Json, + comment: "Envelope with the resulting thread summary.", + required: true, + }], + }, "delete" => ControllerSchema { namespace: "threads", function: "delete", @@ -275,6 +308,13 @@ fn handle_message_append(params: Map) -> ControllerFuture { }) } +fn handle_generate_title(params: Map) -> ControllerFuture { + Box::pin(async move { + let p = parse::(params)?; + to_json(ops::thread_generate_title(p).await?) + }) +} + fn handle_message_update(params: Map) -> ControllerFuture { Box::pin(async move { let p = parse::(params)?; @@ -313,6 +353,7 @@ mod tests { "create_new", "messages_list", "message_append", + "generate_title", "message_update", "delete", "purge", diff --git a/src/openhuman/tokenjuice/reduce.rs b/src/openhuman/tokenjuice/reduce.rs index 265996c33..806fd6726 100644 --- a/src/openhuman/tokenjuice/reduce.rs +++ b/src/openhuman/tokenjuice/reduce.rs @@ -672,11 +672,12 @@ fn format_inline( summary: &str, facts: &HashMap, ) -> String { - let fact_parts: Vec = facts + let mut fact_parts: Vec = facts .iter() .filter(|(_, &count)| count > 0) .map(|(name, &count)| pluralize(count, name)) .collect(); + fact_parts.sort_unstable(); let mut lines: Vec = Vec::new(); if input.exit_code.map(|c| c != 0).unwrap_or(false) { diff --git a/src/openhuman/tools/impl/agent/complete_onboarding.rs b/src/openhuman/tools/impl/agent/complete_onboarding.rs index e11f8c6ef..79949b6f2 100644 --- a/src/openhuman/tools/impl/agent/complete_onboarding.rs +++ b/src/openhuman/tools/impl/agent/complete_onboarding.rs @@ -109,6 +109,12 @@ pub fn reset_welcome_exchange_count() { pub struct CompleteOnboardingTool; +impl Default for CompleteOnboardingTool { + fn default() -> Self { + Self::new() + } +} + impl CompleteOnboardingTool { pub fn new() -> Self { Self diff --git a/src/openhuman/tools/impl/computer/mouse.rs b/src/openhuman/tools/impl/computer/mouse.rs index 495cf21e9..869bf1ae6 100644 --- a/src/openhuman/tools/impl/computer/mouse.rs +++ b/src/openhuman/tools/impl/computer/mouse.rs @@ -55,7 +55,7 @@ fn require_xy(args: &Value) -> anyhow::Result<(i32, i32)> { } fn validate_coord(name: &str, value: i64) -> anyhow::Result<()> { - if value < 0 || value > MAX_COORD { + if !(0..=MAX_COORD).contains(&value) { anyhow::bail!("'{name}' coordinate {value} is out of range (0..{MAX_COORD})"); } Ok(()) diff --git a/src/openhuman/tools/impl/system/current_time.rs b/src/openhuman/tools/impl/system/current_time.rs new file mode 100644 index 000000000..65bab14a3 --- /dev/null +++ b/src/openhuman/tools/impl/system/current_time.rs @@ -0,0 +1,167 @@ +//! Tool: current_time — returns the current time in UTC and local time zones. +//! +//! Gives the orchestrator (and other agents) a way to ground reasoning that +//! depends on "now" — reminders, scheduling, relative date parsing — without +//! having to shell out to `date`. Read-only, no arguments beyond an optional +//! IANA timezone for a convenience conversion. + +use crate::openhuman::tools::traits::{PermissionLevel, Tool, ToolResult}; +use async_trait::async_trait; +use chrono::{Local, SecondsFormat, Utc}; +use chrono_tz::Tz; +use serde_json::json; + +pub struct CurrentTimeTool; + +impl CurrentTimeTool { + pub fn new() -> Self { + Self + } +} + +impl Default for CurrentTimeTool { + fn default() -> Self { + Self::new() + } +} + +#[async_trait] +impl Tool for CurrentTimeTool { + fn name(&self) -> &str { + "current_time" + } + + fn description(&self) -> &str { + "Get the current date and time in UTC and the machine's local timezone. \ + Optionally convert to a specific IANA timezone (e.g. 'America/Los_Angeles', \ + 'Asia/Kolkata'). Use before scheduling reminders / cron jobs or when the \ + user refers to relative times like 'in 10 minutes', 'tomorrow', 'tonight'." + } + + fn parameters_schema(&self) -> serde_json::Value { + json!({ + "type": "object", + "properties": { + "timezone": { + "type": "string", + "description": "Optional IANA timezone name (e.g. 'Europe/London'). \ + If omitted, only UTC and machine-local are returned." + } + } + }) + } + + fn permission_level(&self) -> PermissionLevel { + PermissionLevel::ReadOnly + } + + async fn execute(&self, args: serde_json::Value) -> anyhow::Result { + tracing::debug!(args = %args, "[current_time] execute start"); + let now_utc = Utc::now(); + let now_local = Local::now(); + + let mut payload = json!({ + "utc": now_utc.to_rfc3339_opts(SecondsFormat::Secs, true), + "local": now_local.to_rfc3339_opts(SecondsFormat::Secs, true), + "local_timezone": now_local.format("%Z").to_string(), + "unix_seconds": now_utc.timestamp(), + "weekday": now_local.format("%A").to_string(), + }); + + if let Some(tz_name) = args.get("timezone").and_then(|v| v.as_str()) { + let trimmed = tz_name.trim(); + tracing::debug!( + tz_name = tz_name, + trimmed = trimmed, + now_utc = %now_utc, + now_local = %now_local, + "[current_time] normalized timezone input" + ); + if !trimmed.is_empty() { + match trimmed.parse::() { + Ok(tz) => { + let converted = now_utc.with_timezone(&tz); + tracing::debug!( + trimmed = trimmed, + converted = %converted, + "[current_time] timezone conversion succeeded" + ); + payload["requested_timezone"] = json!({ + "name": trimmed, + "time": converted.to_rfc3339_opts(SecondsFormat::Secs, true), + "weekday": converted.format("%A").to_string(), + }); + } + Err(error) => { + tracing::debug!( + trimmed = trimmed, + error = %error, + "[current_time] timezone conversion failed" + ); + payload["requested_timezone_error"] = json!(format!( + "Unknown IANA timezone '{trimmed}' — use names like 'America/Los_Angeles'." + )); + } + } + } + } + + tracing::debug!("[current_time] returning payload: {payload}"); + Ok(ToolResult::success(serde_json::to_string_pretty(&payload)?)) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn name_and_permission() { + let tool = CurrentTimeTool::new(); + assert_eq!(tool.name(), "current_time"); + assert_eq!(tool.permission_level(), PermissionLevel::ReadOnly); + } + + #[test] + fn schema_is_object() { + let schema = CurrentTimeTool::new().parameters_schema(); + assert_eq!(schema["type"], "object"); + } + + #[tokio::test] + async fn returns_utc_and_local() { + let result = CurrentTimeTool::new().execute(json!({})).await.unwrap(); + assert!(!result.is_error); + let payload: serde_json::Value = serde_json::from_str(&result.output()).unwrap(); + assert!(payload["utc"].is_string()); + assert!(payload["local"].is_string()); + assert!(payload["unix_seconds"].is_number()); + } + + #[tokio::test] + async fn converts_requested_timezone() { + let result = CurrentTimeTool::new() + .execute(json!({ "timezone": "Asia/Kolkata" })) + .await + .unwrap(); + assert!(!result.is_error); + let payload: serde_json::Value = serde_json::from_str(&result.output()).unwrap(); + assert!(payload["requested_timezone"].is_object()); + assert!(payload["requested_timezone"]["name"].is_string()); + assert!(payload["requested_timezone"]["name"] + .as_str() + .unwrap() + .contains("Asia/Kolkata")); + } + + #[tokio::test] + async fn unknown_timezone_reports_error_field() { + let result = CurrentTimeTool::new() + .execute(json!({ "timezone": "Not/AReal_Zone" })) + .await + .unwrap(); + assert!(!result.is_error); + let payload: serde_json::Value = serde_json::from_str(&result.output()).unwrap(); + assert!(payload["requested_timezone_error"].is_string()); + } +} diff --git a/src/openhuman/tools/impl/system/mod.rs b/src/openhuman/tools/impl/system/mod.rs index 4db62a210..2181f5d60 100644 --- a/src/openhuman/tools/impl/system/mod.rs +++ b/src/openhuman/tools/impl/system/mod.rs @@ -1,3 +1,4 @@ +mod current_time; mod insert_sql_record; mod proxy_config; mod pushover; @@ -6,6 +7,7 @@ mod shell; mod tool_stats; mod workspace_state; +pub use current_time::CurrentTimeTool; pub use insert_sql_record::InsertSqlRecordTool; pub use proxy_config::ProxyConfigTool; pub use pushover::PushoverTool; diff --git a/src/openhuman/tools/ops.rs b/src/openhuman/tools/ops.rs index b371d735f..1efcf7799 100644 --- a/src/openhuman/tools/ops.rs +++ b/src/openhuman/tools/ops.rs @@ -84,6 +84,7 @@ pub fn all_tools_with_runtime( // `agent::harness::subagent_runner` for the dispatch path. Box::new(SpawnSubagentTool::new()), Box::new(CompleteOnboardingTool::new()), + Box::new(CurrentTimeTool::new()), Box::new(CronAddTool::new(config.clone(), security.clone())), Box::new(CronListTool::new(config.clone())), Box::new(CronRemoveTool::new(config.clone())), @@ -394,6 +395,41 @@ mod tests { ); } + #[test] + fn all_tools_includes_current_time() { + let tmp = TempDir::new().unwrap(); + let security = Arc::new(SecurityPolicy::default()); + let mem_cfg = MemoryConfig { + backend: "markdown".into(), + ..MemoryConfig::default() + }; + let mem: Arc = + Arc::from(crate::openhuman::memory::create_memory(&mem_cfg, tmp.path(), None).unwrap()); + + let browser = BrowserConfig::default(); + let http = crate::openhuman::config::HttpRequestConfig::default(); + let cfg = test_config(&tmp); + + let tools = all_tools( + Arc::new(Config::default()), + &security, + mem, + None, + None, + &browser, + &http, + tmp.path(), + &HashMap::new(), + None, + &cfg, + ); + let names: Vec<&str> = tools.iter().map(|t| t.name()).collect(); + assert!( + names.contains(&"current_time"), + "current_time must be registered in the default tool list; got: {names:?}" + ); + } + #[test] fn all_tools_excludes_browser_when_disabled() { let tmp = TempDir::new().unwrap(); diff --git a/src/openhuman/voice/hallucination.rs b/src/openhuman/voice/hallucination.rs index e15e5a3a5..a47e92e4d 100644 --- a/src/openhuman/voice/hallucination.rs +++ b/src/openhuman/voice/hallucination.rs @@ -180,7 +180,7 @@ pub fn is_hallucinated_output(text: &str, mode: HallucinationMode) -> bool { for w in &clean_words { *counts.entry(w.as_str()).or_insert(0) += 1; } - for (_word, count) in &counts { + for count in counts.values() { let ratio = *count as f64 / total as f64; if ratio > 0.6 && *count >= 5 { debug!( diff --git a/src/openhuman/voice/streaming.rs b/src/openhuman/voice/streaming.rs index 26c5f1bdf..ff38da12d 100644 --- a/src/openhuman/voice/streaming.rs +++ b/src/openhuman/voice/streaming.rs @@ -37,7 +37,7 @@ struct ClientCommand { } fn decode_pcm16le_frame(data: &[u8]) -> Option> { - if data.len() % 2 != 0 { + if !data.len().is_multiple_of(2) { return None; }