79 Commits

Include renames

Author	SHA1	Message	Date
3 people	4a5a00cb96	feat(notifications): webview notifications end-to-end (Phase 7) (#741) ... * wip(notifications): core module skeleton + controller registration Phase 7 PR #714 work-in-progress. Shipped: - src/openhuman/webview_notifications/ — mod, types, dispatch, bus, schemas - pub mod declaration in src/openhuman/mod.rs - controllers entry in src/core/all.rs Remaining: schemas line in all.rs, Tauri shell edits (OpenHuman: prefix, feature flag gate, click event emit), new shell module for feature flag state + commands, frontend app/src/lib/webviewNotifications/ IPC wrapper, Redux click handler. * feat(notifications): register webview_notifications schemas in core registry Adds the schemas line next to the already-registered controllers entry so the domain participates in declared-schema validation. v1 controller list is empty (settings live shell-side), but the wiring is in place for future additions. * feat(notifications): add OpenHuman: prefix, feature flag, and click-routing event Updates the CEF notification hook in webview_accounts to: - Prefix OS toast titles with "OpenHuman:" so they visually disambiguate from natively installed apps (Slack, Gmail, Discord desktop) firing the same DM twice. - Gate delivery on a new shell-side feature flag (off by default). - Mirror each fire to the React frontend via a `webview-notification:fired` Tauri event carrying {account_id, provider, title, body, tag}, so the UI can route click-to-focus back to the originating webview. Adds a new notification_settings shell module holding the runtime toggle as an AtomicBool (lock-free read from the CEF callback thread) plus notification_settings_{get,set} Tauri commands for the settings UI to flip the flag. State lives shell-side rather than in the core sidecar so the toggle doesn't require a JSON-RPC round-trip. * feat(notifications): frontend IPC wrapper + Redux click routing Mirrors the Rust-side `webview-notification:fired` Tauri event into Redux so the UI can: - bump an unread badge on the originating account (sidebar surface reads `accounts.unread[accountId]`) - route a subsequent click intent back to the right embedded webview via `focusAccountFromNotification`, which sets `activeAccountId` and clears the unread counter in one action Plumbing: - `app/src/lib/webviewNotifications/` — typed subscribe/unsubscribe, idempotent `started` guard mirroring `webviewAccountService.ts`, `handleNotificationClick(accountId)` as the public click entrypoint so in-app toast UI or a future OS-notification click hook share one Redux intent - `accountsSlice`: `noteWebviewNotificationFired` and `focusAccountFromNotification` reducers (both no-op for unknown account ids — guards against stale payloads from a closed webview) - `App.tsx`: start the service at boot, right next to the existing `startWebviewAccountService()` call Tests: - `accountsSlice.webviewNotifications.test.ts` — reducer behavior - `service.test.ts` — fired dispatches + click focuses via real store Also: `cargo fmt` noise fixups on `src/core/all.rs` and `src/openhuman/webview_notifications/mod.rs` so the branch passes `cargo fmt --check` in CI. --------- Co-authored-by: Jwalin Shah <jshah1331@gmail.com>	2026-04-21 22:25:12 -07:00
4 people	0ce1253fe1	feat(memory): Phase 1 memory tree - multi-source ingestion & canonical chunks (#707) (#732) ... * feat(memory): phase 1 memory tree - multi-source ingestion & canonical chunks (#707) Adds an isolated memory tree layer under src/openhuman/memory/tree/ implementing Phase 1 of the new memory architecture (umbrella #711). Zero edits to existing memory/*.rs files - the new layer coexists with the legacy TinyHumans-backed client. - Source adapters: chat / email / document -> canonical Markdown - Token-bounded chunker with deterministic SHA-256 chunk IDs - SQLite persistence at <workspace>/memory_tree/chunks.db with full provenance metadata (source_kind, source_id, owner, timestamps, tags, time_range) and back-pointer to raw source - Unified JSON-RPC ingest (dispatches on source_kind + JSON payload): openhuman.memory_tree_ingest, _list_chunks, _get_chunk - DataSource enum covering the 8 providers from m.excalidraw step 1 (Discord/Telegram/Whatsapp/Gmail/OtherEmail/Notion/MeetingNotes/DriveDocs) - ~40 unit tests (chunk ID stability, UTF-8-safe splitting, canonicalisation idempotence, store round-trip, filter behavior) Additive only: new tables in a new DB file, new JSON-RPC namespace, no existing behavior changes. Feeds #708 (scoring), #709 (summary trees), #710 (query tools). Closes #707. Parent: #711. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(memory): enhance memory tree functionality and tests - Added support for "memory_tree" namespace description in `namespace_description`. - Implemented clamping for zero token budget in `chunk_markdown` to prevent empty leading chunks. - Introduced detailed logging for document ingestion, including title length. - Updated output schemas in `schemas.rs` for improved clarity. - Enhanced chunk listing with clamping limits and ordering by sequence in `list_chunks`. - Normalized source references in canonicalization functions to drop blank values. - Added comprehensive tests for new features and edge cases in chunking and canonicalization. These changes improve the robustness and usability of the memory tree layer, aligning with ongoing development efforts for Phase 1 of the memory architecture. * fix(memory): address follow-up CodeRabbit comments --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai>	2026-04-21 14:40:27 -07:00
Steven EnamakelandGitHub	fff24a6c6f	fix(prompts): v2 prompt pipeline, integrations_agent, and subagent session plumbing (#642) ... * refactor(transcript): update session transcript paths and enhance directory structure - Changed the source of truth path for session transcripts from `sessions/{DDMMYYYY}/{agent}_{index}.jsonl` to `session_raw/{DDMMYYYY}/{agent}_{index}.jsonl` to better reflect the organization of files. - Updated the logic for creating and resolving transcript paths to accommodate the new directory structure, ensuring compatibility with legacy `.md` files. - Improved documentation to clarify the changes in file organization and their implications for transcript management. This refactor enhances the clarity and maintainability of session transcript handling by establishing a more logical file structure. * refactor(prompts): update tool handling in prompt builders - Replaced `Vec<String>` with `Vec<ToolSummary<'_>>` for available tools in multiple agent prompt builders, enhancing type safety and clarity. - Introduced `render_tool_catalog` and `render_connected_integrations` functions to dynamically generate sections in prompts based on available tools and connected integrations. - Updated the `build` function in various agent prompts to utilize the new rendering functions, ensuring that prompts accurately reflect the current context and available resources. These changes improve the maintainability and functionality of the prompt generation process across different agents. * refactor(prompts): streamline prompt builders for agent templates - Updated the prompt builders for various agents to utilize the sibling `prompt.md` template directly, enhancing clarity and maintainability. - Replaced `Vec<ToolSummary<'_>>` with `Vec<ToolSummary>` and `Vec<ConnectedIntegration>` for improved type safety in test cases. - Adjusted the `build` function to ensure consistent formatting and handling of tool catalogs across different agents. These changes simplify the prompt generation process and prepare the codebase for future enhancements. * refactor(harness): unify tool filtering and prompt loading for debug consistency - Exposed `filter_tool_indices` and `load_prompt_source` as `pub(crate)` to ensure that both the live runner and debug dump share the same filtering and loading logic, eliminating discrepancies. - Enhanced documentation for both functions to clarify their purpose and usage, improving maintainability and understanding of the codebase. These changes streamline the tool management process and enhance the reliability of debug outputs, ensuring consistency across different contexts. * refactor(prompts): enhance prompt builders for agent templates - Updated the prompt builders for various agents to return fully-assembled system prompts, incorporating section helpers from `crate::openhuman::context::prompt`. - Replaced `Vec<ToolSummary<'_>>` with `Vec<ToolSummary>` and `Vec<ConnectedIntegration>` for improved type safety in test cases. - Adjusted the `build` function to ensure consistent formatting and handling of user files, tools, and workspace sections across different agents. These changes streamline the prompt generation process, improve maintainability, and prepare the codebase for future enhancements. * refactor(prompts): enhance prompt context handling for dynamic sources - Updated the prompt builders to support fully-assembled prompts from dynamic sources, allowing for more flexible prompt generation. - Introduced `PromptTool` and `PromptContext` structures to replace `ToolSummary`, improving type safety and clarity in prompt construction. - Refactored the handling of prompt sources in both the subagent runner and session builder to streamline the integration of dynamic prompts and legacy sources. These changes improve the maintainability and functionality of the prompt generation process, ensuring accurate representation of available tools and context in agent interactions. * refactor(prompts): improve prompt context and tool handling - Enhanced the `PromptContext` structure to include additional fields for better context management, such as `skills`, `dispatcher_instructions`, and `tool_call_format`. - Replaced `ToolSummary` with `PromptTool` for improved type safety and clarity in prompt generation. - Updated the handling of dynamic prompt sources in both the subagent runner and debug dump, ensuring consistent integration and rendering of prompts. - Introduced a mechanism to handle empty visible tool names, enhancing the robustness of prompt generation. These changes streamline the prompt construction process and improve the overall maintainability of the codebase. * refactor(prompts): reorganize prompt handling and introduce SystemPromptBuilder - Moved prompt-related types and builders from `openhuman::context::prompt` to `openhuman::agent::prompts` for better modularity. - Introduced `SystemPromptBuilder` to streamline the construction of system prompts, allowing for flexible section management. - Updated module exports to maintain compatibility while enhancing the organization of prompt-related code. These changes improve the clarity and maintainability of the prompt generation process, aligning it more closely with the agents that utilize these prompts. * refactor(prompts): unify agent prompt handling and update CLI references - Removed the "main" alias for the orchestrator in the prompt dumping process, treating it as just another registered agent. - Updated the `debug-agent-prompts.sh` script to reflect this change, ensuring all agents are included uniformly. - Revised documentation and error messages in `agent_cli.rs` to replace references to "main" with "orchestrator" for clarity. - Enhanced the debug dump functionality to maintain consistency across agent prompts, improving overall maintainability and usability. These changes streamline the prompt handling process and clarify the usage of agent identifiers in the CLI, aligning with the new architecture. * refactor(prompts): remove CACHE_BOUNDARY references from agent prompts - Eliminated the CACHE_BOUNDARY marker from various agent prompt files, streamlining the prompt generation process. - Updated the build functions in multiple agents to ensure consistent handling of workspace rendering without the cache boundary. - Refactored related prompt handling logic to enhance clarity and maintainability, aligning with the new architecture. These changes simplify the prompt structure and improve the overall efficiency of prompt generation across agents. * refactor(prompts): remove cache boundary references from tests and prompts - Eliminated all instances of cache boundary references from the subagent runner and related tests, simplifying the prompt handling logic. - Updated test assertions to reflect the removal of cache boundary checks, ensuring consistency across the testing framework. - Refactored the session manager to streamline the system prompt assembly process without relying on cache boundaries. These changes enhance the clarity and maintainability of the prompt generation process, aligning with the recent architectural updates. * refactor(harness): clean up unused imports and streamline code - Removed unnecessary imports from multiple files, including `RandomState`, `Hasher`, and `SerializeMap`, to enhance code clarity and maintainability. - Simplified the structure of several modules by eliminating redundant use statements, ensuring a cleaner and more efficient codebase. These changes contribute to a more organized and readable code structure, aligning with ongoing refactoring efforts. * refactor(prompts): enhance agent prompt structures and integration handling - Updated the `orchestrator`, `skills_agent`, and `welcome` prompts to streamline the rendering of connected integrations and delegation guides. - Introduced dedicated functions for rendering integration information, ensuring clarity in the agent's voice and responsibilities. - Removed redundant sections from the shared prompt builder, allowing each agent to manage its own prompt content more effectively. - Improved test coverage for prompt generation, ensuring accurate representation of connected integrations and skills. These changes enhance the maintainability and clarity of the prompt generation process, aligning with the recent architectural updates. * refactor(session): update integration handling and clean up prompt parameters - Revised documentation for `connected_integrations` in the `Agent` struct to clarify its role in the agent's prompt rendering. - Updated the parameter name in `render_subagent_system_prompt_with_format` to `_connected_integrations` to indicate it is unused, enhancing code clarity. - Cleaned up import statements in the context module for better organization and maintainability. These changes improve the clarity of integration handling and streamline the code structure, aligning with ongoing refactoring efforts. * refactor(agent_cli): simplify command options and improve documentation - Removed the `--skill` option from the `dump-prompt` command, streamlining the command usage and focusing on essential parameters. - Updated documentation to clarify the usage of the `dump-prompt` command and its parameters, enhancing user understanding. - Cleaned up the `DumpFlags` structure by removing unused fields, contributing to a more maintainable codebase. These changes improve the clarity and usability of the agent CLI, aligning with ongoing refactoring efforts. * refactor(cli): streamline dotenv loading and clean up prompt rendering - Introduced `load_dotenv_for_cli` to load environment variables for all CLI entrypoints, ensuring consistent configuration across commands. - Updated documentation to clarify the purpose of the dotenv loading mechanism. - Removed unnecessary blank lines in prompt rendering functions across multiple agents, enhancing code readability. These changes improve the maintainability and clarity of the CLI and prompt handling, aligning with ongoing refactoring efforts. * refactor(debug-agent-prompts): enhance environment loading and streamline workspace resolution - Updated the script to load environment variables from a `.env` file, ensuring consistent configuration for prompt generation. - Simplified workspace resolution by delegating to the binary's internal logic, improving reliability and reducing code duplication. - Revised documentation to clarify the usage of command options and the impact of environment variables on prompt rendering. These changes improve the maintainability and clarity of the debug agent prompts script, aligning with ongoing refactoring efforts. * refactor(agent): remove category filter and simplify agent definitions - Eliminated the `category_filter` from various agent definitions and related tests, streamlining the agent configuration. - Updated the `run_list` function in `agent_cli.rs` to reflect the removal of category filtering, enhancing output clarity. - Revised documentation and comments to remove references to the now-removed category filter, improving overall code maintainability. These changes contribute to a cleaner and more efficient agent architecture, aligning with ongoing refactoring efforts. * feat(agents): introduce integrations_agent and tools_agent for enhanced service handling - Added the `integrations_agent` to manage service integrations via Composio, including a new TOML configuration and prompt structure. - Introduced the `tools_agent` for general ad-hoc tasks using built-in OpenHuman tools, with its own configuration and prompt. - Updated the loader to include both agents in the built-in agent list, increasing the total number of agents from 13 to 14. - Revised orchestrator and welcome prompts to delegate integration tasks to the new `integrations_agent`, ensuring clarity in agent responsibilities. - Enhanced tests to verify the registration and functionality of the new agents, improving overall test coverage. These changes expand the capabilities of the agent architecture, allowing for more specialized handling of integrations and tool usage. * refactor(agents): update references from skills_agent to integrations_agent - Changed all instances of `skills_agent` to `integrations_agent` across various files, including prompts, CLI commands, and tool registrations. - Updated documentation and comments to reflect the new agent name, ensuring clarity in agent responsibilities and usage. - Revised debug scripts to align with the new prompt structure for the integrations agent. These changes enhance consistency in the codebase and improve the clarity of agent interactions. * refactor(agents): rename skills_agent to integrations_agent throughout the codebase - Updated all instances of `skills_agent` to `integrations_agent` in various files, including tests, documentation, and comments. - Ensured consistency in agent references to improve clarity in agent responsibilities and interactions. - Revised related code structures to align with the new naming convention, enhancing overall maintainability. These changes support the transition to the new agent architecture and improve code readability. * refactor(prompts): implement dynamic prompt rendering for enhanced context handling - Introduced `DynamicPromptSection` to allow prompts to be built dynamically using a function pointer, enabling real-time access to the `PromptContext`. - Updated `SystemPromptBuilder` to support dynamic prompts, ensuring that late-arriving state like `connected_integrations` is accurately reflected in the rendered output. - Revised the prompt handling logic in the agent builder to streamline the integration of dynamic prompts, improving overall flexibility and responsiveness. These changes enhance the prompt generation process, aligning with the ongoing improvements in agent architecture and context management. * refactor(prompts): refine delegation guide to display only connected integrations - Updated the `render_delegation_guide` function to list only the toolkits that are actively connected, omitting unauthorized toolkits to prevent hallucinations during delegation. - Revised related tests to ensure the prompt correctly reflects the current state of integrations, including scenarios where no integrations are connected. - Introduced a new utility function to filter out welcome-only tools from non-welcome agents, enhancing the clarity and safety of tool visibility. These changes improve the accuracy and focus of the delegation guide, aligning with the ongoing enhancements in agent prompt handling. * refactor(planner): update tool usage and prompt guidelines for read-only operations - Modified the `agent.toml` configuration to clarify that the planner operates in a read-only mode, specifying that it does not mutate the workspace or memory. - Revised the prompt guidelines to reflect the read-only nature of the planner, emphasizing the need for explicit nodes for any required writes to be handled by downstream agents. These changes enhance the clarity of the planner's operational constraints and improve the overall structure of the planning process. * refactor(config): remove web search enable flag and update related configurations - Eliminated the `OPENHUMAN_WEB_SEARCH_ENABLED` environment variable and associated logic, as web search is now always enabled by default. - Updated the configuration schema to reflect the removal of the enable flag from `WebSearchConfig`. - Adjusted tool registration to ensure web search is always available, simplifying the configuration process. These changes streamline the web search functionality, ensuring it is consistently available across all sessions. * refactor(config): update http_request flag to always enabled - Changed the `http_request` configuration to always be enabled, removing the dependency on the `config.http_request.enabled` flag. - This adjustment simplifies the configuration process and ensures consistent behavior across the application. These changes contribute to a more streamlined configuration and enhance the overall reliability of the onboarding process. * refactor(debug-agent-prompts): transition to dump-all command for agent prompts - Replaced the previous method of listing agent IDs and dumping prompts with a new `dump-all` command that consolidates the functionality into a single call. - Updated the script to handle output directory and workspace options more efficiently, leveraging Rust's `dump_all_agent_prompts` for processing. - Enhanced the handling of the `integrations_agent` to generate separate dumps for each connected toolkit, improving the clarity and organization of output files. - Revised related logging and summary generation to reflect the new structure, ensuring a more streamlined user experience. These changes modernize the prompt dumping process, aligning it with the latest architectural improvements and enhancing usability. * feat(composio): implement dynamic fetching of toolkit actions for integrations - Added a new `fetch_toolkit_actions` function to retrieve the current action catalogue for a specified Composio toolkit, enhancing the responsiveness of the integrations agent. - Updated the `subagent_runner` to utilize the fresh action list at spawn time, ensuring that the toolkit's actions reflect the latest backend state. - Modified the `render_integrations_agent` function to refresh the action catalogue during prompt generation, improving the accuracy of the displayed tools. These changes enhance the integration experience by providing up-to-date action information, aligning with the ongoing improvements in agent functionality. * refactor(integrations-agent): update tool visibility and configuration handling - Modified the `agent.toml` to replace `wildcard` with `named` tools, enhancing control over tool visibility for the integrations agent. - Updated the `subagent_runner` to ensure that tool visibility aligns with the new TOML configuration, preventing unnecessary stripping of tools. - Revised the `render_integrations_agent` function to respect the updated tool scope, improving the accuracy of the tool list generated for subagents. These changes streamline the tool management process, ensuring that only explicitly defined tools are available during agent execution. * feat(composio): add composio_list_connections tool for dynamic integration detection - Introduced the `composio_list_connections` tool in the orchestrator's configuration, allowing the agent to detect newly-authorized Composio integrations mid-session. - Enhanced the `ComposioListConnectionsTool` to filter and return only currently-connected integrations with ACTIVE or CONNECTED status, improving the accuracy of integration management. - Updated the tool's description to clarify its functionality and usage context. These changes enhance the agent's ability to manage integrations dynamically, aligning with ongoing improvements in the integration experience. * fix(prompt): update delegation guide to reference Skills page - Modified the `render_delegation_guide` function to change the reference from **Settings → Integrations** to the **Skills** page for connecting integrations. This update clarifies the user instructions for integration management. * feat(session): introduce session key management for sub-agents - Added `session_key` and `session_parent_prefix` fields to `ParentExecutionContext` to facilitate hierarchical transcript naming for sub-agents. - Updated `persist_subagent_transcript` to generate transcript filenames based on the parent's session key, ensuring a flat file structure that reflects the parent-child relationship. - Enhanced `AgentBuilder` and `Agent` to support the new session key management, allowing for better organization of session transcripts. - Adjusted related functions to ensure proper handling of session keys during agent execution and transcript persistence. These changes improve the clarity and organization of session transcripts, aligning with the ongoing enhancements in agent functionality. * refactor(tests): update integrations agent tests for tool scope and transcript handling - Renamed the `integrations_agent_is_wildcard` test to `integrations_agent_tool_scope_honours_toml` to better reflect its purpose of validating tool scope based on TOML configuration. - Enhanced the test to assert that the `integrations_agent` correctly recognizes named tools instead of a wildcard. - Removed the `integrations_agent_has_extra_tools_for_export` test as it is no longer relevant to the current tool management strategy. - Improved the `latest_in_dir` function to clarify the handling of transcript naming schemes, ensuring proper differentiation between legacy and keyed formats. These changes streamline the testing process and improve the accuracy of tool scope validation, aligning with recent updates in agent functionality. * refactor(subagent_runner): improve transcript persistence and streamline inner loop - Removed the `persist_subagent_transcript` function, transitioning transcript persistence to occur per-iteration within the `run_inner_loop`, enhancing reliability by ensuring transcripts are written immediately after each provider response. - Updated the handling of session keys to maintain consistent naming for transcripts, reflecting the parent-child relationship in the file structure. - Simplified the code by eliminating redundant post-loop transcript writes, aligning with recent changes in agent functionality and improving overall clarity in transcript management. * refactor(agent_cli, subagent_runner, session): improve code readability and formatting - Enhanced formatting in `agent_cli.rs` for better readability by adjusting the structure of string formatting. - Streamlined conditional checks in `subagent_runner.rs` to improve clarity and maintainability. - Simplified the handling of agent IDs in `builder.rs` to reduce line length and improve code flow. - Updated test cases in `tests.rs` for better alignment and readability of expected values. - Improved formatting in `debug_dump.rs` to enhance the clarity of toolkit action fetching and logging. These changes collectively enhance the overall readability and maintainability of the codebase, aligning with ongoing refactoring efforts. * fix(tests): add missing session_key fields to ParentExecutionContext stub * fix: address PR review feedback * fix(prompts-v2): round 2 PR review — dispatcher instructions, workspace-file preservation, cached-tool fallback, transcript persistence - subagent_runner: populate PromptContext.dispatcher_instructions for Dynamic prompts (was empty string, dropping the ## Tool Use Protocol block in render_tools for PFormat/Json/Native sub-agents) - subagent_runner: add post-tool persist_transcript after tool results are appended so a mid-round crash doesn't lose tool outputs - prompts::sync_workspace_file: preserve user-edited workspace files — only overwrite when the file doesn't exist OR its current hash matches the stored builtin hash - context::debug_dump: mirror runner's cached-tool fallback — keep cached action catalogue on empty/error from fetch_toolkit_actions instead of blanking it - core::agent_cli: add entry/exit debug logs around dump_all / dump_prompt calls and a trace log around each prompt file write; update module banner to note --toolkit is required when --agent is integrations_agent	2026-04-17 19:47:31 -07:00
3 people	f098cfda40	test(coverage): batches 13–18 — Rust unit tests for 40+ modules (#530) (#618) ... * test(coverage): batch 13–14 — core/all registry, proxy_config tool (#530) Add 36 new tests: - core/all: registry integrity (no duplicates, schema-controller parity), rpc_method_name, namespace_description, validate_params, schema lookup - proxy_config: parse_scope, parse_string_list (CSV, array, errors), parse_optional_string_update, env_snapshot, proxy_json, security gates All 4096 tests pass. * test(coverage): batch 15 — memory/store/unified/helpers pure functions (#530) Add 31 new tests for helpers module: - vec_to_bytes/bytes_to_vec roundtrip, cosine_similarity (identical, orthogonal, mismatched, zero), collapse_whitespace, normalize_search_text, tokenize_search_terms, normalize_graph_entity/predicate, json_string_array, merge_unique_string_arrays, json_i64 (int/float/missing/string), recency_score (current/old/future), chunk_document_content All 4127 tests pass. * test(coverage): batch 16 — terminal, supervision, security detect, trigger history (#530) Add 27 new tests across 4 modules: - accessibility/terminal: is_text_role, is_terminal_app, looks_like_terminal_buffer, extract_terminal_input_context, noise line detection - channels/runtime/supervision: compute_max_in_flight_messages clamping - security/detect: all sandbox backend fallback paths (landlock, firejail, bubblewrap, docker on non-linux), disabled-via-enabled-false - composio/trigger_history: list_recent with limit, empty store, field validation All 4154 tests pass. * test(coverage): batch 17 — learning, update, encryption, doctor, service schemas + skills/types (#530) Add 49 new tests across 6 modules: - learning/schemas: catalog, schema validation, unknown fallback - update/schemas: check/apply schemas, optional staging_dir, controller parity - encryption/schemas: encrypt/decrypt schemas, read_required helper - doctor/schemas: report/models schemas, read_optional helper (none/null/value/error) - service/schemas: 8 lifecycle functions, restart optional params, daemon_host - skills/types: ToolResult success/error/json/mixed, serde roundtrip, ToolContent All 4203 tests pass. * test(coverage): batch 18 — accessibility/keys key-state probes (#530) Add tests for is_tab_key_down/is_escape_key_down (platform-safe), macOS-specific constant validation. All 4216 tests pass. * test(coverage): batch 19 — service/restart, memory/store/factories (#530) Add 7 new tests: - service/restart: default source/reason, whitespace trimming, empty-string defaults, RestartStatus serde, startup delay noop - memory/store/factories: effective_memory_backend_name, migration-disabled error All 4223 tests pass. * test(coverage): batch 20 — tree_summarizer schemas, subconscious schemas (#530) Add 38 new tests across 2 modules: - tree_summarizer/schemas: catalog parity, all 5 functions, param helpers (read_required, read_optional, read_optional_timestamp), type_name, namespace_input - subconscious/schemas: catalog parity, all 10 functions, required input validation, field/field_req/field_opt helpers All 4261 tests pass. --------- Co-authored-by: Steven Enamakel <31011319+senamakel@users.noreply.github.com>	2026-04-16 20:42:50 -07:00
Steven EnamakelandGitHub	2bb20faa55	feat(threads): dedicated threads controller with per-thread session scoping (#590) ... * feat(conversations): implement thread management features including creation and deletion - Added functionality to create new conversation threads with unique IDs and titles based on the current date and time. - Introduced a deleteThread action to remove existing threads, updating the selected thread accordingly. - Enhanced the UI to display threads in a sidebar, allowing users to select and delete threads easily. - Updated the Conversations component to handle thread loading and selection more efficiently, ensuring a smoother user experience. Also updated the OpenHuman package version to 0.52.15 in Cargo.lock files. * refactor(conversations): rename createThreadLocal to createNewThread and streamline thread creation logic - Updated the function name from `createThreadLocal` to `createNewThread` for clarity and consistency. - Simplified the thread creation process by removing the manual ID and title generation, leveraging the new API method for automatic thread creation. - Adjusted the Conversations component to utilize the new `handleCreateNewThread` function, enhancing readability and maintainability. - Removed unused thread ID and title generation functions to clean up the codebase. This refactor improves the overall structure and clarity of the thread management functionality. * refactor(memory): remove deprecated conversation thread management functions - Eliminated unused functions related to listing, creating, updating, appending, and deleting conversation threads in memory operations. - This cleanup enhances code maintainability and reduces complexity in the memory module. - The refactor focuses on streamlining the conversation management logic, aligning with recent changes in the thread handling API. * refactor(api): update thread API method names and remove deprecated functions - Renamed thread-related API methods to align with the new naming convention, improving clarity and consistency. - Removed deprecated functions related to thread creation, streamlining the API and enhancing maintainability. - Adjusted the implementation of existing methods to reflect the updated API structure, ensuring proper functionality. * refactor(thread): simplify thread state management and remove unused properties - Streamlined the thread state by removing the lastViewedAt property and related logic, enhancing clarity in unread message counting. - Updated the BottomTabBar component to reflect the new unread count logic, which now simply returns the length of threads. - Removed the setLastViewed action from the Conversations component, further simplifying the thread management logic. - Introduced a new deleteThread action to handle thread deletion, ensuring proper state updates and selection handling. - Overall, these changes improve maintainability and reduce complexity in the thread management system. * style(threads): apply cargo fmt * refactor(tabbar): remove conversations unread badge * update(Cargo.lock): bump OpenHuman package version to 0.52.15 * test(threadApi): update RPC method names to threads namespace * refactor(conversations): update model ID for chat functionality - Changed the model ID used in the chatSend function from `agentic-v1` to `reasoning-v1`, clarifying the purpose of each model. - Added comments to explain the distinction between the reasoning model and the agentic model, enhancing code readability and maintainability.	2026-04-16 10:16:04 -07:00
CodeGhost21andGitHub	ea6895f19d	test(coverage): batch 5–8 — Rust unit tests toward 80% for 20 modules (#607) ... * test(coverage): batch 5–8 — Rust unit tests toward 80% for 20 modules (#530) Add 244 new unit tests across 20 modules to push line coverage from 75.06% → 75.70% overall. Modules improved: - api/socket: 77% → 100% - rpc/dispatch: 75% → 100% - config/schema/channels: 77% → 100% - composio/gmail/sync: 78% → 97% - composio/notion/sync: 77% → 96% - webhooks/router: 78% → 96% - agent/hooks: 78% → 92% - config/schema/proxy: 76% → 90% - local_ai/device: 79% → 89% - text_input/schemas: 73% → 89% - agent/harness/interrupt: 76% → 88% - billing/schemas: 79% → 87% - screen_intelligence/schemas: 78% → 81% - about_app/types, health/schemas, app_state/schemas, core/types, config/schema/identity_cost, cron/scheduler Tests cover: schema catalog integrity, param deserialization, serde roundtrips, helper functions, edge cases, error paths. All 3974 tests pass. * test(coverage): batch 9–10 — browser_open, update_memory_md, credentials profiles, cost tracker (#530) Add 53 new tests across 4 more modules: - browser_open: IPv4/IPv6 private ranges, host matching, normalize_domain edges - update_memory_md: empty file, section creation, unknown action, param validation - credentials/profiles: token expiry, CRUD operations, active profile management - cost/tracker: budget warnings, monthly exceeded, model stats aggregation All 4027 tests pass. * test(coverage): batch 11–12 — channels schemas, conversation store (#530) Add 33 new tests: - channels/controllers/schemas: per-function input validation, param deserialization, helper coverage - memory/conversations/store: thread lifecycle (create, delete, idempotent), multi-thread, empty/nonexistent thread, purge empty store All 4060 tests pass.	2026-04-16 10:13:33 -07:00
3 people	7f686003a0	feat(core): gate service lifecycle on user login/logout (#582) (#603) ... * refactor(voice): wrap CancellationToken in Mutex for restart support The VoiceServer singleton uses OnceCell so it can't be recreated. CancellationToken is one-shot — once cancelled, stays cancelled. Wrapping it in std::sync::Mutex + adding fresh_cancel() allows stop() then run() to work within the same process (logout → re-login). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(screen_intelligence): wrap CancellationToken in Mutex for restart support Same pattern as voice server — enables stop() then run() within the same process for logout → re-login cycles. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(credentials): add start/stop helpers for login-gated services Add start_login_gated_services() and stop_login_gated_services() to credentials/ops.rs. Wire start into store_session() (after login) and stop into clear_session() (on logout) so voice, autocomplete, screen intelligence, and local AI are properly managed around auth lifecycle. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(core): defer service startup behind login gate in jsonrpc.rs Replace the unconditional service startup block with a login check. If active_user.toml exists, start services immediately via the new start_login_gated_services() helper. Otherwise defer until the login handler triggers it. Autocomplete shutdown hook remains unconditional. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(screen_intelligence): wait for Stopped state in stop() to prevent restart race stop() now polls until the run-loop sets ServerState::Stopped (5s timeout). Prevents fast logout→login from seeing stale Idle/Running state and skipping the restart. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(dictation): add stop() to prevent listener accumulation across re-logins Store the spawned task JoinHandle and abort it on stop(). Prevents duplicate rdev hotkey listeners from stacking across logout→login cycles. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(credentials): call dictation_listener::stop() on logout Wire the new dictation stop into stop_login_gated_services() so the hotkey forwarder task is aborted when the user logs out. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>	2026-04-16 10:12:56 -07:00
Steven EnamakelandGitHub	2824850708	feat(streaming): thinking tokens, tool call deltas, and progressive channel edits (#549) ... * feat(conversations): implement live streaming for assistant responses - Added support for live streaming of assistant responses in the Conversations component, enhancing user experience during interactions. - Introduced new interfaces for handling streaming state, including `StreamingAssistantState` and related event handlers for text, thinking, and tool argument deltas. - Updated the state management to accommodate streaming data, ensuring smooth updates to the UI as new content arrives. - Enhanced the rendering logic to display provisional assistant bubbles and thinking indicators while responses are being composed. - Refactored existing event handling to integrate with the new streaming functionality, improving overall responsiveness and interactivity. This commit significantly improves the real-time interaction capabilities of the assistant, providing users with immediate feedback during conversations. * feat(channels): progressive-edit streaming for inbound channels ChannelInboundSubscriber now buffers text/tool delta events on a 1s timer and edits the outbound channel message in place, so Telegram/ Slack-style conduits can show a single evolving reply instead of a single atomic bubble at the end. Renders a "🔧 tool …" status line above the partial text and rewrites with the final canonical reply on chat_done. Falls back to atomic-final delivery if the backend's PATCH /channels/:channel/messages/:id endpoint is unavailable or repeatedly fails, so existing channels keep working while the edit endpoint is rolled out. New rest.rs helper: BackendOAuthClient::send_channel_edit. * style: cargo fmt on streaming-related files Auto-applied by cargo fmt after the streaming plumbing changes touched provider traits, the session turn loop, the channel inbound subscriber, and the compatible-provider SSE path. * feat(streaming): compact live preview + Telegram typing indicator UI: while streaming, show only the trailing ~120 chars of assistant text in a monospace ticker-tape bubble (cursor + ellipsis prefix) so the scroll position doesn't jump as tokens arrive. The full response still replaces the preview via addInferenceResponse on chat_done. Backend bridge: ChannelInboundSubscriber now fires Telegram's typing indicator as soon as an inbound message is received and refreshes it every 4 s while the turn is in flight (Telegram's sendChatAction lasts ~5 s). New BackendOAuthClient::send_channel_typing hits POST /channels/:channel/typing; the subscriber latches typing_disabled after two failures so channels without the endpoint stop getting hit. * fix(providers): fall back to JSON parse when upstream ignores stream=true Some OpenAI-compatible backends (including our e2e mock) accept `stream: true` in the request but reply with a regular `application/json` body rather than an SSE stream. The previous implementation blindly pushed the body through the SSE line parser, which produced an empty aggregated response because the body never contained `\n\n` event separators. Detect the non-SSE content-type and fall through to the existing `parse_native_response` path so the caller still gets the aggregated response. No deltas are emitted in this case — there's nothing to stream — but correctness is preserved. Unblocks the json_rpc_protocol_auth_and_agent_hello E2E test. * Enhance event handling for progressive-edit streaming in ChannelInboundSubscriber - Introduced a new `StreamingState` struct to manage the state of progressive-edit streaming, allowing for buffered text and tool deltas. - Implemented a timer-based flushing mechanism for edits, ensuring timely updates to the channel. - Refactored event handling logic to accommodate new event types (`text_delta`, `tool_call`, `tool_result`) and improved error handling for chat events. - Updated the logic for finalizing channel replies to handle both streaming and atomic delivery scenarios, enhancing overall responsiveness and user experience. This commit significantly improves the handling of real-time updates during user interactions, providing a smoother and more interactive experience. * Enhance tool call tracking in AgentProgress and parsing logic - Added a `call_id` field to `ToolCallStarted` and `ToolCallCompleted` variants in the `AgentProgress` enum to uniquely identify tool calls and link them to their respective events. - Updated the `ParsedToolCall` struct to include an optional `id` field for tool calls originating from native responses, ensuring consistent tracking across different call types. - Modified the `parse_tool_call_value` function to initialize the `id` field appropriately, enhancing the parsing logic for tool calls. - Adjusted the `run_tool_call_loop` function to utilize the new `call_id` for progress events, ensuring accurate tracking of tool execution across iterations. - Enhanced the `spawn_progress_bridge` function to handle the new `call_id` field in progress events, improving the overall event handling mechanism. These changes improve the robustness of tool call tracking and enhance the clarity of event relationships within the agent's progress reporting. * Enhance tool call tracking and event handling in Conversations component - Added a `tool_call_id` field to `ChatToolCallEvent` and `ChatToolResultEvent` interfaces for improved tracking of tool calls across events. - Updated event key generation in the Conversations component to include `tool_call_id`, ensuring unique identification of tool events. - Enhanced the logic for managing tool timelines to prevent duplication of entries by reconciling existing tool calls based on `tool_call_id`. - Improved handling of tool argument deltas and results, allowing for more accurate updates to the UI during tool execution. These changes significantly improve the robustness of tool call tracking and enhance the clarity of event relationships within the Conversations component. * Enhance tool call event handling and progress reporting in Agent - Updated the `run_tool_call_loop` function to include stable IDs for tool calls, ensuring unique identification across iterations. - Introduced early completion events for failed tool calls, improving client-side error handling and user experience. - Refactored the `emit_progress` function to use asynchronous sending, ensuring lifecycle events are not dropped due to backpressure. - Enhanced the `turn` method to await progress emissions, improving synchronization during user message processing. These changes significantly improve the robustness of tool call tracking and enhance the clarity of event relationships within the agent's progress reporting. * fix(channels): close stuck drafts, prevent duplicate post, add bridge logs Findings #3, #5, #6 from the follow-up review: channels/bus.rs - chat_done handler no longer returns early on empty reply — it now finalizes with a "(No response from agent.)" fallback so any draft we posted during streaming gets closed off instead of being left showing "_working…_" forever. - StreamingState gained `draft_sent: bool`, set whenever the initial send_channel_message succeeds (even when the backend's response didn't include an id). finalize_channel_reply now checks this flag and silently skips the "no draft → send atomic" fallback when a draft was posted but id was lost — fixes a duplicate-bubble bug where an id-less draft plus a chat_done finalize produced two messages in the user's channel. channels/providers/web.rs - spawn_progress_bridge now logs a scoped entry message on startup (client_id/thread_id/request_id), a per-variant trace/debug line on each AgentProgress event (with call_id/iteration correlation), and an exit message with final round + events_seen count. SubagentFailed is logged at warn level for visibility.	2026-04-13 20:38:00 -07:00
Steven EnamakelandGitHub	8cbb425cea	feat(onboarding): fire welcome agent immediately on completion (#548) ... * Implement proactive welcome agent for onboarding experience - Added a new module `welcome_proactive` to handle immediate welcome messages after user onboarding completion, enhancing user engagement. - Updated `set_onboarding_completed` to trigger the proactive welcome agent upon onboarding status change, ensuring timely delivery of welcome messages. - Refactored the onboarding process to streamline the integration of proactive messaging, improving overall user experience. - Enhanced documentation and logging for better observability of the onboarding flow and proactive message handling. - Introduced tests to validate the functionality of the new welcome agent and its integration into the onboarding process. * fix(socketio): auto-join "system" room so proactive messages reach clients The proactive message subscriber emits with `client_id="system"`, but connected Socket.IO clients only auto-joined a room named after their own sid — so the welcome agent's message was published into an empty room and never reached any frontend. Adds `socket.join("system")` at connect time alongside the existing per-sid join, so every client now receives broadcast proactive events (welcome agent, morning briefing, cron announcements). Also adds scripts/test-proactive-welcome.sh — end-to-end smoke test that resets onboarding flags, spawns a fresh core on port 7789, connects a Socket.IO listener, fires the RPC, and reports pass/miss for every checkpoint in the pipeline including client-side delivery. * fix(welcome): PR review — tighter gating, legacy-cron prune, safer harness - Gate proactive welcome spawn on `!was_chat_completed` so a user whose chat flow already completed (legacy tool path or manual flip) doesn't get a second welcome. - Prune legacy `welcome` cron jobs in `seed_proactive_agents`: one- shot entries left behind by an interrupted earlier run would otherwise double-deliver once the scheduler picks them up. Added a unit test covering the prune + morning-briefing seed in the same call. - Add a post-publish debug log in `welcome_proactive::run_proactive_welcome` so "reached end successfully" is distinguishable from "silently bailed above" by reading the log alone. - Replace ignored `socket.join(...)` results with a helper that logs success + failure per room + client id. Silent join failures on the "system" room make proactive delivery vanish without a trace. - Harden the test harness: * Back up CONFIG_PATH before mutating and restore on exit so the developer's staging profile is never permanently changed (unless `--keep-flags` is passed). * Tolerate inline comments + trailing whitespace in the TOML rewrite; append-at-end instead of prepend when a key is missing, so the rewrite can't accidentally land inside a section header. Not applied: - Unit tests for `run_proactive_welcome` empty-response / serialization- failure paths: empty response is already `anyhow::bail!`-guarded, and the serde_json failure branch is unreachable given a `json!`-built value. Testing either requires mocking `Agent::from_config_for_agent` which has heavy registry/provider dependencies — cost > value.	2026-04-13 19:43:24 -07:00
Steven EnamakelandGitHub	8057f2283c	feat: onboarding Gmail integration + LinkedIn profile enrichment (#524) ... * refactor(composio): restructure toolkit metadata handling and onboarding steps - Removed the old `toolkitMeta.ts` file and replaced it with a new `toolkitMeta.tsx` file that includes updated metadata handling for Composio toolkits, enhancing the integration with React components. - Updated the `ComposioConnectModal` to directly render icons without additional markup, streamlining the component structure. - Modified the `Skills` page to utilize the new icon rendering method, improving consistency across the application. - Enhanced the onboarding process by introducing a new `ContextGatheringStep` component, which gathers user context from connected integrations, improving the onboarding experience. - Updated the `SkillsStep` to reflect changes in toolkit connection handling and display, ensuring a smoother user interaction during onboarding. * feat(apify): introduce Apify integration tools for actor execution and status retrieval - Added new tools for running Apify actors and fetching their run statuses, enhancing automation capabilities. - Updated the integration schema to include an `apify` toggle for user configuration, allowing for flexible integration management. - Enhanced the onboarding experience by modifying the SkillsStep to focus on Gmail integration, streamlining user interactions. - Improved documentation and comments for clarity on the new Apify functionalities and their usage. * feat(learning): add LinkedIn enrichment module and schemas - Introduced a new `linkedin_enrichment` module for enriching user profiles by scraping LinkedIn data from Gmail. - Implemented the `run_linkedin_enrichment` function to handle the enrichment pipeline, including Gmail search, scraping via Apify, and data persistence. - Added controller schemas for the learning domain, enabling integration with the existing controller framework. - Updated the `all.rs` file to register the new learning controllers and schemas, enhancing the overall functionality of the learning system. * feat(linkedin): implement PROFILE.md generation for LinkedIn enrichment - Added functionality to generate a PROFILE.md file from scraped LinkedIn data, summarizing user profiles for agent context. - Updated the `run_linkedin_enrichment` function to write PROFILE.md to the workspace, enhancing data persistence. - Introduced helper functions for rendering and summarizing LinkedIn profiles, improving the overall enrichment process. - Ensured minimal PROFILE.md creation even when scraping fails, maintaining essential user context. * feat(onboarding): enhance ContextGatheringStep for LinkedIn enrichment pipeline - Updated the ContextGatheringStep to integrate a new pipeline for LinkedIn enrichment, replacing the previous Gmail profile fetching stages. - Implemented a progress animation and logging for the enrichment process, improving user feedback during data retrieval. - Refactored stage definitions to align with the new pipeline structure, enhancing clarity and maintainability. - Introduced error handling and status updates for each stage of the enrichment process, ensuring robust user experience. * feat(instructions): refactor tool instruction generation for clarity and flexibility - Introduced helper functions `tool_instructions_preamble` and `append_tool_entry` to streamline the construction of tool instructions. - Updated `build_tool_instructions` to utilize the new helper functions, improving code readability and maintainability. - Added `build_tool_instructions_filtered` to allow for generating instructions from a filtered list of tools, enhancing flexibility in tool usage. - Adjusted the startup process to use the filtered instructions, ensuring only relevant tools are included in the system prompt. * refactor(toolkitMeta): simplify component structure and improve readability - Refactored the `BrandIcon` component to streamline its props definition, enhancing code clarity. - Consolidated SVG path definitions in various icons for better readability and maintainability. - Updated the `ContextGatheringStep` to simplify the RPC call syntax, improving code conciseness. - Enhanced logging in the LinkedIn enrichment process for clearer tracking of Gmail searches and scraping stages. * fix: address PR review — USER.md→PROFILE.md consistency, Composio tool filtering, and quality fixes - Replace USER.md with PROFILE.md across all prompt paths: channels_prompt.rs, subconscious/prompt.rs, workspace/ops.rs bootstrap, and channel tests - Remove Composio tool description from main agent system prompt (tool_descs) so skills_agent is the only agent that sees Skill-category tools - Add "learning" namespace description for CLI help discovery - Fix react-hooks/set-state-in-effect: wrap synchronous setState in queueMicrotask in ContextGatheringStep - Derive SkillsStep displayToolkits from backend allowlist with error/retry UI - Add KNOWN_COMPOSIO_TOOLKITS alternate slug variants (google_calendar, etc.) - Add namespaced debug logging to onboarding handlers and pipeline - Deduplicate MemoryClient creation in linkedin_enrichment persist functions * fix: use setTimeout instead of queueMicrotask for ESLint compatibility * refactor(onboarding): streamline debug logging in handleContextNext function - Consolidated debug logging in the handleContextNext function to improve clarity and reduce verbosity. - Removed unnecessary line breaks for a more concise code structure. * refactor(linkedin): enhance enrichment pipeline with structured stage results - Introduced a new `EnrichmentStage` struct to capture detailed results for each stage of the LinkedIn enrichment process. - Updated the `LinkedInEnrichmentResult` to include a vector of stages, allowing for structured reporting of success, failure, and skipped stages. - Improved error handling and logging throughout the enrichment pipeline, ensuring better traceability of issues during execution. - Adjusted the API response to include stage results, enhancing the frontend's ability to display detailed enrichment outcomes. * chore(workflows): update macOS E2E test configuration and comment out Linux E2E job - Modified the description and default values in the macOS E2E test input options for clarity. - Commented out the entire Linux E2E job configuration to prevent execution while maintaining the setup for future use.	2026-04-13 14:16:53 -07:00
Steven EnamakelandGitHub	ae9ac30db6	feat: add proactive welcome onboarding flow (#522) ... * feat: add proactive agents for onboarding experience - Introduced two new proactive agents: "Morning Briefing" and "Welcome Message". - The Morning Briefing agent provides a daily summary of the user's calendar, tasks, and relevant context, scheduled to run automatically at 7 AM. - The Welcome Message agent delivers a personalized greeting after onboarding, utilizing user information for a tailored experience. - Updated the cron job system to seed these agents upon onboarding completion, enhancing user engagement from the start. - Added comprehensive tests to ensure the correct functionality of the new agents and their integration into the system. * feat: implement proactive message handling for user engagement - Added a new `ProactiveMessageSubscriber` to route proactive messages (e.g., morning briefings, welcome messages) to the user's active channel, enhancing user engagement. - Introduced `ProactiveMessageRequested` event in the event bus to facilitate proactive message delivery. - Updated the configuration schema to include a preferred channel for proactive messages, allowing for tailored user experiences. - Seeded default proactive agent cron jobs to ensure timely delivery of messages post-onboarding. - Enhanced the startup process to register the proactive message subscriber, ensuring proactive messages are consistently routed. - Comprehensive tests added to validate the functionality of proactive message handling and delivery mechanisms. * feat: enhance welcome agent functionality and onboarding process - Updated the welcome agent's description to clarify its role as the first point of contact for new users, focusing on guiding them through remaining onboarding steps. - Increased the maximum iterations for the agent from 4 to 6 to allow for more complex interactions. - Introduced a new `complete_onboarding` tool to check the user's setup status and mark onboarding as complete, improving the onboarding experience. - Enhanced the prompt structure to provide clearer guidance on the agent's workflow, including steps for checking setup status, greeting users, and completing onboarding. - Updated the tools list to include `complete_onboarding`, ensuring the welcome agent can effectively manage user onboarding. - Comprehensive documentation added to clarify the agent's functionality and improve user engagement. * feat: enhance welcome agent functionality and onboarding process - Updated the welcome agent's description to clarify its role as the first point of contact for new users, focusing on guiding them through setup and onboarding. - Increased the maximum iterations for the agent from 4 to 6 to allow for more comprehensive interactions. - Introduced a new `complete_onboarding` tool to check the user's setup status and mark onboarding as complete, improving the onboarding experience. - Enhanced the prompt structure to provide clearer guidance on the agent's workflow, including steps for checking setup status, greeting users, and completing onboarding. - Updated the tools list to include the new `complete_onboarding` tool, ensuring it is available for the welcome agent's operations. - Improved documentation for the welcome agent's prompt, emphasizing tone guidelines and what not to do during user interactions. * style: apply rustfmt to onboarding flow changes * refactor: streamline proactive message subscriber registration and enhance onboarding agent tests - Replaced the direct registration of the proactive message subscriber with a new `register_web_only_proactive_subscriber` function, ensuring it is only registered once during domain startup. - Updated the welcome agent test to reflect the addition of the `complete_onboarding` tool, verifying its presence and adjusting the expected number of tools. - Enhanced the onboarding process by improving logging for proactive agent seeding transitions, providing clearer insights during onboarding state changes. - Refactored the cron job system to support agent definitions, allowing for more flexible job scheduling and execution based on defined agent parameters. - Improved documentation and comments across the affected modules to clarify the purpose and functionality of changes made. * refactor: improve proactive subscriber registration and logging - Streamlined the registration process for the web-only proactive message subscriber, enhancing clarity and maintainability. - Consolidated logging statements in the onboarding completion function for improved readability and consistency. - These changes aim to enhance the overall structure and logging clarity within the proactive messaging and onboarding processes.	2026-04-12 19:15:42 -07:00
Steven EnamakelandGitHub	a115758c6c	Improve local voice readiness and add Composio trigger history (#516) ... * feat: enhance local AI asset state management and error handling - Updated the Home component to improve readiness checks for local AI assets, ensuring a more robust UI experience. - Refactored the LocalAiService to provide clearer state management for STT and TTS models, including handling on-demand downloads and error logging. - Enhanced the voice status function to accurately reflect the availability of transcription backends, improving overall system reliability. - Introduced warnings for on-demand model downloads to inform users about potential delays in functionality. * feat: implement polling for voice server status in OverlayApp - Added a polling mechanism to periodically check the voice server status every 2 seconds, ensuring the overlay remains in sync with the server state. - Introduced logic to activate or dismiss the speech-to-text (STT) mode based on the server's recording or transcribing state, enhancing user experience and responsiveness. - Utilized the existing `callCoreRpc` function to fetch server status, improving reliability in state management during brief disconnects or reconnections. * feat: refine prompt handling in LocalAiService - Updated the prompt construction logic in `LocalAiService` to enhance clarity and functionality. - When `no_think` is set, the system prompt now includes a directive for the model to respond with only the final answer, improving response accuracy. - Refactored the prompt and system parameters to ensure they are correctly formatted and passed to the `OllamaGenerateRequest`, enhancing overall request handling. * feat: enhance STT readiness checks in VoicePanel and useVoiceSkillStatus - Updated the STT readiness logic in `VoicePanel` to account for both 'ready' and 'ondemand' states, improving the accuracy of readiness assessments. - Refined the `useVoiceSkillStatus` hook to ensure it only blocks when the local AI's STT state is explicitly 'missing', enhancing overall system reliability and user experience. * feat(composio): implement ComposeIO trigger history component and hook - Added `ComposeioTriggerHistory` component to display a list of ComposeIO trigger events with formatted timestamps and payloads. - Introduced `useComposeioTriggerHistory` hook to manage fetching and state of trigger history entries, including error handling and loading states. - Updated `Webhooks` page to integrate the new component and hook, replacing previous webhook activity display with ComposeIO trigger history. - Created utility functions for formatting timestamps and payloads for better readability in the UI. - Established backend support for fetching trigger history through new Tauri commands, ensuring robust data handling and storage. * style: apply repo formatting fixes * feat: add fs2 dependency and enhance ComposeIO trigger history handling - Introduced the `fs2` crate to manage file locking, improving the reliability of file operations in the ComposeIO trigger history. - Updated `ComposioTriggerHistoryStore` to utilize exclusive file locks during archive writing, ensuring data integrity. - Enhanced error handling for file operations, providing clearer logging for failures related to file access and locking. - Refactored the initialization logic for global trigger history to prevent duplicate setups, improving overall stability.	2026-04-12 16:43:33 -07:00
Steven EnamakelandGitHub	ef9c117e9e	test: expand agent harness coverage (#513) ... * feat(tests): add comprehensive unit tests for hooks and memory context - Introduced a suite of tests for the `fire_hooks` function, ensuring that all hooks are dispatched even if one fails, enhancing reliability in the hook execution flow. - Added tests for the `sanitize_tool_output` function to validate the correct mapping of success and failure messages for various tool outputs. - Implemented a mock memory structure to facilitate testing of memory context building, ensuring that working memory is prioritized and deduplication occurs correctly. - Enhanced the `build_memory_context` function tests to verify filtering and truncation of memory entries, improving the robustness of memory management in the system. - Overall, these tests aim to strengthen the codebase by ensuring that critical functionalities related to hooks and memory context are thoroughly validated. * feat(tests): enhance unit tests for provider alias resolution and prompt options - Updated the `provider_alias_and_route_selection_round_trip` test to dynamically resolve the first provider from the registry, ensuring accurate alias resolution. - Added new tests for `DumpPromptOptions` and `ComposioStubTools`, validating default settings and expected tool names. - Introduced tests for rendering main agent dumps, ensuring tool instructions and skill counts are correctly included in the output. - Enhanced prompt handling tests to cover cache boundary extraction and subagent render options, improving overall test coverage and reliability. * feat(tests): enhance error handling and formatting in unit tests - Added new tests for `AgentError` variants to validate string formatting and error recovery from `anyhow`. - Improved formatting consistency in existing tests for better readability. - Enhanced the `sanitize_tool_output` test to ensure accurate mapping of success and failure messages. - Updated memory loader tests to enforce minimum limits and budget constraints, ensuring robust memory management. - Overall, these changes aim to strengthen the test suite by improving coverage and clarity in error handling scenarios. * feat(tests): add unit tests for tool filtering and subagent dump rendering - Introduced new tests to validate the filtering of tools based on named scopes and disallowed tools, ensuring correct tool selection in debug dumps. - Added tests for rendering subagent dumps, including handling file prompt fallbacks and missing files without panicking. - Enhanced the workspace prompt handling to prefer custom prompt locations, improving the flexibility and reliability of agent prompt rendering. - Overall, these additions strengthen the test suite by covering critical functionalities related to tool filtering and prompt rendering. * feat(tests): add comprehensive unit tests for memory loader and multimodal helpers - Introduced new tests for the memory loader to validate behavior when the header exceeds budget and when recall fails, ensuring robust memory management. - Added tests for multimodal helpers, covering image marker counting, payload extraction, and MIME type normalization, enhancing the reliability of multimodal interactions. - Overall, these additions strengthen the test suite by improving coverage and ensuring correct functionality in memory handling and multimodal processing. * feat(tests): add unit tests for tool execution and agent behavior - Introduced new tests for the `run_tool_call_loop` function, validating the rejection of vision markers for non-vision providers and ensuring correct streaming of final text chunks. - Added tests to verify that CLI-only tools are blocked in prompt mode and that native tool results are persisted as tool messages. - Enhanced the `Agent` class with tests for error handling and event publishing during single runs, ensuring robust agent behavior in various scenarios. - Overall, these additions strengthen the test suite by improving coverage and reliability in tool execution and agent interactions. * feat(tests): enhance tool execution and agent behavior tests - Added new tests for the `run_tool_call_loop` function, including scenarios for auto-approving supervised tools on non-CLI channels and handling unknown tools with default max iterations. - Introduced `ErrorResultTool` and `FailingTool` to simulate error conditions during tool execution, improving coverage of error handling in the agent. - Updated the `ScriptedProvider` to return results wrapped in `anyhow::Result`, ensuring consistent error handling across test cases. - Overall, these enhancements strengthen the test suite by validating tool execution paths and agent behavior under various conditions. * refactor(tests): improve test readability and structure - Enhanced formatting in various test cases for better clarity and consistency, including the use of line breaks and indentation. - Updated assertions to improve readability by aligning them with Rust's idiomatic style. - Overall, these changes aim to strengthen the test suite by making it more maintainable and easier to understand. * docs(agent): enhance module documentation for agent domain - Added comprehensive documentation to the `mod.rs` file, detailing the agent domain's purpose, key components, and their functionalities. - Improved clarity on how LLMs interact with the system, manage conversation history, and handle autonomous behaviors. - This update aims to enhance understanding and maintainability of the agent domain within the OpenHuman project. * docs(agent): improve documentation and formatting across multiple files - Enhanced comments and documentation in `dispatcher.rs`, `error.rs`, `hooks.rs`, and `harness/mod.rs` for better clarity and consistency. - Adjusted formatting in the `TurnContext` and `ToolCallRecord` structs to improve readability and understanding of their purpose and functionality. - Overall, these changes aim to strengthen the documentation and maintainability of the agent domain within the OpenHuman project. * feat(tests): add comprehensive tests for memory loader and agent behavior - Introduced new tests for the `DefaultMemoryLoader`, validating error propagation during primary recall failures and ensuring correct context emission when working memory is present. - Added tests to verify behavior when the working memory section exceeds budget constraints, enhancing memory management robustness. - Overall, these additions strengthen the test suite by improving coverage and reliability in memory handling and agent interactions. * refactor(tests): improve formatting and readability in memory loader tests - Reformatted the `load_context` calls in memory loader tests for better readability by chaining method calls. - Enhanced documentation comments in the `pformat.rs` file to clarify the purpose and functionality of functions. - Improved consistency in spacing and formatting across various sections of the codebase, including the `interrupt.rs` and `builder.rs` files. - Overall, these changes aim to enhance code clarity and maintainability within the test suite and related modules. * feat(tests): add new tests for self-healing interceptor and local AI provider - Introduced tests to validate the detection of missing commands in the `SelfHealingInterceptor`, ensuring proper handling of recognized and unrecognized patterns. - Added tests for the `ensure_polyfill_dir` method to confirm directory creation and path exposure. - Enhanced local AI provider tests to verify correct behavior when the service is ready and the appropriate tier is set, as well as ensuring local metadata is utilized during provider resolution. - Overall, these additions strengthen the test suite by improving coverage and reliability in self-healing and local AI functionalities. * refactor(tests): enhance test structure and external ID handling in escalation tests - Updated the `envelope` function to accept an `external_id` parameter, improving flexibility in test scenarios. - Modified various test cases to utilize specific external IDs, enhancing clarity and ensuring accurate event assertions. - Introduced a mutex lock in local AI tests to prevent race conditions, ensuring reliable test execution. - Overall, these changes improve the robustness and maintainability of the test suite for escalation and local AI functionalities. * refactor(tests): remove redundant test modules and improve test organization - Eliminated unused test modules from `hooks.rs`, `memory_loader.rs`, `fork_context.rs`, `interrupt.rs`, and `parse.rs` to streamline the codebase. - Enhanced overall test organization by consolidating relevant tests into appropriate files, improving maintainability and readability. - These changes aim to simplify the test structure and focus on active test cases, ensuring a cleaner and more efficient testing environment. * refactor(tests): improve test formatting and readability - Reformatted assertions in multiple test cases for better readability by aligning them with Rust's idiomatic style. - Enhanced the structure of test cases by using line breaks and consistent indentation, improving overall clarity. - These changes aim to strengthen the test suite by making it more maintainable and easier to understand. * refactor(api): improve string containment check and formatting in transcript handling - Updated the `key_bytes_from_string` function to use a more concise containment check for special characters. - Changed the message writing in the `write_transcript` function to utilize `writeln!` for better formatting. - Enhanced the condition in `latest_in_dir` to use `is_none_or` for improved readability and clarity. - These changes aim to streamline code and enhance maintainability across the API and transcript handling modules. * refactor(code): streamline function implementations and improve readability - Removed unnecessary whitespace in `run_voice_server_command` for cleaner code. - Simplified directory search logic in `bundled_openclaw_prompts_dir` by using `find` instead of a loop. - Updated `request_accessibility_access` to use direct references for keys and values, enhancing clarity. - Improved documentation formatting in `mod.rs` and `types.rs` for better consistency. - Refactored `Config` initialization in `load.rs` to use struct update syntax for clarity. - Added `#[allow(clippy::too_many_arguments)]` annotations in multiple functions to address linter warnings. - Enhanced type definitions and function signatures for better type safety and readability in various modules. - Overall, these changes aim to improve code maintainability and readability across the project. * chore(ci): update typecheck workflow and pre-push hooks to include clippy checks - Modified the GitHub Actions workflow to run clippy with warnings treated as errors for the `openhuman` package. - Enhanced the pre-push hook to include clippy checks, ensuring code quality before pushing changes. - Updated package.json to define a new script for running clippy, integrating it into the format check process. - These changes aim to improve code quality and maintainability by enforcing stricter linting rules. * refactor(api): simplify condition in key_bytes_from_string function - Streamlined the condition in the `key_bytes_from_string` function to improve readability by consolidating the if statement into a single line. - This change enhances code clarity while maintaining the original functionality of the key validation process. * chore(package): update format:check script to remove clippy integration - Modified the `format:check` script in `package.json` to exclude the clippy check, streamlining the formatting process. - This change simplifies the formatting workflow while maintaining the integrity of Rust formatting checks. * refactor(core): enhance documentation and structure in core modules - Improved documentation across various core functions, including `build_registered_controllers`, `run_from_cli_args`, and `dispatch`, to clarify their purpose and usage. - Streamlined comments to provide clearer guidance on the flow of operations and error handling. - Enhanced the structure of the `EventBus` and `NativeRegistry` to improve readability and maintainability. - Overall, these changes aim to improve code clarity and facilitate easier navigation and understanding of the core components. * refactor(core): reorganize imports in engine.rs for clarity - Adjusted the import statements in `engine.rs` to improve organization and readability. - Moved the macOS-specific import of `validate_focused_target` to a more appropriate location and ensured consistent ordering of imports. - These changes aim to enhance code clarity and maintainability within the core module. * refactor(core): enhance WebChannelEvent structure and documentation - Introduced a new `WebChannelEvent` struct to standardize event payloads for chat-related activities, including fields for event name, client ID, thread ID, request ID, and optional response details. - Improved documentation for the `attach_socketio` function, clarifying its role in setting up Socket.IO event handlers and the associated chat logic. - Removed unused structs and streamlined the event handling process to improve code clarity and maintainability across the core module. * refactor(core): streamline Socket.IO event handlers for clarity and consistency - Refactored the Socket.IO event handlers in `attach_socketio` to improve readability by standardizing the formatting and structure of the code. - Enhanced the organization of the event handling logic for `rpc:request`, `chat:start`, and `chat:cancel` events, making it easier to follow the flow of operations. - These changes aim to improve code maintainability and facilitate easier navigation within the Socket.IO integration. * feat(core): add new structs for Socket RPC and chat events - Introduced `SocketRpcRequest`, `ChatStartPayload`, and `ChatCancelPayload` structs to facilitate handling of Socket.IO events related to chat functionality. - These additions enhance the structure and clarity of the event payloads, improving the maintainability of the Socket.IO integration. * refactor(core): remove unused json_type_name function from socketio.rs - Eliminated the `json_type_name` function from `socketio.rs` as it was not utilized in the current codebase. - This change helps to clean up the code and improve maintainability by removing unnecessary functions. * chore(ci): update clippy command in typecheck workflow - Modified the clippy command in the GitHub Actions workflow to remove the `-D warnings` flag for the `openhuman` package, allowing warnings to be displayed without failing the build. - This change aims to improve the development experience by providing more flexibility during code analysis while still encouraging code quality. * chore(husky): remove clippy check from pre-push hook - Eliminated the clippy command from the pre-push hook to streamline the pre-push checks. - Updated the failure message to reflect the removal of clippy, focusing on format, lint, TypeScript, and Rust errors only. - This change simplifies the pre-push process while maintaining essential checks for code quality. * refactor(tests): add macOS-specific imports for enhanced test coverage - Introduced conditional imports for macOS in the tests module of `engine.rs` to support platform-specific functionality. - This change improves the test setup for macOS environments, ensuring compatibility and enhancing overall test coverage. * refactor(tests): update tool call execution in test cases - Modified the `execute_tool_call` method calls in multiple test cases to include a second parameter, improving the accuracy of the tests. - This change ensures that the tests reflect the latest method signature and enhances the reliability of the test outcomes.	2026-04-12 13:30:24 -07:00
Steven EnamakelandGitHub	1cb70e3234	feat: P-Format tool calls, user memory injection, planner upgrades (#511) ... * feat(agent): introduce debug agent prompts script and CLI for prompt inspection - Added `scripts/debug-agent-prompts.sh` to dump system prompts for built-in agents, facilitating prompt engineering reviews. - Implemented `openhuman agent` CLI commands for inspecting agent definitions and rendering prompts. - Updated `.gitignore` to exclude local prompt dumps and added version bump for `openhuman` to 0.52.5. - Introduced `debug_dump` module for shared prompt rendering logic, enhancing debugging capabilities across the application. * feat(pformat): introduce P-Format tool calls for efficient parameter handling - Added a new module `pformat` to implement P-Format ("Parameter-Format") tool calls, which significantly reduce token usage in tool invocations. - Updated the agent module to include the new `pformat` module. - Enhanced the context management by integrating user memory summaries from the tree summarizer, optimizing the prompt rendering process. - Introduced constants for user memory character limits to ensure efficient memory usage across namespaces. - Implemented a new `UserMemorySection` in the prompt to display distilled long-term context, improving the agent's ability to leverage learned information during interactions. - Refactored the tree summarizer store to collect root-level summaries with character caps, ensuring manageable context sizes in prompts. * feat(dispatch): implement PFormatToolDispatcher for efficient tool call handling - Introduced `PFormatToolDispatcher`, a new dispatcher that utilizes P-Format for compact tool call syntax, significantly reducing token usage during interactions. - Updated the agent's session builder to support the new dispatcher, allowing for a seamless transition to P-Format as the default for text-based providers. - Enhanced the prompt rendering process to accommodate P-Format, ensuring backward compatibility with existing JSON tool calls. - Modified the context management to reflect the new tool call format, improving the overall efficiency of prompt generation and tool interaction. - Added tests to validate the correct rendering of tool signatures in P-Format, ensuring that the new format is correctly integrated into the system. * feat(tests): add comprehensive tests for PFormatToolDispatcher functionality - Introduced multiple tests for the `PFormatToolDispatcher`, validating its ability to parse tool calls from both P-Format and JSON formats. - Implemented tests to ensure correct handling of multiple tool call tags and the dispatcher’s fallback behavior to JSON when P-Format is ignored. - Enhanced the `render_main_agent_dump` function to include the dispatcher instructions, ensuring accurate context for tool usage in debug outputs. - Updated the tree summarizer tests to verify namespace filtering and summary collection with respect to character limits, improving overall test coverage and reliability. * feat(prompt): enhance tool signature rendering with P-Format support - Introduced a new function `render_pformat_signature_for_box_tool` to generate P-Format signatures for tools, improving consistency in tool call formatting. - Updated the `render_subagent_system_prompt` function to utilize the new P-Format signatures, ensuring alignment with the main agent's tool call protocol. - Enhanced documentation to clarify the use of P-Format in tool calls, including detailed instructions for argument handling and formatting within the tool use protocol. * feat(planner): update agent configuration and enhance planning context - Revised the `when_to_use` description to emphasize the agent's ability to gather real context through memory recall and web searches. - Increased `max_iterations` from 5 to 8 to allow for more complex planning scenarios. - Changed `omit_memory_context` to false, enabling the agent to utilize memory context during planning. - Updated the tools list to include `memory_recall`, `memory_store`, `memory_forget`, and `web_search_tool`, enhancing the agent's capabilities for context gathering. - Expanded the prompt documentation to instruct users on gathering context before planning, ensuring more informed and effective task decomposition. * style: apply cargo fmt to new and modified files * feat(debug): enhance error handling and logging in debug-agent-prompts and tool dispatching - Added error handling for missing output directory in `debug-agent-prompts.sh`, ensuring clearer feedback for users. - Introduced a verbose flag in `agent_cli.rs` to control logging output, improving usability during debugging. - Refactored tool response parsing in `dispatcher.rs` to prefer p-format calls while maintaining compatibility with JSON, enhancing response handling. - Updated session builder in `builder.rs` to finalize dispatcher selection after tool list preparation, ensuring accurate tool handling. - Improved debug dump functionality in `debug_dump.rs` to clarify the context of the dump process, ensuring better understanding of the output. - Enhanced prompt rendering functions to support explicit tool call formats, improving flexibility in subagent interactions. * feat(cli, prompt): enhance logging and tool call format handling - Added debug logging for the agent subcommand in `cli.rs`, improving traceability during command execution. - Updated the `render_subagent_system_prompt_with_format` function in `prompt.rs` to support multiple tool call formats (P-Format, JSON, Native), ensuring consistent output and flexibility for subagent interactions. - Refactored tool call protocol documentation to clarify usage across different formats, enhancing user understanding and implementation. * fix: address PR review — 8 findings 1. scripts/debug-agent-prompts.sh: validate --out argument exists 2. agent_cli.rs: silence logger in run_list before Config::load_or_init 3. dispatcher.rs: per-tag p-format/JSON selection (no all-or-nothing) 4. builder.rs: move pformat_registry build after orchestrator tools 5. debug_dump.rs: fall back to bundled prompt location for File sources 6. store.rs: fix char/byte mixing in total_cap computations 7. prompt.rs: thread ToolCallFormat into render_subagent_system_prompt 8. cli.rs: add debug trace on agent dispatch + lower dispatcher log	2026-04-12 00:08:36 -07:00
Steven EnamakelandGitHub	4c195e984a	feat(composio): native Rust providers for per-toolkit profile, sync, triggers (#509) ... * chore(composio): update Composio version and enhance periodic sync functionality - Bump Composio version to 0.52.5 in Cargo.lock. - Introduce periodic synchronization for Composio connections, allowing for regular updates based on defined intervals. - Register both Composio trigger and connection created subscribers to handle events effectively. - Implement a new periodic sync scheduler that manages sync operations for active connections, improving data freshness and integration reliability. - Enhance Composio provider implementations to support user profile fetching and sync operations, ensuring seamless integration with the backend. * fix(composio): address review feedback on provider system - bus: poll list_connections with exponential backoff (500ms→4s, 60s cap) until the connection is observed in ACTIVE/CONNECTED before firing on_connection_created, replacing the fixed 2s sleep that raced the OAuth handoff. - ops: parse_sync_reason now returns Result and rejects unrecognized reason strings instead of silently coercing them to Manual; new unit tests cover the happy path and rejection cases. - periodic: add a static OnceLock guard inside start_periodic_sync so the scheduler is only spawned once even when both startup paths call into it. - gmail/providers: redact PII in tracing output — log has_email + email_domain instead of the raw address, and drop display_name + email from the default on_connection_created log. - notion: align the comment with the actual filter (recently-edited pages, databases intentionally excluded for snapshot size reasons). * feat(composio): enhance sync handling and periodic scheduler - Introduced `record_sync_success` function to track successful syncs for connections, preventing redundant re-firing by the periodic scheduler. - Updated `ComposioTriggerSubscriber` and `ComposioConnectionCreatedSubscriber` to utilize the new sync recording mechanism. - Refactored periodic sync logic to share a process-wide map for last successful sync timestamps, improving synchronization efficiency across event-driven and periodic paths. - Adjusted imports in `mod.rs` to include the new sync recording function. * fix(notion): improve error handling in snapshot persistence - Updated the error handling for the `persist_snapshot` function to enhance clarity in error messages, ensuring that failures are logged with specific context for easier debugging.	2026-04-11 23:48:31 -07:00
Steven EnamakelandGitHub	658a75b39e	feat(triage): reusable trigger triage agent (commit 1 of 3) (#510) ... * feat(triage): implement trigger triage agent for external event classification - Introduced a new `trigger_triage` agent to classify incoming external triggers (e.g., Composio webhooks, cron jobs) and determine appropriate actions (drop, acknowledge, react, escalate). - Added `TriggerEvaluated`, `TriggerEscalated`, and `TriggerEscalationFailed` events to the event bus for tracking triage decisions. - Implemented a `TriggerEnvelope` structure to standardize the input for the triage pipeline, ensuring source-agnostic handling of triggers. - Developed a decision parser to interpret classifier outputs and enforce response contracts. - Enhanced the event bus with functions to publish triage-related events, improving observability and logging. - Updated the Cargo.lock to version 0.52.5 to reflect the new changes. * feat(triage): implement trigger triage agent for external event classification - Introduced a new `trigger_triage` agent to classify incoming external triggers (e.g., Composio webhooks, cron jobs) and determine appropriate actions (drop, acknowledge, react, escalate). - Added `TriggerEvaluated`, `TriggerEscalated`, and `TriggerEscalationFailed` events to the event bus for tracking triage decisions. - Implemented a structured decision parser and a triage pipeline that processes triggers and applies decisions based on the classifier's output. - Enhanced the Composio integration to utilize the new triage functionality, ensuring seamless handling of incoming events. - Updated tests to validate the new triage logic and ensure correct behavior across various scenarios. * feat(triage): enhance triage evaluation with retry logic and error handling - Introduced a configuration-based approach to resolve providers for triage turns, allowing for retries on local failures by falling back to remote providers. - Added a `TurnOutcomeFailure` struct to encapsulate recoverable errors during triage processing, enabling clearer error reporting and decision-making for retries. - Implemented caching for provider decisions to optimize performance and reduce redundant configuration loads. - Enhanced the `run_triage` function to handle local and remote execution paths, improving resilience against transient failures. - Updated the `resolve_provider` function to support both local and remote provider resolution based on configuration settings, ensuring flexibility in triage operations. - Improved logging for error scenarios to aid in debugging and monitoring triage outcomes. * refactor(triage): improve error handling and code formatting in evaluator and routing - Enhanced error handling in the `run_triage` function to improve clarity when both local and remote executions fail. - Refactored code formatting in the `cache_snapshot` and `decide_fresh` functions for better readability. - Updated test assertions for improved clarity and consistency in formatting, ensuring better maintainability. * feat(triage): local routing + real subagent escalation + trigger_reactor agent (commit 2 of 3) - Rewrite routing.rs with local-vs-remote provider resolution: LocalAiService adapter wrapping Ollama behind the Provider trait, ModelTier floor check (>= Ram4To8Gb), 60s decision cache with Local/Remote/Degraded states, and mark_degraded() for the retry path. - Add parse-failure / local-failure → remote retry in evaluator.rs: TurnOutcomeFailure wrapper distinguishes local vs remote errors; run_triage retries once on remote when used_local=true and marks the routing cache Degraded so subsequent triggers skip local. - Wire real escalation in escalation.rs via run_subagent: builds a full Agent from config, extracts a ParentExecutionContext from its accessors, installs it via with_parent_context, and calls run_subagent for react (trigger_reactor) / escalate (orchestrator). - New trigger_reactor built-in agent (agents/trigger_reactor/): narrow tool scope (memory_recall, memory_store, read_workspace_state, spawn_subagent), agentic model hint, max 6 iterations. - 33 triage tests passing (routing cache, tier scoring, retry path with stateful stub, local parse failure eligibility). * feat(triage): add triage_evaluate schema and handler for trigger evaluation - Introduced a new controller schema for `triage_evaluate`, enabling the trigger-triage classifier to process synthetic trigger payloads for testing and replay. - Implemented the corresponding handler to manage the evaluation process, returning parsed decisions and timing metadata. - Enhanced the input structure to include required and optional fields for comprehensive trigger evaluation, including support for dry runs. * feat(triage): expose RPC surface + default-on (commit 3 of 3) - Add `openhuman.agent_triage_evaluate` JSON-RPC method to agent/schemas.rs: accepts a synthetic trigger payload with an optional `dry_run` flag. When dry_run=true the decision is not acted on (no sub-agent dispatch). Canonical test/replay hook for exercising the triage pipeline without forging a Socket.IO webhook. - Flip the composio subscriber from opt-in (`OPENHUMAN_TRIGGER_TRIAGE_ENABLED`) to default-on with an opt-out escape hatch (`OPENHUMAN_TRIGGER_TRIAGE_DISABLED`). Every Composio trigger now flows through the triage pipeline unless the env var is explicitly set. - Update composio/bus.rs tests to match the new opt-out semantics.	2026-04-11 23:16:57 -07:00
Steven EnamakelandGitHub	403f239ca5	refactor: remove QuickJS skills runtime (#508) ... * refactor: remove quickjs skills runtime * style: apply repo formatting * refactor: clean up error reporting and connection handling - Removed the 'skill' source option from the error report structure to streamline error reporting. - Refactored the ConnectionsPanel component to simplify connection status badge rendering and improve clarity. - Updated the CronJobsPanel to enhance logging for cron job loading processes. - Adjusted SkillCard component to use a more consistent type for icons. - Deleted outdated end-to-end tests for Gmail and Notion skills, improving test suite maintainability. * fix: remove unnecessary ESLint disable comment in Conversations component - Cleaned up the Conversations component by removing the ESLint disable comment for exhaustive dependencies in the useEffect hook, improving code clarity and maintainability. * fix: remove unnecessary whitespace in Conversations component - Eliminated an extra line of whitespace in the Conversations component, enhancing code readability and maintainability. * refactor: streamline SkillCard imports for improved clarity - Combined import statements in the SkillCard component to enhance code readability and maintainability.	2026-04-11 14:57:35 -07:00
Steven EnamakelandGitHub	afb95e72a5	refactor(event-bus): native typed request/response surface + channels encapsulation (#505) ... * feat(event-bus): introduce typed request/response API for enhanced inter-module communication - Added a typed request/response surface to the existing event bus, allowing modules to execute requests through a shared controller registry. - Implemented `request_global` and `request_controller_global` functions for executing typed requests, enhancing the API's usability. - Updated documentation to reflect the new capabilities and usage patterns for the event bus, including when to use the request API versus traditional event publishing. - Added tests to validate the functionality of the new request/response features, ensuring robust integration with existing event bus operations. * refactor(event-bus): restructure event bus module and update references - Moved the event bus implementation from `src/openhuman/event_bus/` to `src/core/event_bus/`, establishing a clearer module hierarchy. - Updated all references throughout the codebase to reflect the new location of the event bus, ensuring consistency and reducing confusion. - Enhanced documentation to clarify the usage of the event bus and its core types, improving developer experience. - Introduced new files for event handling, requests, and subscribers, streamlining the event bus functionality and making it more modular. - Added tests to validate the new structure and ensure that the event bus operates correctly after the refactor. * refactor(event-bus): enhance event bus with native request/response surface - Updated the event bus to include a native, in-process typed request/response surface, allowing for zero serialization of Rust types and direct communication between modules. - Replaced the previous request API with a more streamlined approach using `register_native_global` and `request_native_global` functions. - Improved documentation to clarify the usage of the event bus, detailing when to use broadcast events versus native requests. - Removed the old request/response implementation to simplify the event bus structure and enhance maintainability. - Added examples and guidelines for registering and using native request handlers, improving developer experience and usability. * feat(agent): introduce native request handlers for agentic turns - Added a new `bus` module to encapsulate native event-bus handlers for the agent domain, including the `agent.run_turn` handler for executing agentic turns. - Updated the event bus registration process to include the new agent handlers, allowing for direct in-process request/response communication without serialization. - Refactored the channel message processing to dispatch agentic turns through the native bus, enhancing modularity and testability. - Improved documentation to clarify the usage of the new agent handlers and their integration with the event bus. - Added tests to validate the functionality of the new request handlers and ensure proper routing through the event bus. * refactor(tests): implement global bus handler lock for channel dispatch tests - Introduced a `use_real_agent_handler` function to manage the global bus handler lock during channel dispatch tests, ensuring exclusive access to the `agent.run_turn` handler. - Updated multiple test files to utilize the new handler function, improving test reliability by preventing race conditions during concurrent test execution. - Enhanced documentation to clarify the usage of the bus handler lock in tests that interact with the global native request registry. * feat(tests): add integration tests for Discord channel dispatch - Introduced a new test file `discord_integration.rs` to validate the end-to-end functionality of the Discord dispatch path within the channels module. - Implemented tests to ensure proper handling of inbound messages, reaction capabilities, and conversation history management specific to Discord. - Updated `mod.rs` to include the new Discord integration tests, enhancing overall test coverage for the channels module. * feat(tests): add Telegram integration tests for channel dispatch - Introduced new tests in `telegram_integration.rs` to validate the end-to-end functionality of the Telegram dispatch path within the channels module. - Implemented tests to ensure proper handling of threaded inbound messages, automatic acknowledgment reactions, and response routing through the agent bus handler. - Enhanced test coverage for Telegram, ensuring that the `supports_reactions()` capability is honored and that the dispatch pipeline operates correctly for both Telegram and Discord channels. * feat(tests): add testing utilities for event bus stubbing - Introduced a new `testing` module in the event bus to provide shared utilities for stubbing the global native bus registry. - Implemented `mock_bus_stub` and `MockBusGuard` to facilitate safe installation and restoration of stub handlers in tests, preventing race conditions. - Updated existing tests to utilize the new mocking utilities, enhancing test reliability and clarity in handling agent bus interactions. - Improved documentation to guide users on using the new testing features effectively. * style(event-bus): clean up formatting and remove unnecessary line breaks - Removed trailing whitespace and unnecessary line breaks in the event bus module files to improve code readability and maintainability. - Consolidated import statements and function definitions for a cleaner code structure across the event bus and agent modules. * fix(memory): point conversations/bus.rs at crate::core::event_bus Incoming `memory/conversations/bus.rs` from upstream/main still imports from the old `openhuman::event_bus` path. This branch relocated the bus to `core::event_bus`, so the merge left the import unresolved and the crate failed to compile. Rewire both references (`use` + fully-qualified `subscribe_global` call) to the canonical `crate::core::event_bus` path.	2026-04-11 01:05:50 -07:00
Steven EnamakelandGitHub	462b4d2895	Move conversation persistence into workspace memory store (#503) ... * Move conversation persistence into workspace memory store * Apply formatter updates for conversation persistence * Enhance conversation persistence by registering a new subscriber for channel events. Update domain subscriber registration to include workspace directory, ensuring proper message handling and persistence across channels. Refactor event structure to include message ID and reply target for improved tracking. Additionally, adjust module visibility for context management.	2026-04-10 23:04:59 -07:00
Steven EnamakelandGitHub	1c7318603e	feat(composio): backend-proxied Composio integration end-to-end (#501) ... * feat(composio): add Composio integration module with OAuth support - Introduced a new Composio module for backend-proxied access to various OAuth integrations. - Implemented ComposioClient for handling API requests related to toolkits, connections, and actions. - Added RPC operations for listing toolkits, managing connections, and executing actions. - Registered Composio controllers and schemas for integration with the existing system. - Created a debug script for testing Composio OAuth flow against the live backend. - Updated Cargo.lock to version 0.52.3 to reflect the new changes. * feat(composio): implement Composio connection modal and API integration - Added ComposioConnectModal for managing Composio toolkit connections, mirroring the user experience of existing modals. - Introduced composioApi for backend communication, including functions for listing toolkits, managing connections, and handling OAuth authorization. - Created toolkitMeta for displaying metadata of Composio toolkits in the Skills grid. - Developed hooks for fetching and managing Composio integrations, ensuring real-time updates on connection status. - Updated Skills page to integrate Composio toolkits, providing a seamless user experience for connecting and managing integrations. * chore: update OpenHuman version to 0.52.3 and add debug-composio-trigger script - Bumped OpenHuman package version in Cargo.lock to 0.52.3. - Introduced a new script, debug-composio-trigger.mjs, for Socket.IO live listening of Composio trigger events, facilitating testing and debugging of webhook interactions. * style(composio): apply Prettier + rustfmt from pre-push hook * feat(composio): enhance ComposioConnectModal and improve polling logic - Updated ComposioConnectModal to handle various connection phases more effectively, including 'waiting' for pending connections. - Introduced new refs for managing polling state and in-flight requests to prevent overlapping executions. - Improved error handling during polling, providing clearer feedback on OAuth timeouts. - Added logic to resume polling if the modal opens while an OAuth handoff is in progress. - Refactored connection handling in the modal for better clarity and maintainability. * refactor(composio): improve JSON payload handling and connection verification - Updated debug-composio-login.sh to build JSON payloads using jq for safer handling of toolkit data. - Enhanced connection verification logic in debug-composio-trigger.mjs to prioritize newly created connections and provide warnings for missing toolkits. - Refactored composio operations in ops.rs to return a more explicit error type, improving clarity in error handling across RPC operations.	2026-04-10 22:52:18 -07:00
Steven EnamakelandGitHub	57d307ad1e	feat(agent): simplify harness + trim bundled prompts (#500) ... * Add built-in agent definitions and prompts for orchestrator, planner, code executor, skills agent, researcher, critic, archivist, and tool maker - Introduced a new module for built-in agent definitions, allowing for easy addition of agents through a structured format. - Created TOML configuration files for each agent, detailing their properties such as id, display name, usage context, and tool specifications. - Developed corresponding prompt files that outline the responsibilities and rules for each agent, enhancing their functionality and usability. - Implemented a loading function to parse these definitions into usable agent structures, ensuring all agents are correctly initialized and validated. * Remove deprecated archetypes and related components from the multi-agent harness - Deleted the `archetypes.rs`, `dag.rs`, `executor.rs`, and `types.rs` files, which contained the definitions and implementations for agent archetypes, task DAGs, and execution logic. - Updated the `builtin_definitions.rs` and `definition.rs` files to remove references to the now-removed archetypes, streamlining the agent definition process. - Refactored the `mod.rs` file to eliminate unused modules and clarify the structure of the multi-agent harness. - Adjusted comments and documentation to reflect the removal of the DAG orchestration flow, emphasizing the current sub-agent delegation model. * Remove unused `ArchetypeConfig` from schema exports in `mod.rs` to streamline configuration management. * Implement context guard and memory management features in the harness module - Introduced `ContextGuard` to monitor context utilization and trigger auto-compaction when usage exceeds defined thresholds. - Added `scrub_credentials` function to sanitize sensitive information from tool outputs. - Implemented history management functions to trim conversation history and build compaction transcripts. - Created `build_tool_instructions` to generate tool usage guidelines for the system prompt. - Developed `memory_context` functions to manage user memory and relevant context for conversations. - Enhanced the `tool_loop` to integrate the new context management and tool invocation logic. This update improves the agent's ability to manage context effectively, ensuring efficient memory usage and secure handling of sensitive data. * Refactor agent module imports to use the harness instead of loop_ - Updated import paths in `dispatcher.rs`, `memory_loader.rs`, `mod.rs`, `dispatch.rs`, and `startup.rs` to replace references from the `loop_` module to the `harness` module. - This change enhances module organization and aligns with the recent architectural updates in the agent structure. * Remove cost tracking and context assembly modules from the agent structure - Deleted `cost.rs` and `context_assembly.rs` files, which handled token cost tracking and context assembly for the multi-agent harness, respectively. - Updated `mod.rs` files to remove references to the deleted modules, streamlining the agent's organization and focusing on essential components. - This cleanup enhances maintainability and aligns with the current architectural direction of the project. * Remove identity module and related configurations from the agent structure - Deleted the `identity.rs` file, which contained the AIEOS identity handling logic and related structures. - Updated `mod.rs` and other files to remove references to the deleted identity module, streamlining the agent's organization. - This change simplifies the codebase and aligns with the decision to rely solely on OpenClaw markdown files for identity management. * Enhance agent prompts with improved formatting and clarity - Added spacing for better readability in the prompts of the Archivist, Code Executor, Critic, Orchestrator, Researcher, Skills Agent, and Tool Maker agents. - Updated the table formatting in the Orchestrator prompt for consistency and clarity. - These changes improve the overall presentation and usability of agent documentation, making it easier for users to understand agent responsibilities and rules. * Remove REPL functionality and related components from the agent structure - Deleted the `repl.rs` file, which contained the implementation for the interactive REPL. - Removed references to the REPL in `cli.rs`, `mod.rs`, and other related files, streamlining the agent's organization. - Eliminated unused REPL session handling logic from the agent schemas and local AI operations, enhancing maintainability and focusing on essential components. - This cleanup aligns with the current architectural direction of the project, simplifying the codebase. * Refactor imports and clean up configuration exports - Updated import paths in `startup.rs` to include `host_runtime` from the correct module. - Streamlined the export statements in `mod.rs` and `schema/mod.rs` for better organization and clarity. - Removed unnecessary line breaks in the export lists to enhance readability and maintainability of the configuration schema. * Remove unused history management functions and clean up agent module exports - Deleted the `history.rs` and `session.rs` files, which contained functions for managing conversation history and session handling. - Updated `mod.rs` to remove references to the deleted modules and streamline the agent's organization. - Cleaned up import statements in `mod.rs` and other files to enhance clarity and maintainability of the codebase. * Add agent harness module with builder, runtime, and turn management - Introduced a new `harness` module for the `Agent`, encapsulating the `AgentBuilder`, runtime accessors, and turn lifecycle management. - Implemented the `AgentBuilder` fluent API for constructing `Agent` instances with customizable configurations. - Developed the `turn` method to handle user interactions, tool execution, and context management within the agent. - Created a `runtime` module for public accessors and utility functions related to agent operations. - Added comprehensive unit and integration tests to ensure the functionality of the new agent structure and its components. * Remove classifier and traits modules from the agent structure - Deleted the `classifier.rs` and `traits.rs` files, which contained the classification logic and core agent traits, respectively. - Updated `mod.rs` to remove references to the deleted modules, streamlining the agent's organization. - This cleanup enhances maintainability and focuses on essential components of the agent architecture. * Refactor agent prompt structure and remove unused files - Updated the `prompt.rs` file to streamline the orchestrator's logic by removing unnecessary tool documentation references and simplifying the file list. - Deleted several prompt files (`AGENTS.md`, `BOOTSTRAP.md`, `CONSCIOUS_LOOP.md`, `MEMORY.md`, `README.md`, `TOOLS.md`) to declutter the project and focus on essential components. - Adjusted the `harness/mod.rs` file to change module visibility from public to crate-level for better encapsulation. - These changes enhance maintainability and align with the current architectural direction of the project. * Refactor bootstrap file handling and update tests - Simplified the list of bundled prompt files in `prompt.rs` by removing references to `AGENTS.md` and `TOOLS.md`, focusing on essential identity files. - Adjusted the logic for injecting `MEMORY.md` to be optional, ensuring it only appears if it exists. - Updated test cases in `common.rs`, `identity.rs`, and `prompt.rs` to reflect the changes in the bootstrap file structure and ensure accurate validation of the new logic. - These modifications enhance clarity and maintainability of the codebase while aligning with the current project architecture. * Implement agent delegation tools and browser automation features - Introduced new tools for agent delegation, including `ArchetypeDelegationTool`, `AskClarificationTool`, `DelegateTool`, and `SkillDelegationTool`, enhancing the agent's ability to manage tasks through specialized sub-agents. - Added `SpawnSubagentTool` for delegating tasks to sub-agents, allowing for more complex workflows and improved task management. - Implemented browser automation capabilities with `BrowserOpenTool`, enabling secure opening of approved URLs in the Brave Browser. - Enhanced the `BrowserTool` with pluggable backends for improved automation and user interaction. - Added `ImageInfoTool` for extracting metadata from images, supporting future multimodal capabilities. - Organized tools into a new `impl` module structure for better maintainability and clarity in the codebase. * Refactor imports and clean up tool implementations - Updated import statements across various tool implementations to ensure consistency and clarity, particularly in the `traits.rs` file. - Removed unnecessary line breaks and adjusted module paths for better organization. - Cleaned up test files by removing trailing whitespace, enhancing code readability and maintainability. - These changes streamline the codebase and improve the overall structure of tool-related components. * Update built-in definitions and prompt handling for clarity and consistency - Revised the test for built-in agent definitions to dynamically reference the length of `BUILTINS`, enhancing maintainability. - Clarified documentation for the `system_prompt` field in `AgentDefinition`, emphasizing the default behavior and usage of inline prompts. - Simplified the `build_system_prompt` function by removing unnecessary conditional logic, ensuring a more straightforward return of the prompt string. * Enhance agent definition loading and documentation clarity - Updated the `load_file` function to reject definitions with missing or empty `system_prompt`, ensuring custom definitions are properly validated. - Added a new test to verify that definitions lacking a `system_prompt` are correctly rejected, improving robustness. - Clarified documentation for built-in definitions and TOML parsing, emphasizing compile-time guarantees and runtime checks. - Improved the logic for checking the existence of `MEMORY.md` to prevent errors from stray directories, enhancing file handling reliability.	2026-04-10 21:41:45 -07:00
Steven EnamakelandGitHub	31297ad19d	refactor(memory): remove GLiNER/GLiREL ingestion phase (#499) ... * refactor(memory): replace GLiNER model with heuristic extraction - Removed GLiNER-related code and dependencies from the memory ingestion pipeline, transitioning to a heuristic-only extraction approach. - Updated documentation and comments to reflect changes in extraction methods. - Adjusted tests to ensure compatibility with the new heuristic extraction configuration. - Bumped version of the tokenizers dependency and updated Cargo.lock accordingly. * chore: apply formatting and tauri lockfile sync	2026-04-10 20:52:36 -07:00
YellowSnnowmannandGitHub	aee9c52e88	fix(channels): Telegram threading, live listeners, core restart, and webhook cleanup (#485) ... * feat(config): enhance world-readable config warning mechanism - Introduced a new static variable to track previously warned world-readable config files, preventing duplicate warnings. - Updated the warning logic to only log a warning for each unique world-readable config file, improving log clarity and reducing noise. - Added new `ChannelReactionReceived` and `ChannelReactionSent` events to the DomainEvent enum, expanding event handling capabilities in the event bus. - Included tests for the new reaction events to ensure proper functionality and integration. * feat(logging): add log file constraints and event filtering - Introduced functions to parse log file constraints from environment variables and filter log events based on these constraints. - Enhanced the `init_for_cli_run` function to apply the new filtering logic, improving log management and clarity. - Updated the `conversation_history_key` function to include thread context for Telegram, ensuring accurate message targeting. - Added a new trait method `supports_reactions` to the `Channel` trait, indicating support for emoji reactions. - Implemented integration tests for Telegram channel features, including reaction handling and thread message forwarding. * feat(telegram): enhance message handling with reactions and typing indicators - Added support for emoji reactions in Telegram responses, allowing for contextual acknowledgment of user messages. - Implemented a decision heuristic for when to use reactions, improving user interaction quality. - Introduced a typing indicator that activates immediately upon receiving a message, providing instant feedback to users. - Updated the channel delivery instructions to include new reaction syntax and guidelines for usage. - Enhanced tests to cover new reaction handling and message acknowledgment features, ensuring robust functionality. * fix(tests): update route key for Telegram message handling tests - Changed the route key in tests from `telegram_alice` to `telegram_alice_chat-1` to match the updated `conversation_history_key` format for Telegram. - This adjustment ensures accurate routing and consistency in message handling tests. * refactor(tests): streamline message handling in runtime tool calls - Refactored the message handling tests to utilize a `ChannelMessage` struct for improved clarity and maintainability. - Updated the route key generation to use the `conversation_history_key` function, ensuring consistency in message routing. - Simplified the invocation of `process_channel_message` by directly passing the constructed message, enhancing readability. * fix(telegram): enhance finalize_draft method to support thread context - Updated the `finalize_draft` method in the `Channel` trait and its implementation for `TelegramChannel` to accept an optional `thread_ts` parameter, allowing for message threading. - Adjusted related message handling functions to utilize the new parameter, ensuring proper message context during sending. - Modified tests to reflect changes in the `finalize_draft` method signature, enhancing the robustness of message handling in threaded conversations. * refactor(tests): ran format * feat(discord, telegram): implement core process restart on channel connection - Added functionality to restart the core process when a channel connection requires a restart, enhancing the user experience by automating the process. - Implemented error handling to log any issues during the restart, ensuring users are informed to restart the app if necessary. - Updated both Discord and Telegram configuration components to include this new behavior, improving consistency across channel integrations. * feat(core-update): enhance core update logging and error handling - Added warnings for outdated sidecar versions and potential mismatches in UI features, improving user awareness of version compatibility. - Implemented detailed error logging for failed attempts to fetch the latest core release, providing users with clear instructions for manual updates if necessary. - Enhanced logging for reusing existing core RPC endpoints, alerting users to potential issues with stale connections. * feat(channels): implement real-time channel listeners and enhance logging - Added support for real-time channel listeners for Telegram and Discord, ensuring inbound bot messages are polled during `openhuman run`. - Introduced a method to check for configured listening integrations, preventing unnecessary listener spawning when not needed. - Enhanced logging for channel connection events and message handling, providing better visibility into channel operations and user interactions. - Updated the Telegram channel connection to log the count of allowed users and mention-only settings for improved debugging. * chore(dependencies): update openhuman version to 0.51.18 and refactor imports in channel config components - Bumped the openhuman package version from 0.49.17 to 0.51.18 in Cargo.toml and Cargo.lock. - Refactored import statements in DiscordConfig.tsx and TelegramConfig.tsx to maintain consistency and ensure proper functionality. * Implement webhook deletion for long polling in TelegramChannel - Added `delete_webhook_for_long_polling` method to clear the Bot API webhook, enabling `getUpdates` long polling. - Updated error handling in `fetch_bot_username` to call the new method when a 409 conflict indicates an active webhook, allowing for retries after webhook deletion. - Enhanced logging for better traceability of webhook deletion and polling conflicts. * Refactor Discord and Telegram connection handling to ensure channel connection updates are dispatched regardless of restart requirement. Improved error handling during core process restart and enhanced logging for connection status.	2026-04-10 15:06:08 +05:30
Steven EnamakelandGitHub	6410db1fad	feat(thu-fullrun): overlay attention, skills sync, credits & settings refresh (#479) ... * Update Conversations component to enhance user messaging for budget limits. Changed the warning text for exhausted weekly inference budget to improve clarity and user experience. * feat(schemas): add new configuration option for vision model usage - Introduced a new optional boolean field `use_vision_model` in the schemas for enabling vision LLM for screenshot analysis. - Updated the screen intelligence schemas to include a required `consent` field for starting sessions, replacing the previous `sample_interval_ms` field. - Enhanced the `ttl_secs` field description for clarity and modified the `capture_policy` to `screen_monitoring` for better understanding of its purpose. * feat(CoreStateProvider): enhance state management with optimistic updates and error handling - Implemented optimistic local commits for `setAnalyticsEnabled` and `setOnboardingCompletedFlag` to provide instant UI feedback while ensuring state consistency through authoritative snapshot refreshes. - Added error handling for the `refresh` function calls in `setAnalyticsEnabled`, `setOnboardingCompletedFlag`, and `clearSession` to log failures, improving robustness in state management during user interactions. - Updated dependencies in the `useCallback` hooks to include `refresh`, ensuring proper state updates and synchronization with the core. * feat(paths): centralize runtime path resolution for user-scoped skills data - Introduced a new module `paths.rs` to handle the resolution of runtime paths for skills, ensuring that `skills_data` and `workspace` directories are scoped per user. - Updated `bootstrap_skill_runtime` and `bootstrap_skills_runtime` functions to utilize the new path resolution logic, improving consistency and clarity in directory management. - Enhanced error logging for directory creation failures to include the specific path that failed, aiding in debugging. - Added a new optional field `overlay_ttl_ms` in the autocomplete schemas to support overlay time-to-live configuration. * feat(ScreenIntelligencePanel): optimize config synchronization to prevent user edit clobbering - Introduced a reference to track the last synced configuration signature, ensuring that user edits are preserved during periodic updates from the CoreStateProvider. - Updated the effect to compare serialized configuration values, allowing for re-sync only when actual changes occur, enhancing user experience and preventing unintended data loss. * feat(SkillManager): implement initial sync after OAuth completion - Added functionality to trigger an initial data sync immediately after OAuth completion, ensuring users see fresh data without waiting for the next scheduled sync. - Updated comments to clarify the change in sync behavior due to recent modifications in the Rust core, which no longer auto-triggers sync on OAuth completion. * fix(UsageLimitModal, Conversations): enhance user messaging for budget limits - Updated warning messages in both UsageLimitModal and Conversations components to provide clearer information regarding weekly limits and reset times. - Improved clarity in user notifications to enhance overall experience when budget limits are reached. * refactor(SkillSetupModal): improve session mode handling for skill configuration - Updated the SkillSetupModal component to lock the mode at mount time, ensuring users remain in the setup wizard during their session even if the skill is marked as complete. - Simplified mode management by replacing the forceSetup state with a sessionMode state, allowing explicit mode switching while maintaining a consistent user experience. * feat(SkillManager): enhance setup flow for OAuth-based skills - Updated the `startSetup` method to handle OAuth-based skills more effectively by implementing a fallback to core RPC for skills without a frontend runtime. - Improved error handling to treat missing `onSetupStart` implementations as successful completion for pure OAuth skills, allowing the setup wizard to display the "Connected!" screen. - Added detailed logging for both local runtime and core RPC fallback scenarios to improve traceability during the setup process. * feat(Home): enhance local AI status handling and asset management - Introduced a new state for local AI assets, allowing for better tracking of model file readiness. - Updated the loading logic to fetch both local AI status and assets concurrently, improving performance and error handling. - Implemented a mechanism to hide the Local Model Runtime card once all models are fully downloaded, enhancing user experience. - Added comprehensive comments to clarify the logic behind model readiness checks based on asset states. * refactor(Credits): update credit balance structure and terminology - Renamed credit categories in the RewardsCouponSection and PayAsYouGoCard components for clarity, changing "General credits" to "Promo credits" and "Top-up credits" to "Team top-up." - Updated the credit balance API to reflect the new structure, replacing `balanceUsd` and `topUpBalanceUsd` with `promotionBalanceUsd` and `teamTopupUsd`. - Adjusted normalization logic in the credits API to accommodate the new credit balance fields. - Modified tests to ensure correct handling of the updated credit balance structure. * feat(Settings): reorganize billing settings and update descriptions - Added a new top-level billing section to the settings, promoting it out of the Account & Security category for better visibility. - Updated the description for the Account & Security section to remove billing references, focusing on recovery phrase, team management, and linked account access. - Adjusted the settings navigation to accommodate the new billing section, ensuring proper routing and user experience. * refactor(Config): change logging level from info to debug for environment overrides - Updated logging statements in the Config implementation to use debug level instead of info, reducing verbosity during runtime while maintaining necessary traceability for configuration loading. * feat(Overlay): implement overlay attention event handling and refactor overlay app structure - Introduced a new overlay module to manage attention events, allowing the core to publish messages to the overlay window. - Enhanced the OverlayApp component to handle dictation and attention events, improving user interaction with the overlay. - Refactored the overlay state management to support different modes (idle, stt, attention) and added auto-dismiss functionality for attention messages. - Removed the Browser Access Toggle from the Skills page, streamlining the UI and focusing on core functionalities. - Updated tests to reflect changes in the Skills component and removed unnecessary mocks related to browser access. * fix(OverlayBubbleChip): reset typewriter animation on new bubble identity - Updated the OverlayBubbleChip component to reset the typewriter animation correctly when a new bubble is displayed by using the `key` prop. - Refactored the cleanup logic in the useEffect hook to ensure proper interval management and state reset, enhancing the user experience with bubble transitions. * refactor(rest): streamline key_bytes_from_string function and improve readability - Simplified the condition for checking the ASCII key length and character restrictions in the key_bytes_from_string function. - Consolidated the import statements for base64 engines into a single line for better clarity. - Adjusted test data formatting for improved readability in the key_bytes_from_string_tests module. * enhance(logging): improve color detection logic for terminal output - Updated the color detection logic in the logging module to prioritize environment variables (`NO_COLOR`, `FORCE_COLOR`, `CLICOLOR_FORCE`) for better control over color output. - Added detailed comments explaining the color resolution order, enhancing code clarity and maintainability. * test(Home): add mock for openhumanLocalAiAssetsStatus in tests - Enhanced the Home and HomeBootstrapButtons test files by adding a mock implementation for openhumanLocalAiAssetsStatus, which resolves to an object with null result and empty logs. This improves the test setup for local AI asset status handling. * refactor(SkillSetupModal): improve session mode handling and loading state - Updated the SkillSetupModal component to ensure session mode is determined after the first snapshot resolution, preventing premature defaults to the setup wizard. - Introduced a loading state to display a message while waiting for the skill setup status, enhancing user experience during the modal's initial render. - Refactored the SkillManager to throw errors for real failures during setup, ensuring proper error handling and user feedback. * refactor(Config): simplify logging for invalid proxy scope values - Consolidated the logging statement for invalid OPENHUMAN_PROXY_SCOPE values into a single line, improving code readability while maintaining the warning functionality.	2026-04-09 23:01:56 -07:00
Steven EnamakelandGitHub	6465f3d314	feat(agent): sub-agents, reasoning→agentic routing, layered context pipeline (#474) ... * Enhance agent architecture with sub-agent support and memory optimizations - Refactored the `Agent` struct to use `Arc` for shared ownership of the provider, tools, and tool specifications, enabling efficient memory management and concurrent access. - Introduced a `NullMemoryLoader` to optimize memory usage for sub-agents, allowing them to operate without incurring the cost of memory recall. - Added new methods in the `Agent` implementation to facilitate sharing of the provider, tools, and tool specifications with sub-agents, enhancing their operational efficiency. - Implemented a new `SystemPromptBuilder` method for constructing prompts specifically for sub-agents, ensuring they receive tailored context while minimizing unnecessary information. - Established a framework for loading custom agent definitions from TOML files, allowing for dynamic agent configuration and specialization. - Introduced a `ForkContext` to support efficient sub-agent execution in fork mode, leveraging shared resources for improved performance and reduced token usage. * Enhance agent definition management and sub-agent functionality - Introduced a global `AgentDefinitionRegistry` to manage built-in and custom agent definitions from TOML files, ensuring idempotent initialization. - Added new RPC handlers for listing, fetching, and reloading agent definitions, improving the flexibility of agent management. - Refactored the `Agent` struct to streamline sub-agent execution, including enhancements to the task execution flow and context handling. - Updated the orchestrator configuration to support fork mode for sub-agents, optimizing resource usage and performance. - Improved error handling and logging for agent definition loading and initialization processes, enhancing system reliability. * Add end-to-end test for sub-agent spawning and response integration - Implemented a new asynchronous test to validate the full path of a parent agent issuing a `spawn_subagent` tool call. - The test ensures that the sub-agent's output is correctly folded into the parent's response, verifying the interaction between the parent agent and the sub-agent. - Enhanced the `AgentDefinitionRegistry` to support global initialization of built-in agents, ensuring consistent behavior across tests. - Updated the handling of tool calls and memory configuration to facilitate the new test scenario, improving overall test coverage for agent interactions. * Enhance agent tool filtering with category support - Introduced a new `category_filter` in `AgentDefinition` to restrict tool visibility based on their category (System or Skill). - Updated the `from_archetype` function to apply the category filter for the `SkillsAgent` archetype. - Modified `SubagentRunOptions` to include a `category_filter_override` for dynamic filtering during sub-agent execution. - Enhanced the `filter_tool_indices` function to incorporate category filtering logic, ensuring tools are correctly filtered based on their defined categories. - Updated relevant tests to validate the new category filtering functionality, improving overall test coverage for agent interactions. * Implement layered context reduction pipeline for agent - Introduced a new `context_pipeline` module to manage a layered context reduction strategy, enhancing memory efficiency during agent interactions. - Added stages for tool-result budgeting, history trimming, microcompaction, autocompaction, and session memory extraction, each with specific triggers and cache implications. - Updated the `Agent` struct to include a `context_pipeline` field, ensuring state persistence across turns. - Enhanced the `AgentBuilder` to initialize the context pipeline by default. - Implemented tests to validate the functionality and stability of the context pipeline, ensuring consistent behavior across agent sessions. - Refactored relevant components to integrate the new context management features, improving overall agent performance and memory handling. * Enhance agent context pipeline with tool result budgeting and microcompaction - Renamed variable for clarity in tool execution result handling. - Implemented a new stage in the context pipeline to apply a byte budget to tool results, ensuring efficient memory usage. - Added logging for budget application, including details on original and final byte sizes. - Integrated microcompaction stages before tool calls to manage history and reduce memory footprint, with appropriate logging for outcomes. - Updated the agent's session memory management to track turn counts, facilitating better resource handling across iterations. * Refactor agent context pipeline for session memory extraction and tool call management - Simplified method calls in the `Agent` struct for clarity and efficiency. - Enhanced session memory extraction logic to spawn a background archivist sub-agent when thresholds are met. - Improved context pipeline handling for tool call recording and usage tracking. - Updated documentation and comments for better understanding of session memory extraction process. - Refactored microcompact function for cleaner code structure and readability. * refactor(agent): split agent.rs into focused submodules - Convert agent.rs (1988 lines) into agent/ folder with six files: * types.rs — Agent + AgentBuilder struct defs * builder.rs — AgentBuilder fluent API + Agent::from_config factory * turn.rs — turn lifecycle, tool dispatch, context pipeline wiring * runtime.rs — public accessors, run_single/run_interactive, helpers * tests.rs — integration tests with shared fakes * mod.rs — glue + top-level `run` convenience function - Drop misc external inspiration references from doc comments in the context_pipeline module and fork_context — the files now stand on their own design language. * fix(agent): address review comments on sub-agent + context pipeline PR Inline comment fixes: - definition.rs: YAML/TOML inconsistency — module doc, PromptSource, source bookkeeping, and load() all now uniformly document the TOML format. - subagent_runner.rs: render_subagent_system_prompt previously only appended PromptSource::Inline bodies, silently dropping PromptSource::File content. Thread the preloaded archetype_body through as an explicit &str parameter so both source variants render. Drops the unused SystemPromptBuilder + tools_for_prompt wiring while we're here. - prompt.rs: remove DateTimeSection from SystemPromptBuilder::for_subagent. Local::now() would make the sub-agent system prompt change per call and break KV-prefix cache stability. Document the invariant. Nitpick fixes: - agent/turn.rs: drop the no-op mark_extraction_started() call — the immediately-following mark_extraction_complete() clears the in-flight flag anyway. - context_pipeline/pipeline.rs: call guard.record_compaction_success() on microcompact success so a prior streak of autocompaction failures doesn't leave the circuit breaker tripped after a successful reduction. - context_pipeline/tool_result_budget.rs: remove the unnecessary out.clone() in the truncation return — capture final_bytes first, then move out. - definition_loader.rs: replace brittle reg.len() == 10 with assert!(reg.len() > 1) plus the existing targeted .get() checks. - executor.rs: tracing::warn! on unknown sandbox override values so typos surface during development; explicitly accept "none" and empty string as valid defaults. - fork_context.rs: add parent_context_visible_inside_scope test mirroring fork_context_visible_inside_scope, with minimal stub Provider/Memory impls so the test stays self-contained. - schemas.rs: drop redundant serde_json::to_value(serde_json::json!(...)) wrapping in handle_list_definitions + handle_get_definition. - event_bus/events.rs: add SubagentSpawned/SubagentCompleted/SubagentFailed cases to all_variants_have_correct_domain. - tools/ops.rs: add all_tools_includes_spawn_subagent regression test. - tools/spawn_subagent.rs: sort/dedup the known skill list in place instead of cloning into a second vec. Tests: 2316 passed / 0 failed (up from 2314; two new tests added). fmt + clippy clean on all touched files. * udpate prompts * Enhance AgentBuilder and runtime with event context and interactive CLI improvements - Added `event_context` method to `AgentBuilder` for setting `session_id` and `channel` for `DomainEvent`s, improving event tagging and correlation. - Updated `run_interactive` method in `Agent` to dispatch messages through `run_single`, ensuring consistent lifecycle event handling and error sanitization for interactive turns. * Optimize configuration handling in AgentBuilder by lazily creating Arc for full config in reflection hook. This change reduces unnecessary cloning when learning is enabled, improving performance. * Add fork-mode test for sub-agent spawning in agent - Introduced a new test, `turn_dispatches_spawn_subagent_in_fork_mode`, to validate the behavior of the agent when spawning a sub-agent in fork mode. - The test ensures that the parent agent correctly processes the sub-agent's output and maintains the expected response sequence. - Enhanced the test setup with a mock provider and memory configuration to simulate the agent's environment effectively. * Refactor sync RPC handling in skills to treat missing onSync as no-op - Updated the `handle_sync` function to log a debug message and return a no-op response when a skill does not implement the `onSync` handler, preventing unnecessary RPC errors for skills that do not require periodic syncs. - This change improves logging clarity and reduces error noise in logs and dashboards for skills that are not designed to handle sync operations.	2026-04-09 21:09:34 -07:00
Steven EnamakelandGitHub	3d17bfb23d	Refactor core cross-domain flows onto event bus (#468) ... * Refactor core cross-domain flows onto event bus * feat(agent): enhance error handling and message processing - Introduced a new constant for maximum error message length in the Agent. - Added methods for comparing conversation messages and determining new entries for turns. - Implemented a function to sanitize error messages, improving clarity and consistency in error reporting. - Updated the `run_single` method to utilize the new error handling and message processing logic, ensuring better tracking of conversation history and error states. * feat(supervision): initialize global event bus and register health subscriber - Added initialization of the global event bus with default capacity to ensure channel health events have a live bus and subscriber target. - Registered a health subscriber to enhance monitoring capabilities within the supervised listener context.	2026-04-09 16:44:53 -07:00
Steven EnamakelandGitHub	63b17ca55c	refactor: remove dead frontend modules and obsolete API layers (#469) ... * refactor: remove unused socket, agent tool registry, daemon health, and API service files - Deleted the `useSocket` hook, `AgentToolRegistry`, `DaemonHealthService`, and various API service files including `actionableItemsApi`, `apiKeysApi`, `feedbackApi`, `inferenceApi`, `managedDmApi`, and `settingsApi`. - This cleanup reduces code complexity and improves maintainability by removing obsolete components that are no longer in use. * feat: add knip configuration and commands to package.json - Introduced new scripts for running knip in development and production modes in both package.json files. - Added a knip.json configuration file to specify entry points and project files for dependency analysis. - Updated yarn.lock to include new dependencies related to knip, enhancing the project's dependency management capabilities. This commit improves the project's tooling for managing dependencies and ensures better code quality through automated checks. * refactor: remove unused components and clean up dependencies - Deleted several unused components related to intelligence features, including ActionPanel, InputGroup, SectionCard, and ValidatedField, to streamline the codebase. - Removed mock data and country data files that are no longer in use, enhancing maintainability. - Cleaned up the package.json by removing the @heroicons/react dependency, which is no longer required. - This commit improves the overall project structure and reduces complexity by eliminating obsolete code. * refactor: remove IntelligenceApiService and redefine ConnectedTool interface - Deleted the `IntelligenceApiService` class and its associated backend API methods to streamline the codebase. - Introduced a local definition of the `ConnectedTool` interface in `useIntelligenceApiFallback.ts` for better encapsulation and clarity. - This refactor enhances maintainability by eliminating unused code and consolidating relevant types within the appropriate context. * style: format knip config * chore(dependencies): update OpenHuman to version 0.52.0 in Cargo.lock * feat: implement Daemon Health Service for polling and state management - Introduced the `DaemonHealthService` class to poll the Rust core health snapshot and synchronize the frontend daemon store. - Added methods for setting up a health listener, parsing health snapshots, and updating the daemon store based on health data. - Implemented a timeout mechanism to handle disconnection scenarios, enhancing the reliability of the daemon's health monitoring. - This addition improves the application's ability to maintain an accurate representation of the daemon's health status in real-time. * chore(knip): update entry points in knip configuration - Modified the `entry` field in `knip.json` to include `src/main.tsx` alongside existing test specifications. - This change ensures that the main application file is included in dependency analysis, improving project structure and tooling. * refactor: streamline code and enhance readability across multiple modules - Consolidated multiple `replace` calls into single calls using arrays for improved efficiency in text processing. - Simplified default implementations for several structs, removing redundant code. - Updated query mapping in database interactions to enhance clarity and maintainability. - Improved logging and error handling by refining how state and error messages are processed. - Enhanced the readability of various functions by restructuring conditional checks and simplifying logic. These changes collectively improve code maintainability and performance across the application. * Merge remote-tracking branch 'origin/fix/cleanup' into fix/cleanup * fix: update error handling in bootstrap_after_login function - Changed the error parameter in the inspect_err closure to an underscore to indicate it is unused. - This minor adjustment improves code clarity and adheres to Rust conventions for unused variables.	2026-04-09 16:41:27 -07:00
Steven EnamakelandGitHub	30eec2ad88	docs: comprehensive documentation for core Rust modules (#470) ... * feat(core): enhance RPC controller and dispatch logic - Added comprehensive documentation for the core RPC controller and dispatch modules, detailing their purpose and functionality. - Introduced new functions for managing registered controllers and schemas, including validation and invocation methods. - Improved the structure of the `RpcOutcome` type to standardize response formats across domain-specific handlers. - Enhanced the local AI operations module with additional functionalities for agent interactions, model management, and audio processing. - Updated the dispatcher to better route RPC calls to their respective handlers, ensuring a more robust and maintainable architecture. These changes improve the clarity and usability of the RPC system, facilitating easier integration and interaction with various components of the OpenHuman platform. * docs: comprehensive documentation for core Rust modules * chore(dependencies): update OpenHuman to version 0.52.0 in Cargo.lock and add knip dependency in package.json	2026-04-09 16:17:53 -07:00
Steven EnamakelandGitHub	acc6246e59	Refactor core-polled app state and screen intelligence status (#464) ... * refactor(accessibility): remove device control and predictive input features from accessibility settings - Updated accessibility-related components and tests to eliminate device control and predictive input features. - Adjusted AccessibilityPanel and ScreenIntelligencePanel to reflect the removal of these features. - Modified related tests to ensure consistency with the updated accessibility status structure. - Cleaned up accessibility session parameters and state management to focus solely on screen monitoring. * refactor(accessibility): streamline featureOverrides state initialization - Simplified the initialization of featureOverrides state in AccessibilityPanel and ScreenIntelligencePanel components for better readability. - Consolidated parameter definitions in startAccessibilitySession to enhance clarity and maintainability. - Removed unnecessary re-exports in the screen_intelligence engine module to clean up the codebase. * chore(dependencies): update OpenHuman to version 0.51.19 in Cargo.lock * chore(dependencies): update OpenHuman version to 0.51.19 in Cargo.lock * feat(restart): implement core process restart functionality - Added a new `SystemRestartRequested` event to the `DomainEvent` enum to handle restart requests. - Introduced a `RestartSubscriber` that listens for restart events and manages the process respawn. - Created a `service_restart` function to publish restart requests via the event bus. - Updated service schemas to include a new `restart` controller with parameters for source and reason. - Enhanced documentation to reflect changes in behavior and added necessary code comments. * feat(accessibility): add last restart summary to Screen Intelligence Panel - Introduced `lastRestartSummary` to the accessibility state and updated relevant components to display the last successful core restart information. - Modified `PermissionsSection` and `ScreenIntelligencePanel` to include the new summary. - Updated tests to validate the display of the last restart summary and ensure proper state management during core restarts. - Refactored accessibility slice to handle the new restart summary in state updates. * feat(core): enhance startup process with restart delay and subscriber registration - Added a call to apply startup restart delay from environment variables in `run_core_from_args`. - Updated the `bootstrap_skill_runtime` function to register a `RestartSubscriber` for handling restart requests, ensuring consistent respawn logic across triggers. - Introduced a new `core_process` field in the `AccessibilityEngine` to track the core process status, including its PID and start time. - Implemented a helper function to capture the core process start time using `OnceLock` for efficient initialization. * feat(screen-intelligence): refactor accessibility state management and UI components - Replaced direct Redux state access with a new `useScreenIntelligenceState` hook across multiple components, including `AccessibilityPanel`, `ScreenIntelligencePanel`, and their respective subcomponents. - Streamlined permission and session handling by consolidating related functions and removing unnecessary dispatch calls. - Updated tests to mock the new state management approach, ensuring consistent behavior and validation of UI elements. - Removed the `SessionAndVisionSection` component to simplify the structure and improve maintainability. - Introduced a new API file for screen intelligence to encapsulate related functionality and improve code organization. * refactor(tests): clean up and optimize test files for accessibility and screen intelligence panels - Removed redundant imports and streamlined the structure of test files for `AccessibilityPanel` and `ScreenIntelligencePanel`. - Consolidated core process state initialization in test mocks for better readability. - Updated dependency imports and ensured consistent mocking of state management hooks across tests. - Enhanced the `ScreenPermissionsStep` component by improving the dependency array in the useEffect hook for better performance. * refactor(store): remove unused authentication and user management code - Deleted the `UserProvider`, `authSlice`, `authSelectors`, `userSlice`, `teamSlice`, and related test files to streamline the codebase. - This cleanup enhances maintainability by removing legacy code that is no longer in use. - Updated the store configuration to reflect the removal of these slices and ensure proper state management. * refactor(webhooks): reorganize types and remove legacy state management - Moved `TunnelRegistration` and `WebhookActivityEntry` types to a new `types.ts` file for better organization. - Updated imports in `TunnelList` and `WebhookActivity` components to reference the new types location. - Refactored `useWebhooks` hook to eliminate Redux state management in favor of local state, enhancing performance and reducing complexity. - Removed unused `aiSlice`, `inviteSlice`, and `webhooksSlice` along with their associated tests to streamline the codebase. * refactor(daemon): migrate state management from Redux to a custom store - Introduced a new `store.ts` file to manage daemon state, replacing the previous Redux slice. - Updated components and hooks to utilize the new state management approach, enhancing performance and reducing complexity. - Removed the legacy `daemonSlice` and associated Redux logic, streamlining the codebase. - Adjusted imports in various components and hooks to reference the new store structure. * refactor(screen-intelligence): integrate core state management and enhance status handling - Replaced direct state management in `useScreenIntelligenceState` with a new core state approach, utilizing `useCoreState` for improved performance and consistency. - Updated status fetching and permission handling to leverage the core state snapshot, streamlining the logic and reducing redundant API calls. - Introduced a new `CoreRuntimeSnapshot` interface to encapsulate runtime statuses, including screen intelligence, local AI, autocomplete, and service states. - Adjusted related components and hooks to align with the new state management structure, enhancing maintainability and readability. - Updated tests to validate the new runtime state structure and ensure proper functionality across the application. * refactor(components): reorganize imports and streamline function formatting - Moved the import of `Tunnel` and `tunnelsApi` in `TunnelList.tsx` for better organization. - Reformatted function definitions in `store.ts`, `useDaemonHealth.ts`, `useDaemonLifecycle.ts`, `useWebhooks.ts` for improved readability. - Cleaned up the structure of test files in `coreRpcClient.test.ts` by consolidating object properties for clarity. - These changes enhance code maintainability and readability across the application. * test(screen-intelligence): fix duplicate hook imports * fix(tests): update ScreenIntelligenceDebugPanel test to use baseState for refresh status and vision calls * refactor(invites): simplify error message rendering in Invites component - Consolidated the conditional rendering of the load error message in the Invites component for improved readability. - This change enhances the clarity of the code without altering functionality. * refactor(daemon): streamline state management and function definitions - Removed the `healthTimeoutId` from the `DaemonUserState` interface and related functions to simplify state management. - Converted several functions in `store.ts` to arrow function syntax for consistency and improved readability. - Updated the `Invites` component to handle asynchronous loading and error states more effectively, ensuring that in-flight requests are properly managed. - Refactored the `CoreStateProvider` to enhance the refresh logic and prevent multiple simultaneous refreshes. - Introduced a new `register_domain_subscribers` function in `jsonrpc.rs` to centralize event bus subscriber registration, improving code organization and maintainability. * fix: add debug logging, atomic restart guard, and idempotent subscriber registration - CoreStateProvider: add namespaced debug logger for polling failure diagnostics - service/bus.rs: add AtomicBool gate to prevent duplicate restart spawns - service/bus.rs: use OnceLock for idempotent RestartSubscriber registration - Invites.tsx: add debug log in loadInviteCodes catch block * style: apply prettier formatting to CoreStateProvider * fix: sanitize error logging, serialize refresh, and demote restart logs - CoreStateProvider: sanitize error objects in poll failure logs to avoid leaking tokens/headers - CoreStateProvider: move in-flight guard into refresh() via shared promise so all callers (poll, updateLocalState, storeSessionToken) are serialized - CoreStateProvider: log refreshTeams errors instead of swallowing them - service/bus.rs: demote duplicate-restart log to debug, omit reason from log output to avoid free-form text emission * style: apply cargo fmt to service/bus.rs	2026-04-09 15:51:27 -07:00
Steven EnamakelandGitHub	d66ee0d4de	feat: consolidate overlay into desktop app and add compact orb demo (#450) ... * feat(overlay): implement overlay window functionality and related RPC integration - Added a new OverlayApp component to handle overlay-specific UI and functionality. - Introduced an overlay window configuration in tauri.conf.json, allowing for a transparent, always-on-top overlay. - Implemented parent RPC communication for the overlay to interact with the main application. - Updated main.tsx to conditionally render the OverlayApp based on the current window context. - Enhanced CSS styles to support the overlay's visual requirements. This commit establishes the foundation for overlay functionality, improving user experience with a dedicated interface for specific tasks. * refactor(overlay): remove overlay functionality and related configurations - Deleted the overlay module and its associated files, including process management and configuration settings. - Removed environment variable checks and overlay-related logic from the core process and configuration schema. - Updated documentation to reflect the removal of overlay features, simplifying the codebase and improving maintainability. This commit streamlines the application by eliminating unused overlay components, enhancing overall performance. * feat(overlay): enhance overlay bubble functionality and styling - Added a new CSS animation for overlay bubble appearance, improving visual feedback. - Introduced an OverlayBubble interface to manage bubble properties such as tone and text. - Updated OverlayApp component to include a new OverlayBubbleChip for displaying messages with dynamic styling based on tone. - Adjusted overlay window dimensions in tauri.conf.json for a more compact design. This commit improves the user experience by providing visually distinct overlay messages and a refined interface. * feat(rotating-tetrahedron): add inverted color support and refactor canvas component - Introduced an optional `inverted` prop to the `RotatingTetrahedronCanvas` component, allowing for dynamic color changes based on the prop value. - Updated fill and edge materials to reflect the inverted state, enhancing visual customization. - Refactored the component to improve readability and maintainability by utilizing the new prop in the rendering logic. - Adjusted the effect dependencies to include the `inverted` prop for proper reactivity. This commit enhances the user experience by providing a more flexible and visually appealing tetrahedron display. * fix(rotating-tetrahedron): adjust opacity and emissive intensity for inverted colors - Updated the opacity of the fill material in the RotatingTetrahedronCanvas component to enhance visual clarity when the inverted prop is true. - Reduced the emissive intensity for the inverted state to improve the overall appearance of the tetrahedron. This commit refines the visual representation of the rotating tetrahedron, ensuring better contrast and aesthetics based on user preferences. * feat(rotating-tetrahedron): enhance dynamic color handling and performance - Implemented useRef hooks for fill and edge materials in the RotatingTetrahedronCanvas component to optimize rendering performance. - Updated the useEffect hook to adjust material properties based on the inverted state, improving visual consistency. - Refactored animation speed handling to utilize a reference for smoother updates. - Cleaned up resource management by ensuring materials are disposed of correctly when the component unmounts. This commit enhances the visual fidelity and performance of the rotating tetrahedron, providing a more responsive and visually appealing experience. * fix(overlay): adjust bubble alignment and overlay positioning - Changed the text alignment of the OverlayBubbleChip component from left to right for improved readability. - Updated the vertical positioning logic in the Tauri overlay to account for a right margin, ensuring consistent placement of the overlay window. These adjustments enhance the visual presentation and positioning of overlay elements, contributing to a better user experience. * fix(overlay): update overlay dimensions and bubble styling - Adjusted the overlay dimensions for a more refined appearance. - Modified the bubble tone classes for improved color consistency and readability. - Enhanced the text size and line height in the OverlayBubbleChip component for better visual clarity. - Updated the orb button size and styling to enhance user interaction. These changes contribute to a more polished and user-friendly overlay experience. * feat(overlay): enhance overlay scenario handling and text display - Introduced a new scenario management system in the OverlayApp component, allowing for dynamic cycling between different overlay states. - Added a new text display feature for scenario two, providing real-time feedback as the text is typed out. - Refactored the bubble rendering logic to accommodate the new scenario structure, improving the overall user interaction experience. These changes enhance the functionality and interactivity of the overlay, making it more engaging for users. * fix(overlay): reorder demo scenarios * fix(overlay): update overlay dimensions and improve text display logic - Adjusted overlay dimensions for better visual consistency. - Enhanced text display in OverlayBubbleChip to show text progressively based on bubble content. - Refactored the handling of scenario text in OverlayApp to streamline the display logic. These changes contribute to a more polished and engaging user experience in the overlay. * refactor(overlay): simplify type parameters in RPC functions - Removed unnecessary generic type parameter from `unwrapCliCompatibleJson` and `callParentCoreRpc` functions for improved clarity and conciseness. - These changes enhance code readability and maintainability without altering functionality.	2026-04-09 15:01:14 -07:00
3 people	0b23ae6b96	fix(voice): resolve dictation pipeline in embedded Tauri app (#466) ... * fix(voice): guard dictation_listener when voice_server is active macOS only supports one rdev::listen() global event tap per process. When voice_server.auto_start is true, skip starting the separate dictation_listener — the voice server owns the single listener and forwards hotkey events itself. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(voice): add TRANSCRIPTION_BUS broadcast channel Make publish_dictation_event public so the voice server can forward hotkey events. Add a new TRANSCRIPTION_BUS broadcast channel with subscribe_transcription_results() and publish_transcription() for delivering completed transcriptions to frontend clients via Socket.IO. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(voice): forward hotkey events and route transcription delivery In the voice server's hotkey handler, forward events to the dictation bus so Socket.IO clients receive dictation:toggle even without the separate dictation_listener running. In process_recording_bg, detect when the OpenHuman app is focused and deliver transcription via Socket.IO instead of OS-level Cmd+V paste, preventing text from disappearing into the unfocused WebView. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(voice): bridge transcription results to Socket.IO Subscribe to TRANSCRIPTION_BUS and emit dictation:transcription events to all connected Socket.IO clients. This completes the Rust-side pipeline for delivering transcribed text to the frontend. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(socket): queue listeners registered before socket connects Add a pendingListeners queue to socketService so that on()/once() calls made before the socket is established are replayed once the connection opens, preventing silently dropped event listeners. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(voice): use dedicated unauthenticated socket for dictation events Replace the auth-gated socketService dependency with a direct Socket.IO connection to the core process (127.0.0.1:7788). This bypasses the SocketProvider auth requirement and ensures dictation:toggle and dictation:transcription events are received regardless of login state. Dispatches the existing dictation://insert-text DOM event to bridge transcribed text into the Conversations chat input. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(voice): fallback to default audio config when preferred config fails macOS may advertise a 16kHz F32 config as supported but reject it at stream creation time (known cpal quirk). Add a fallback path that retries with the device's default_input_config(), handling F32, I16, and U16 sample formats with proper resampling. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(voice): add 3s timeout on Ollama LLM cleanup Wrap the Ollama inference call in a 3-second tokio::time::timeout so dictation feels responsive. If cleanup doesn't complete in time, fall back to raw Whisper text immediately instead of blocking for 2+ minutes. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>	2026-04-09 14:54:40 -07:00
3 people	f0118ab674	fix(autocomplete): graceful shutdown on app exit (#460) ... * fix(autocomplete): graceful shutdown on app exit Stop the autocomplete engine and quit the Swift overlay helper process when the core receives a shutdown signal, preventing orphan processes. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(core): extract shutdown logic into generic facility Move domain-specific autocomplete cleanup out of the jsonrpc adapter into a new `core::shutdown` module with a hook registry. Adds SIGTERM handling alongside SIGINT via `tokio::select!` on Unix platforms. Addresses CodeRabbit review feedback on PR #460: - jsonrpc.rs stays transport-only (no domain logic) - Process responds to both SIGINT and SIGTERM gracefully Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(autocomplete): suppress duplicate error notifications and auto-stop after repeated failures The autocomplete polling loop was showing an error notification badge on every single refresh cycle when a dependency was unavailable (e.g. Ollama not running, macOS Automation permission denied). This caused floods of identical macOS notifications. Changes: - Track last notified error message; skip badge if identical to previous - Count consecutive errors; auto-stop engine after 5 failures with a clear log message, preventing endless notification spam - Reset counters on successful refresh or engine restart Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>	2026-04-09 11:59:33 -07:00
3 people	9764a875e8	fix(subconscious): seed defaults into per-user workspace + fix Intelligence page stale log (#462) ... * fix(subconscious): seed defaults and spawn heartbeat on startup The subconscious engine was only constructed lazily on the first engine-routed RPC (trigger, tasks_add, status). Because handle_tasks_list bypasses the engine and reads the store directly, a fresh install showed an empty Subconscious panel until the user clicked "Run now", even though SubconsciousEngine::new() seeds the 3 default system tasks on construction. Separately, HeartbeatEngine::run() — the periodic tick loop — was never spawned in production code. The only callers of HeartbeatEngine were tests, so ticks never fired automatically; users had to trigger each evaluation manually. Both issues are fixed together in run_server_inner, following the existing start_if_enabled pattern used by voice, screen_intelligence, and autocomplete: 1. Call get_or_init_engine() at startup to construct the SubconsciousEngine eagerly, which runs seed_default_tasks via from_heartbeat_config. Construction is idempotent via OnceLock; seeding is idempotent by title match, so repeat startups do not duplicate the defaults. 2. Construct HeartbeatEngine with the heartbeat config and workspace_dir, then tokio::spawn heartbeat.run() so the periodic tick loop runs for the process lifetime. The loop re-acquires the shared engine via get_or_init_engine() on each tick. Guarded by config.heartbeat.enabled so users who disable the heartbeat get neither startup seeding nor the background loop. Add engine_construction_seeds_default_tasks integration test that locks in the invariant: constructing SubconsciousEngine on a fresh workspace_dir must leave the 3 default system tasks in the store, with no tick, trigger, or explicit seed call. Also asserts that reconstructing the engine on the same workspace does not duplicate the defaults. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(subconscious): defer engine bootstrap until after login Default system tasks seeded at sidecar startup into the pre-login global workspace (`~/.openhuman/workspace/`) instead of the per-user workspace (`~/.openhuman/users/<id>/workspace/`) the UI reads from after login. The engine singleton is built lazily via `get_or_init_engine()` and cached in a `OnceLock`. `Config::load_or_init` resolves `workspace_dir` from `active_user.toml` — which does not exist until after login. When the engine was constructed on startup it therefore seeded into the global default, then the frozen singleton kept pointing at that path for the rest of the session while RPC handlers like `tasks_list` re-loaded config per call and read from the correct per-user path, silently returning an empty list. Fix: - `subconscious/global.rs`: add `bootstrap_after_login()` (idempotent via `BOOTSTRAPPED: AtomicBool`) which builds the engine against the now-correct per-user workspace and spawns the heartbeat loop. Track the heartbeat `JoinHandle` in a static so it can be aborted cleanly. Add `reset_engine_for_user_switch()` that aborts the heartbeat, clears the engine option, and resets the bootstrap flag. - `core/jsonrpc.rs`: replace the unconditional eager init on startup with a conditional one that only bootstraps if `active_user.toml` already exists (so a user logged in from a previous session still gets the engine up immediately after restart). - `credentials/ops.rs`: call `bootstrap_after_login()` at the end of `verify_and_store_session` so a fresh login triggers seeding against the per-user workspace. Call `reset_engine_for_user_switch()` in `clear_session` so logout tears down the engine + heartbeat loop and a subsequent login rebuilds them against the new user. Verified locally: sidecar restart with no `active_user.toml` logs "bootstrap deferred — waiting for login"; post-login logs "seeded 3 tasks on init" + "heartbeat periodic loop spawned"; and `subconscious.tasks_list` returns the 3 system defaults from the per-user DB. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(subconscious): bound config load + guard frontend poll Two related fixes for the Intelligence page freezing on a stale subconscious activity-log snapshot while ticks kept progressing in the sidecar. Root cause (backend): the subconscious RPC handlers were the only outlier in the entire JSON-RPC surface that called the raw `Config::load_or_init()` instead of the shared `load_config_with_timeout()` wrapper that every other domain schemas.rs uses (cron, webhooks, voice, team, skills, service, referral, doctor, …). `load_or_init` constructs a fresh `SecretStore` and runs a chain of `decrypt_optional_secret` calls on every invocation, which may IPC to the OS keychain — slow, unbounded, no caching. Under the Intelligence page's 3-second poll (4 parallel RPCs × ~7 keychain round-trips each = ~28 keychain calls every 3s), this pileup was enough to pin the frontend's `Promise.all` past the poll interval. Root cause (frontend): `useSubconscious.refresh()` uses `fetchingRef` as an in-flight guard. The ref is only cleared inside the `finally` block that runs after `Promise.all` settles. With no per-RPC timeout on the client side either, a single slow backend call would leave the ref stuck `true`, and every subsequent 3s `setInterval` tick would silently early-return at the top of `refresh`. The poller kept firing, but every call was a no-op — so the UI froze on whatever snapshot it last successfully fetched, even though the backend was still ticking through new decisions. Backend fix (`src/openhuman/subconscious/schemas.rs`): - Replace the local `load_config()` helper body to delegate to `crate::openhuman::config::load_config_with_timeout()`. Matches the 28 other domain schemas.rs files and brings subconscious handlers under the same 30s bound used everywhere else. Frontend fix (`app/src/hooks/useSubconscious.ts`): - Add a `withTimeout` helper (2.5s per-RPC, strictly less than the 3s poll interval) that races each of the 4 parallel RPCs against a timeout and resolves `null` on timeout — matching the existing `.catch(() => null)` contract so downstream setState logic is unchanged. - Clear `fetchingRef.current = false` in the useEffect cleanup so a late-returning request or a React Strict Mode double-mount in dev can't leave the ref stuck `true` for the next mount. Defense in depth: the backend bound prevents a permanent hang and matches repo conventions, while the frontend bound guarantees the 3s poll loop can never be pinned beyond one tick regardless of server-side latency. Verified locally — `cargo check` clean, `tsc --noEmit` clean, all 18 pre-existing warnings in unrelated modules. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style(jsonrpc): cargo fmt the startup bootstrap block CI ran `cargo fmt --all -- --check` and flagged the conditional bootstrap block in `run_server_inner` — `let already_logged_in` should fold onto one line, the `.and_then` closure body should inline, the `match ... .await` chain should fold, and the short log!() calls should not break across lines. No behavior change. Fixes three jobs on PR #462 that were all failing at the same `cargo fmt --all -- --check` step (Rust Quality, Rust Tests, Type Check TypeScript — the last one chains cargo fmt after its prettier check). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-09 22:10:55 +05:30
3 people	0f578382d1	fix(autocomplete): auto-start engine when config enabled (#412) (#442) ... * fix(autocomplete): add start_if_enabled for engine auto-start at boot (#412) The autocomplete engine was never started automatically when config had autocomplete.enabled = true. Add start_if_enabled() that checks config and starts the global engine singleton during core process startup. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(autocomplete): wire engine startup into core server init (#412) Call start_if_enabled() after config load in the JSON-RPC server so the autocomplete engine runs automatically when the core process boots with autocomplete enabled. Remove stale E2E test assertions that conflicted with the new startup path. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(autocomplete): auto-start engine when enabled via set_style RPC (#412) When the frontend enables autocomplete through set_style(enabled=true), automatically start the engine so suggestions begin immediately without requiring a separate start call or app restart. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>	2026-04-09 02:56:58 +05:30
Mega MindandGitHub	371bcd34ea	Feat/chat issue (#441) ... * feat: display app version in settings panel * fix(onboarding): auto-refresh accessibility state after grant (#351) * style(onboarding): apply formatter for issue #351 fix * fix(onboarding): ESLint + typed mock for ScreenPermissionsStep; clear flag in handler Consolidate tauriCommands imports and drop redundant mock cast. Handle granted accessibility in focus/visibility callback instead of a follow-up effect. Made-with: Cursor * feat(env): add support for custom dotenv path and update dependencies - Introduced an optional environment variable `OPENHUMAN_DOTENV_PATH` to specify a custom path for dotenv files, enhancing configuration flexibility. - Updated `Cargo.toml` to include the `dotenvy` dependency for improved dotenv file handling. - Enhanced the `.env.example` file with a new comment for the custom dotenv path. - Added data-testid attributes and button types in `SkillDebugModal` and `Skills` components for better testability. - Created new tests for Gmail and Notion third-party skills to ensure proper functionality of sync and debug tools. - Added documentation for memory sync functions to clarify usage patterns and function details. * fix: address CodeRabbit review on PR #441 - Dotenv: treat empty OPENHUMAN_DOTENV_PATH as unset; propagate from_path errors - Document OPENHUMAN_DOTENV_PATH parent-env requirement in .env.example - Memory docs: MD040 fence language; clarify skill namespace vs integration id - QuickJS bootstrap: modern helpers, generic platform.notify log, template URLs - Skills UI: type=button on close/settings; async waitFor in sync tests - Gmail OAuth e2e: workspace env matches MemoryClient; env/engine drop guards; redact secrets from logs - Add replace_global_engine for test teardown Made-with: Cursor	2026-04-09 01:51:30 +05:30
YellowSnnowmannandGitHub	90fac95d7a	Fix: telegram replies (#436) ... * feat(config): enhance world-readable config warning mechanism - Introduced a new static variable to track previously warned world-readable config files, preventing duplicate warnings. - Updated the warning logic to only log a warning for each unique world-readable config file, improving log clarity and reducing noise. - Added new `ChannelReactionReceived` and `ChannelReactionSent` events to the DomainEvent enum, expanding event handling capabilities in the event bus. - Included tests for the new reaction events to ensure proper functionality and integration. * feat(logging): add log file constraints and event filtering - Introduced functions to parse log file constraints from environment variables and filter log events based on these constraints. - Enhanced the `init_for_cli_run` function to apply the new filtering logic, improving log management and clarity. - Updated the `conversation_history_key` function to include thread context for Telegram, ensuring accurate message targeting. - Added a new trait method `supports_reactions` to the `Channel` trait, indicating support for emoji reactions. - Implemented integration tests for Telegram channel features, including reaction handling and thread message forwarding. * feat(telegram): enhance message handling with reactions and typing indicators - Added support for emoji reactions in Telegram responses, allowing for contextual acknowledgment of user messages. - Implemented a decision heuristic for when to use reactions, improving user interaction quality. - Introduced a typing indicator that activates immediately upon receiving a message, providing instant feedback to users. - Updated the channel delivery instructions to include new reaction syntax and guidelines for usage. - Enhanced tests to cover new reaction handling and message acknowledgment features, ensuring robust functionality. * fix(tests): update route key for Telegram message handling tests - Changed the route key in tests from `telegram_alice` to `telegram_alice_chat-1` to match the updated `conversation_history_key` format for Telegram. - This adjustment ensures accurate routing and consistency in message handling tests. * refactor(tests): streamline message handling in runtime tool calls - Refactored the message handling tests to utilize a `ChannelMessage` struct for improved clarity and maintainability. - Updated the route key generation to use the `conversation_history_key` function, ensuring consistency in message routing. - Simplified the invocation of `process_channel_message` by directly passing the constructed message, enhancing readability. * fix(telegram): enhance finalize_draft method to support thread context - Updated the `finalize_draft` method in the `Channel` trait and its implementation for `TelegramChannel` to accept an optional `thread_ts` parameter, allowing for message threading. - Adjusted related message handling functions to utilize the new parameter, ensuring proper message context during sending. - Modified tests to reflect changes in the `finalize_draft` method signature, enhancing the robustness of message handling in threaded conversations. * refactor(tests): ran format	2026-04-08 11:52:38 -07:00
YellowSnnowmannandGitHub	45a821645a	feat(refer) : Implement referral system with UI components and API integration (#430) ... * Implement referral system with UI components and API integration - Added to document the referral system, including reward structure, rules, data model, migration, core services, and API endpoints. - Created component to display referral stats and allow users to apply referral codes. - Integrated referral functionality into the onboarding process with for applying referral codes. - Updated page to include the new . - Implemented API calls in for fetching referral stats and applying referral codes. - Added tests for the referral API to ensure proper functionality and data normalization. - Introduced device fingerprinting for referral code application to prevent abuse. This commit establishes a comprehensive referral system, enhancing user engagement and incentivizing referrals. * Update OLLAMA_BASE_URL to local development address * Enhance referral system and onboarding process - Added a new function to format reward rates from basis points to percentage for better display in the ReferralRewardsSection. - Improved loading state management in the referral stats loading process to prevent race conditions. - Updated the Onboarding component to handle referral step skipping more effectively, ensuring a smoother user experience. - Fixed a typo in the WelcomeStep component's button label for clarity. - Enhanced error handling in the referral API to provide clearer feedback on failures. These changes improve the usability and reliability of the referral system and onboarding experience.	2026-04-08 20:16:46 +05:30
Steven EnamakelandGitHub	4ee518cf31	feat(tree-summarizer): hierarchical summary tree module with CLI (#423) ... * feat(tree-summarizer): implement hierarchical summarization engine and event handling - Introduced a new `tree_summarizer` module to manage hierarchical time-based summaries, organizing data into a tree structure (root → year → month → day → hour). - Added functionality to ingest raw content, summarize it into hour leaves, and propagate summaries upward through the tree. - Implemented event handling for summarization completion and tree rebuild events, enhancing observability and modularity. - Created RPC operations for ingesting content, triggering summarization, querying the tree, and retrieving tree status. - Added comprehensive tests to ensure the reliability of the summarization process and event handling. This update significantly enhances the summarization capabilities of the system, allowing for efficient data organization and retrieval. * feat(tree-summarizer): add CLI support for tree summarization commands - Introduced a new `tree-summarizer` command to the CLI, allowing users to ingest content, run summarization jobs, query the summary tree, check status, and rebuild the tree. - Updated the CLI help documentation to include the new command and its subcommands. - Added a new module `tree_summarizer_cli` to encapsulate the tree summarization functionality. This enhancement improves the usability of the summarization features, providing a streamlined interface for managing hierarchical summaries directly from the command line. * style: apply cargo fmt to tree_summarizer module * feat(tree-summarizer): implement TreeSummarizerEventSubscriber for observability logging - Added a new `TreeSummarizerEventSubscriber` to log events related to tree summarization, enhancing observability. - Updated the `start_channels` function to register the new subscriber. - Refactored the `run_summarization` function to group buffered entries by hour and publish events upon completion of summarization. - Improved documentation and added tests for the new subscriber functionality. This update aims to provide better insights into the summarization process and facilitate future cross-module workflows. * refactor(tree_summarizer): streamline buffer backup and function signature - Simplified the buffer backup process by consolidating the rename operation with context handling for better error reporting. - Cleaned up the function signature of `derive_node_ids_from_hour_id` for improved readability. These changes enhance code clarity and maintainability within the tree summarization engine. * feat(tree_summarizer): enhance tree summarization with metadata support - Updated the `tree_summarizer_ingest` function to accept an optional metadata parameter, allowing users to include additional context during content ingestion. - Refactored related functions to validate and handle metadata, improving the overall robustness of the summarization process. - Adjusted the buffer write functionality to store metadata alongside content, enhancing the data structure for future retrieval and processing. These changes aim to enrich the summarization capabilities and provide more context for ingested content. * refactor(tree_summarizer): improve error message formatting in node ID validation - Enhanced the formatting of error messages in the `validate_node_id` function for better readability and consistency. - Adjusted the string formatting to use multi-line syntax, improving clarity in error reporting. - Minor formatting changes in the `strip_buffer_frontmatter` function to enhance code readability. These changes aim to improve the maintainability and clarity of error handling within the tree summarization module. * refactor(tree_summarizer): enhance buffer management and summarization process - Replaced the buffer draining mechanism with a non-destructive read approach, allowing for safer data handling during summarization. - Introduced a new `buffer_delete` function to explicitly manage the deletion of buffer entries after successful processing. - Updated the `run_summarization` function to reflect these changes, ensuring that buffer entries are only deleted after durable writes are confirmed. - Improved the backup process for the buffer directory during tree rebuilds, ensuring it is preserved outside the tree structure. These modifications aim to improve data integrity and clarity in the summarization workflow. * refactor(tree_summarizer): improve markdown parsing and timestamp handling - Updated the `parse_node_markdown` function to trim trailing whitespace from the body after splitting frontmatter, enhancing data cleanliness. - Modified test cases to use specific timestamps instead of the current time, ensuring consistent and predictable test results. - Adjusted assertions in tests to reflect the new timestamp-based ordering of entries. These changes aim to improve the robustness of markdown parsing and the reliability of test outcomes in the tree summarization module.	2026-04-08 01:56:25 -07:00
Steven EnamakelandGitHub	cd2a4a9a87	feat(screen-intelligence): OCR-only mode without vision model (#424) ... * feat(settings): add 'Use Vision Model' option to Screen Intelligence Panel - Introduced a new checkbox in the Screen Intelligence Panel to toggle the use of a vision model for richer context extraction from screenshots. - Updated state management to handle the new option and integrated it into the configuration and processing logic. - Adjusted related tests and configurations to support the new feature, ensuring compatibility across the application. * feat(cli): add --no-vision-model option for screen intelligence - Introduced a new command-line option `--no-vision-model` to allow users to skip the vision model and use OCR and text LLM only. - Updated the CLI options parsing to handle the new flag and modified the bootstrap logic to respect this setting. - Enhanced usage documentation to reflect the new option and its alias `--ocr-only` for clarity. * fix(screen-intelligence): read use_vision_model from engine runtime config The processing worker was reading use_vision_model from the persisted config file (Config::load_or_init), so the CLI --no-vision-model flag had no effect. Now reads from the engine's in-memory runtime config which the CLI correctly overrides via apply_config(). Also moves image compression before OCR pass. * fix: add use_vision_model to test fixtures and fix rustfmt Add the new use_vision_model field to all AccessibilityConfig test fixtures so TypeScript compilation passes. Also includes rustfmt auto-fix for screen_intelligence_cli.rs.	2026-04-08 01:27:59 -07:00
YellowSnnowmannandGitHub	fb8987bcad	Improve inline autocomplete reliability, sanitization, and debug logging (#407) ... * Enhance autocomplete functionality and logging. Increased debounce time for autocomplete suggestions and added minimum context character requirement. Improved inline suggestion handling with new cleanup logic for tab acceptance. Introduced a new logging option for autocomplete-only logs in CLI. Updated various components to support these changes, including sanitization and error handling in the autocomplete engine. * Add autocomplete CLI adapter for improved argument handling This commit introduces a new module, , which encapsulates the argument parsing and logging logic specific to the autocomplete namespace in the CLI. Key features include extraction of leading verbose flags, handling of the flag, and improved help message printing. The existing CLI command handling has been refactored to utilize this new adapter, enhancing code organization and maintainability. * Refactor inline completion sanitization and enhance context handling	2026-04-07 09:11:42 -07:00
Steven EnamakelandGitHub	4afa751024	feat(memory): global singleton, CLI, graph extraction fixes & light storage (#383) ... * feat(memory): add CLI support for memory commands - Introduced a new `memory` subcommand in the CLI for memory ingestion, graph inspection, and debugging. - Implemented various subcommands including `ingest`, `docs`, `graph`, `query`, and `namespaces` for comprehensive memory management. - Updated the CLI entry point to route the `memory` command appropriately, enhancing the command-line interface functionality. * refactor(memory_cli): streamline memory command ingestion and improve error handling - Simplified the ingestion process by removing unnecessary workspace directory creation and embedding logic. - Updated the ingestion function to utilize the `create_memory_client` for better client management. - Changed the limit parameter type from `usize` to `u32` for consistency and improved error handling in command arguments. - Enhanced logging for ingestion start to focus on model name only, removing redundant extraction mode information. * refactor(memory): implement global memory client singleton for improved resource management - Introduced a new `global.rs` module to manage a process-global memory client singleton, ensuring consistent access across subsystems. - Updated `create_memory_client` to utilize the global client, enhancing memory management and reducing resource contention. - Refactored various modules to replace local memory client instances with the global singleton, improving performance and reliability. - Adjusted CLI and screen intelligence components to leverage the global memory client for document persistence and ingestion operations. This refactor enhances the architecture by centralizing memory client management, leading to better resource utilization and simplified code structure. * feat(memory): add put_doc_light for screen-intelligence, skip vectors/graph Screen-intelligence captures are too frequent and ephemeral to justify vector embedding and GLiNER graph extraction per frame. Adds a lightweight storage path (put_doc_light) that persists the document row and markdown file without chunking, embedding, or graph extraction. Three-tier storage: - put_doc_light: DB + markdown only (screen-intelligence) - put_doc: DB + markdown + vectors + background graph (skill sync) - ingest_doc: full synchronous pipeline (CLI, debugging) * style: apply cargo fmt formatting * fix: add missing window_id field in AppContext test helper * fix(test): fall back to per-call MemoryClient when global not initialized In tests with isolated OPENHUMAN_WORKSPACE, the process-global singleton may not be initialized or may point at the wrong directory. Fall back to creating a client from Config (which respects env vars) when the global is not ready. * fix(test): use per-call MemoryClient for screen-intelligence persistence put_doc_light does no background work (no vectors, no graph), so a per-call client created from Config is safe and avoids the global singleton which may point at a different workspace in test suites.	2026-04-06 22:45:08 -07:00
3 people	a98817917c	feat(screen-intelligence): standalone server with OCR + vision pipeline (#382) ... * feat(screen-intelligence): add new commands for diagnostics and vision processing - Introduced `doctor` command for system readiness diagnostics, checking permissions and platform support. - Added `vision` command to analyze frames and persist vision summaries. - Updated CLI usage documentation to reflect new command options and improved verbosity handling. - Enhanced the `run_server` function to provide detailed endpoint information for better user guidance. This update improves the functionality and usability of the screen intelligence CLI, enabling better diagnostics and vision processing capabilities. * feat(screen-intelligence): update CLI endpoints for status monitoring - Added `tower-http` dependency for enhanced HTTP capabilities. - Updated endpoint documentation in `run_server` to reflect changes from SSE to long-polling for status updates. - Renamed `/events` endpoint to `/watch` with a query parameter for interval control, improving clarity and usability. This update enhances the screen intelligence CLI by providing more flexible status monitoring options. * feat(screen-intelligence): integrate standalone server for screen intelligence - Added a new `server` module to handle the standalone screen intelligence server functionality. - Updated CLI commands to include an `--auto-start` option for initiating capture sessions on server boot. - Enhanced the `run_server` function to provide detailed endpoint information and improved logging for server status. - Integrated the screen intelligence engine with JSON-RPC and REST endpoints for better debugging and usability. This update significantly enhances the screen intelligence capabilities by allowing it to run independently and providing more flexible configuration options. * refactor(screen-intelligence): simplify CLI options and server configuration - Removed the `--port` and `--auto-start` options from the CLI for the `screen-intelligence run` command, streamlining the command usage. - Updated the server configuration to focus on session duration and logging, enhancing clarity and usability. - Adjusted the documentation to reflect the new command structure and improved logging details for the screen intelligence server. This refactor improves the user experience by simplifying command options and enhancing the clarity of server operations. * feat(screen-intelligence): enhance screenshot management and CLI options - Updated the CLI usage documentation to include the new `--keep` option for retaining screenshots after processing. - Modified the server configuration to support the `keep_screenshots` flag, allowing for immediate saving of screenshots to disk. - Enhanced the `run_server` function to reflect the updated configuration and logging details regarding screenshot retention. - Improved the capture logic to prioritize window ID for more reliable screenshot capturing on macOS. These changes improve the usability and functionality of the screen intelligence feature by providing better control over screenshot management. * refactor(accessibility): improve capture logic and window ID handling - Updated the capture mode logic to prioritize reliable window ID capture on macOS, falling back to fullscreen capture to avoid issues with region-based capture. - Refactored the method for resolving the frontmost window ID to utilize Swift for better performance and reliability, replacing the previous JavaScript-based approach. - Enhanced the frame processing in the accessibility engine to track processed timestamps, preventing re-analysis of the same screenshot and improving efficiency. - Adjusted the vision summary parsing to support both JSON and plain text formats, ensuring better compatibility and usability. These changes enhance the reliability and performance of the accessibility features, particularly in macOS environments. * feat(accessibility): integrate Apple Vision OCR and enhance LLM context analysis - Implemented Apple Vision OCR for extracting text from screenshots, improving accuracy and reducing hallucination. - Refactored the vision processing flow to include structured prompts for LLM context analysis, enhancing the quality of the output summary. - Updated the vision summary structure to include detailed fields such as APP, DOING, FOCUS, and MOOD, providing clearer insights into the captured content. - Improved error handling and logging for image processing and OCR operations, ensuring better reliability and debugging capabilities. These changes significantly enhance the accessibility engine's ability to analyze and summarize screen content effectively. * refactor(screen-intelligence): enhance capture logic and summary structure - Improved capture mode handling to prioritize window ID availability, with a fallback to bounds-based capture before defaulting to fullscreen. - Updated the vision summary structure to clearly separate synthesized summaries, visual context, and raw OCR text for better clarity and usability. - Enhanced logging to provide more informative output during capture operations, aiding in debugging and performance monitoring. These changes improve the reliability and clarity of the screen intelligence features, particularly in managing capture contexts and summarizing visual information. * refactor(accessibility): enhance capture logic to reject fullscreen fallback - Updated the screen capture logic to prevent falling back to fullscreen mode when no valid window ID or bounds are available, improving privacy and user intent. - Added detailed logging for cases where fullscreen capture is refused, aiding in debugging and understanding capture decisions. - Introduced tests to ensure that the new logic correctly rejects fullscreen capture under invalid conditions, enhancing reliability. These changes improve the accessibility engine's handling of screen captures, ensuring that only relevant content is captured. * feat(screen-intelligence): implement capture and processing workers for enhanced vision analysis - Introduced a dedicated `capture_worker` to manage screenshot capturing from the foreground context, ensuring efficient frame handling and immediate saving of screenshots when configured. - Added a `processing_worker` to analyze captured frames using Apple Vision OCR and LLM, synthesizing insights and persisting results to unified memory. - Refactored the `AccessibilityEngine` to integrate these workers, improving the overall architecture and separation of concerns in the screen intelligence module. - Enhanced logging throughout the capture and processing workflows for better debugging and performance monitoring. These changes significantly improve the screen intelligence capabilities by enabling real-time capture and analysis of visual data, enhancing user experience and functionality. * Merge remote-tracking branch 'upstream/main' into fix/screen-intgellignce-2 * refactor(screen-intelligence): improve type handling and visibility in capture and processing modules - Updated the calculation of baseline milliseconds in `capture_worker` to ensure proper type handling with floating-point division. - Made several fields in `SessionRuntime` and `EngineState` public for better accessibility within the crate. - Changed the visibility of the `analyze_frame` function in `processing_worker` to public within the crate, allowing it to be called from `engine.rs`. These changes enhance type safety and improve the modularity of the screen intelligence components, facilitating better integration and usage across the codebase. * refactor(screen-intelligence): update visibility and helper function usage in engine and processing modules - Changed the `SessionRuntime` struct to be public within the crate, enhancing accessibility. - Updated the `analyze_frame` function parameter to use a reference instead of a direct reference to the `AccessibilityEngine`, improving clarity. - Refactored calls to `persist_vision_summary` to utilize the helper function from the `super::helpers` module, promoting better organization and code reuse. These changes streamline the code structure and improve the modularity of the screen intelligence components. * feat(screen-intelligence): introduce input handling and autocomplete features - Added a new `input.rs` module to manage input actions, including keyboard/mouse automation and predictive text functionalities. - Implemented methods for handling input actions, generating autocomplete suggestions, and committing selected suggestions within the `AccessibilityEngine`. - Created a `state.rs` module to encapsulate engine state management, including session runtime and engine state structures. - Enhanced the `engine.rs` module to support session lifecycle management and integrate new input functionalities. - Introduced a `vision.rs` module for vision-related query methods, improving the overall architecture and modularity of the screen intelligence components. These changes significantly enhance user interaction capabilities and streamline the management of input actions and vision processing. * fix(screen-intelligence): update worker imports to use state module Workers imported AccessibilityEngine from engine.rs but it was moved to state.rs during the refactor. Fix the import paths. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(screen-intelligence): clean up formatting and improve readability in various modules - Reformatted print statements in `screen_intelligence_cli.rs` for better readability. - Simplified assertions in tests within `capture.rs` to enhance clarity. - Streamlined function definitions and calls in `focus.rs`, `engine.rs`, and `processing_worker.rs` for improved code organization. - Updated import statements in `mod.rs` and `server.rs` to maintain consistency and clarity. These changes enhance the overall readability and maintainability of the codebase, promoting better coding practices across the screen intelligence components. * refactor(screen-intelligence): update import paths for AccessibilityEngine and EngineState - Changed the import statements in `tests.rs` to source `AccessibilityEngine` and `EngineState` from the `state` module instead of the `engine` module. - This adjustment aligns with recent refactoring efforts to improve module organization and maintainability. These changes enhance code clarity and ensure consistency in module usage across the screen intelligence components. * fix(screen-intelligence): fix test compilation and assertions - Update test imports from engine to state module - Add mock env var path to processing_worker::analyze_frame for test support - Update parser tests for plain-text mode (non-JSON fallback) - Handle missing window_id gracefully in capture_scheduler test Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(screen-intelligence): enhance code clarity and performance in various modules - Updated command-line usage documentation in `screen_intelligence_cli.rs` to reflect new options. - Improved string handling for truncation in `run_start_session` and `run_vision` functions to use character counts instead of byte lengths, ensuring accurate truncation for multi-byte characters. - Added intentional blocking sleep in `resolve_frontmost_window_id` to minimize impact on the Tokio runtime during app switches. - Implemented a consistent default confidence score in vision processing and improved YAML escaping in `persist_vision_summary`. - Refactored session management in `capture_worker` and `engine` modules to reduce lock contention and improve performance during I/O operations. These changes enhance the overall performance, maintainability, and user experience of the screen intelligence components. * fix(tests): update confidence assertion in vision summary test - Adjusted the expected confidence value in the `parse_vision_missing_fields` test to reflect the new default confidence score of 0.8, ensuring consistency across JSON and plain-text branches. This change improves the accuracy of the test and aligns it with recent updates in the vision processing logic. * refactor(screen-intelligence): improve code formatting and readability in various modules - Enhanced string formatting for truncation in `run_vision` to improve clarity. - Reformatted variable declarations in `capture_worker` for better readability. - Updated YAML escaping logic in `helpers.rs` for consistency and clarity. These changes contribute to improved maintainability and readability of the codebase. * refactor(screen-intelligence): reorganize analyze_frame function for improved flow - Moved configuration validation to the beginning of the `analyze_frame` function to ensure proper setup before processing. - Reintroduced the Apple Vision OCR and Vision LLM steps in the correct order, enhancing the logical flow of the analysis process. These changes improve the readability and maintainability of the code, ensuring a clearer structure for future modifications. --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-06 21:16:49 -07:00
3 people	8627cee960	Feat/overlay (#378) ... * feat(autocomplete): add overlay TTL configuration to AutocompletePanel - Introduced `overlay_ttl_ms` parameter to the Autocomplete configuration, allowing users to set the overlay display duration. - Updated AutocompletePanel to include a new input field for adjusting the overlay TTL in milliseconds. - Enhanced parsing and saving logic to handle the new configuration parameter. - Added corresponding tests to ensure functionality and validate the new overlay TTL feature. This update improves user control over the autocomplete overlay behavior, enhancing the overall user experience. * Refactor accessibility code for improved readability and consistency - Simplified log statements in `precompile_helper_background` for better clarity. - Reformatted `detect_input_monitoring_permission` check in `keys.rs` for enhanced readability. - Rearranged imports in `mod.rs` to maintain consistent structure. - Improved formatting of `ElementBounds` initialization across multiple test cases in `overlay.rs` and `types.rs` for better visual alignment. - Enhanced test context creation in `types.rs` for improved clarity. These changes enhance code maintainability and readability across the accessibility module. * fix(overlay): parent core RPC, voice toggle, and debug for #342 - Pass OPENHUMAN_OVERLAY_PARENT_RPC_URL from sidecar spawn and strip inherited OPENHUMAN_CORE_PORT so the overlay no longer fights for the parent listen port. - Overlay UI uses HTTP JSON-RPC to the parent sidecar for globe, debug, and voice STT so state matches the main app; add parentCoreRpc helper mirroring legacy method aliases. - Skip embedded JSON-RPC server when parent URL is set; use OPENHUMAN_OVERLAY_EMBEDDED_CORE_PORT (default 7799) for standalone dev. - Fix screen intelligence status method name; add voice_status polling, STT section, collapsible debug summary, and connection banner when core is unreachable. - Voice capture switch gates the mic with real STT availability from voice_status. Closes #342 Made-with: Cursor * fix: address CodeRabbit review on overlay/autocomplete (PR #378) - helper: correlate JSON-RPC replies with monotonic request ids; discard mismatched lines until deadline (fixes stale response after timeout). - helper: serialize Swift compile via HELPER_COMPILE_LOCK (precompile vs first use). - Overlay Tab hint: pass tab_hint from Rust from accept_with_tab; Swift hides hint when empty. - keys: re-check Input Monitoring on an interval when denied so grant without restart works. - engine: use non-zero confidence placeholder (0.75) until inline_complete returns scores. - overlay dedupe: suppress identical badge only within 400ms, not for process lifetime. Co-authored-by: Code review feedback <noreply@github.com> Made-with: Cursor * fix(ci): resolve clippy and warning issues for Rust gates - Use match on anchor_bounds in autocomplete overlay (avoid unnecessary unwrap) - Drop unused test imports in registry_ops and rpc dispatch - Prefix unused notion_doc_id in subconscious integration test Made-with: Cursor * fix(overlay): improve parent RPC URL handling in App component - Updated the useEffect hook to manage the parent RPC URL more robustly by introducing a mounted flag to prevent state updates on unmounted components. - Added error handling to set the parent RPC URL to null in case of invocation failure, enhancing the reliability of the component's behavior. * feat(overlay): implement timeout handling for parent core RPC requests - Introduced a default timeout for parent core RPC requests, enhancing reliability by preventing indefinite waiting for responses. - Added an AbortController to manage request timeouts, throwing a specific error message when a timeout occurs. - Updated the `callParentCoreRpc` function to accept a customizable timeout parameter, improving flexibility for RPC calls. * fix(overlay): allow stopping active recording regardless of config state - Updated the main button handler in the App component to always permit stopping an active recording when the status is "listening", improving user experience and control over the recording process. - Removed redundant code that previously checked the status before stopping the recording, streamlining the logic. * feat(overlay): update Cargo.lock with new dependencies and versions - Added new packages including `alsa`, `alsa-sys`, `arboard`, `block`, `cocoa`, `core-foundation`, `core-graphics`, `coreaudio-rs`, `coreaudio-sys`, `cpal`, `crunchy`, and `dasp_sample` to enhance functionality and support for audio processing and system interactions. - Updated existing dependencies to their latest versions for improved performance and compatibility. - Modified the `show_overlay` function in `ops.rs` to include an additional parameter, enhancing the overlay display functionality. * feat(autocomplete): add overlay_ttl_ms parameter to Autocomplete interfaces - Introduced a new optional parameter `overlay_ttl_ms` to both `AutocompleteSetStyleParams` and `AutocompleteConfig` interfaces, allowing for customizable overlay timeout settings. - This enhancement improves the flexibility of the autocomplete feature by enabling developers to specify how long the overlay should remain visible. --------- Co-authored-by: Code review feedback <noreply@github.com> Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai>	2026-04-06 21:11:27 -07:00
Steven EnamakelandGitHub	3851d1ef67	fix(voice): anti-hallucination, clipboard paste, Fn key reliability (#380) ... * fix(dictation): update hotkey default value and documentation - Changed the default global hotkey for dictation from "CmdOrCtrl+Shift+D" to "Fn" in both the configuration schema and the associated documentation. - Updated the hotkey parsing function to recognize "Fn" as a valid key, enhancing the flexibility of hotkey configurations. - Added a test case to ensure the "Fn" key can be parsed correctly, improving the robustness of the hotkey handling functionality. * fix(voice): update default activation mode and hotkey in configuration - Changed the default activation mode for voice and dictation from "tap" to "push" in the respective configuration schemas. - Updated the default hotkey for voice commands from "ctrl+shift+space" to "Fn" across various modules and documentation. - Adjusted related tests to reflect the new defaults, ensuring consistency in behavior and expectations. * feat(voice): integrate embedded global voice server startup - Added a new asynchronous function `start_if_enabled` to the voice server module, which initializes the embedded voice server based on configuration settings. - Updated the server run logic to check if the voice server should auto-start, enhancing the startup process for the core application. - Integrated the new server startup function into the main server run logic, ensuring the voice server is launched if enabled in the configuration. * feat(voice): add VoicePanel for managing voice server settings - Introduced a new `VoicePanel` component to handle voice server configurations, including startup options, hotkeys, and runtime controls. - Updated routing in the settings page to include the new voice settings section. - Enhanced the `useSettingsNavigation` hook to support navigation to the voice settings. - Added tests for the `VoicePanel` to ensure functionality and reliability of the voice server management features. * refactor(dictation): update documentation and improve component initialization - Revised comments in `DictationHotkeyManager` to clarify the component's mounting process within the app tree. - Removed unused imports and unnecessary state management from `ServiceBlockingGate`, streamlining the component's logic. - Updated tests for `ServiceBlockingGate` to reflect changes in behavior, ensuring accurate rendering of child components based on service status. - Enhanced the `Cargo.lock` file by updating dependencies to their latest versions for improved stability and security. * fix(voice): update default skip_cleanup setting and enhance VoicePanel options - Changed the default value of `skip_cleanup` in the voice server configuration from `false` to `true` to improve transcription handling. - Reordered options in the `VoicePanel` component to ensure "Natural cleanup" is displayed alongside "Verbatim transcription" for better user clarity. - Updated tests to reflect the new default settings and ensure proper functionality of the VoicePanel component. * feat(window): add window management commands for Tauri application - Introduced a new module `window.ts` containing functions for managing window visibility and state in a Tauri application. - Implemented commands to show, hide, toggle visibility, minimize, maximize, close, and set the title of the main window. - Added checks to ensure commands are only executed in a Tauri environment, enhancing compatibility with web contexts. * feat(tauriCommands): add comprehensive Tauri command modules - Introduced multiple new modules for Tauri commands, including `accessibility`, `autocomplete`, `config`, `conscious`, `core`, `cron`, `hardware`, `localAi`, and `window`. - Each module contains functions for managing specific functionalities such as accessibility permissions, autocomplete suggestions, configuration settings, and hardware interactions. - Implemented checks to ensure commands are executed only in a Tauri environment, enhancing compatibility and reliability. - This addition significantly expands the command capabilities of the Tauri application, providing a robust framework for future development. * feat(voice): enhance audio transcription with initial prompt support - Added functionality to transcribe audio with an optional initial prompt, allowing for vocabulary bias and improved conversational continuity. - Updated the `transcribe_pcm_f32` and `transcribe_wav_file` functions to accept an `initial_prompt` parameter, enhancing recognition of specific vocabulary. - Implemented peak RMS energy tracking during audio recording for silence detection, ensuring recordings below a defined threshold are skipped. - Enhanced the voice server configuration to include a silence threshold and custom dictionary for better transcription context. - Introduced methods to build and manage recent transcripts for improved continuity across consecutive recordings. - Updated tests to validate new features and ensure proper functionality of the transcription process. * feat(voice): enhance VoiceServerConfig with silence detection and custom dictionary - Added `silence_threshold` to the `VoiceServerConfig` for improved silence detection, allowing recordings with low RMS energy to be skipped. - Introduced `custom_dictionary` to bias transcription towards specific vocabulary, enhancing recognition of names and technical terms. - Updated the `voice_transcribe` and `voice_transcribe_bytes` functions to utilize the new `initial_prompt` parameter for better context during transcription. - Adjusted the `transcribe_pcm_i16` function to accept additional parameters for improved flexibility in handling audio input. * feat(voice): add silence threshold and custom dictionary features to VoicePanel - Implemented a new input for setting the silence threshold, allowing recordings with low RMS energy to be skipped. - Added functionality for a custom dictionary, enabling users to add specific vocabulary words to improve transcription accuracy. - Updated the VoiceServerSettings interface and related functions to support the new features, ensuring seamless integration with existing settings. - Enhanced the UI in the VoicePanel to facilitate user interaction with the new settings. * feat(voice): propagate silence threshold and custom dictionary to voice server command - Added `silence_threshold` and `custom_dictionary` parameters to the `run_voice_server_command` function, ensuring these settings are utilized during voice server operations. - Enhanced integration with the existing voice server configuration to support improved transcription accuracy and silence detection. * fix(tauriCommands): update import paths for coreRpcClient - Adjusted import paths for `callCoreRpc` in multiple Tauri command modules to ensure correct referencing from the updated directory structure. - This change enhances module organization and maintains consistency across the codebase. * feat(dependencies): update Cargo.lock and Cargo.toml for new packages and versions - Added new dependencies including `arboard`, `fax`, `fax_derive`, `gethostname`, `half`, `quick-error`, `tiff`, and `x11rb` to enhance functionality and support for clipboard operations, transcription improvements, and system interactions. - Updated existing dependencies to their latest versions for better performance and compatibility. - Modified `VoicePanel` to utilize the updated settings and ensure proper handling of voice server configurations. - Enhanced the text input mechanism to use clipboard-paste for improved reliability in text insertion. * fix(voice): update skip_cleanup default value and enhance logging - Changed the default value of `skip_cleanup` in `VoiceServerConfig` from `true` to `false` to align with expected behavior and improve transcription handling. - Added detailed logging in the transcription cleanup process to provide better insights into the LLM state and cleanup decisions. - Removed unused functions related to unreliable key releases in hotkey handling to simplify the codebase. - Updated tests to reflect the new default settings for `skip_cleanup` and ensure proper functionality across components. * style: apply linter formatting fixes * fix(voice): remove unused warn import in hotkey module * test(voice): add silence threshold and custom dictionary to VoicePanel tests - Updated tests for the VoicePanel component to include new parameters: `silence_threshold` and `custom_dictionary`. - Ensured that the tests reflect the latest configuration settings for improved transcription accuracy and functionality.	2026-04-06 18:15:48 -07:00
4 people	11f718d8bc	feat(voice): standalone voice dictation server with hotkey support (#368) ... * feat: add standalone voice dictation server with hotkey support - Introduced a new `voice` subcommand to the CLI for running a standalone voice dictation server that listens for a hotkey, records audio, transcribes it using Whisper, and inserts the result into the active text field. - Implemented configuration options for the voice server, including hotkey combination, activation mode (tap or push), and an option to skip LLM post-processing. - Added audio capture functionality using the `cpal` crate and integrated hotkey listening with the `rdev` crate for global key event handling. - Enhanced the configuration schema to include voice server settings and updated the main configuration structure accordingly. - Updated relevant modules and tests to ensure consistent behavior and functionality across the application. This feature enhances user interaction by allowing voice dictation directly into any active text field, improving accessibility and usability. * feat: add voice dictation server with hotkey support - Introduced a standalone voice dictation server that listens for a configurable hotkey to start recording audio, transcribes it using whisper, and inserts the transcribed text into the active text field. - Added CLI support for the `voice` command, allowing users to manage the voice server's configuration, including hotkey and activation mode settings. - Implemented configuration structures for the voice server, including options for automatic start, hotkey combination, activation mode, and cleanup behavior. - Enhanced audio capture functionality using the `cpal` library for microphone input and integrated text insertion using the `enigo` library for simulating keyboard input. - Updated relevant modules and schemas to support the new voice server features, ensuring a cohesive integration within the OpenHuman platform. * refactor: streamline voice server command and enhance audio capture functionality - Updated the `run_voice_server_command` function to initialize the configuration with environment overrides instead of loading from a file, improving performance and flexibility. - Refactored the audio capture logic in `start_recording` to enhance thread management and error handling, ensuring a more robust audio stream setup. - Improved the handling of audio stream creation and playback, ensuring that all cpal objects are managed on the same thread as required, enhancing stability during recording operations. * fix: remove unused import in voice server module - Eliminated the `HotkeyListenerHandle` import from the `server.rs` file, streamlining the code and improving clarity by removing unnecessary dependencies. * feat(voice): auto-enable LLM cleanup when local model is ready The postprocessor now checks the local LLM state and automatically enables transcription cleanup when the model is downloaded and ready, even if not explicitly configured. Falls back gracefully to raw text when the LLM is unavailable. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: apply cargo fmt + prettier formatting Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(voice): dictation config, hotkey lifecycle, and WebSocket streaming (#332) Add the foundational infrastructure for voice dictation (EPIC #332): **Rust core:** - New `DictationConfig` schema with serde defaults and env var overrides (enabled, hotkey, activation_mode, llm_refinement, streaming, interval) - RPC controllers: `config_get_dictation_settings` / `config_update_dictation_settings` - WebSocket endpoint `/ws/dictation` for streaming PCM16 transcription with periodic partial inference and final LLM refinement - Microphone permission declaration (`NSMicrophoneUsageDescription`) in Tauri macOS bundle config **Frontend:** - `useDictationHotkey` hook: fetches config from core RPC, auto-registers global hotkey, listens for `dictation://toggle` events - `DictationHotkeyManager` headless component mounted in App.tsx - Fix voice RPC response type mismatch: voice handlers return flat results (no `{result, logs}` wrapper), so remove incorrect `CommandResponse<T>` wrapping from `openhumanVoiceStatus`, `openhumanVoiceTranscribe`, `openhumanVoiceTranscribeBytes`, and `openhumanVoiceTts` Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(tauri): remove invalid infoPlist config that breaks tauri dev The `infoPlist` field in tauri.conf.json expects a string path, not an inline object. Remove it for now — microphone permission will be added via a proper Info.plist supplement in the production build pipeline. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * style: apply Prettier formatting to dictation files Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * format files * feat(dictation): integrate dictation listener and event broadcasting - Added a global dictation hotkey listener that activates based on configuration. - Implemented a web channel bridge to handle dictation events and broadcast them to connected clients. - Updated the voice module to include the new dictation listener functionality. This enhances the voice dictation capabilities by ensuring real-time event handling and client communication. * update code * format * feat(voice): enhance voice server configuration and functionality - Updated `Cargo.toml` to mark voice-related dependencies as optional. - Introduced `VoiceActivationMode` enum for better control over voice server activation. - Refactored voice server command handling and dictation event broadcasting to support new features. - Added conditional compilation for voice features across various modules, ensuring they are only included when enabled. This commit improves the modularity and configurability of the voice server, allowing for more flexible integration and usage. * refactor: clean up whitespace and formatting in core and voice modules - Removed unnecessary blank lines in `cli.rs`, `jsonrpc.rs`, `schemas.rs`, and `socketio.rs` to improve code readability. - Adjusted import order in `mod.rs` for better organization. This commit enhances the overall code quality by ensuring consistent formatting across multiple files. * chore: update Dockerfile and test workflow to install additional system dependencies - Added installation of system dependencies (cmake, ALSA, X11) in the Dockerfile for improved build support. - Updated the GitHub Actions workflow to reflect the new dependencies, ensuring consistent environment setup for testing. This commit enhances the build environment by including necessary libraries for audio and GUI support. * format * fix claude * format --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: oxoxDev <nikhil@tinyhumans.ai>	2026-04-06 16:08:52 -07:00
3 people	07b1df4f24	feat(event_bus): wire webhooks, channels & skills through the event bus (#379) ... * feat(event_bus): enhance domain event handling across modules - Added new `DomainEvent` variants for channel and skill events, including `ChannelMessageReceived`, `ChannelMessageProcessed`, `ChannelConnected`, `ChannelDisconnected`, `SkillLoaded`, `SkillStopped`, and `SkillStartFailed`. - Implemented event publishing in the channels and skills modules to track message processing and skill lifecycle events. - Created dedicated event bus handler files for the skills and webhooks domains, preparing for future subscriber implementations. - Updated documentation in `CLAUDE.md` to reflect the new domain events and their usage. These changes improve the observability and modularity of the system by leveraging an event-driven architecture for cross-module communication. * feat(event_bus): implement channel and webhook event handling - Introduced `ChannelInboundSubscriber` to handle inbound channel messages, triggering the agent inference loop and sending replies via the backend REST API. - Added `WebhookRequestSubscriber` to manage incoming webhook requests, routing them to the appropriate skill and handling responses. - Updated the global event bus initialization in `bootstrap_skill_runtime` to register both channel and webhook subscribers. - Enhanced `DomainEvent` with new variants for channel inbound messages and webhook requests, improving event-driven communication across modules. These changes enhance the modularity and responsiveness of the system by leveraging an event-driven architecture for channel and webhook interactions. * refactor(event_bus): update domain event documentation and subscriber initialization - Revised the documentation in `CLAUDE.md` to provide a concise overview of domain events and their associated subscriber files, enhancing clarity for future development. - Updated the `start_channels` function to initialize `WebhookRequestSubscriber` and `ChannelInboundSubscriber`, ensuring proper event handling for webhooks and channel messages. - Streamlined the event bus subscriber registration process, reinforcing the modular architecture of the system. These changes improve the maintainability and usability of the event bus framework, facilitating better cross-module communication. * style: apply cargo fmt formatting Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(event_bus): remove duplicate subscriber registration in start_channels WebhookRequestSubscriber and ChannelInboundSubscriber were registered in both bootstrap_skill_runtime() and start_channels(), causing events to be handled twice when both paths run in the same process. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(event_bus): prevent subscriber handles from being dropped on function exit SubscriptionHandle::drop aborts the background task. Since bootstrap_skill_runtime() returns immediately after setup, the local handles were dropped, cancelling both subscribers. Use std::mem::forget to leak the handles so the tasks live for the entire process. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(event_bus): ensure subscriber handles persist beyond function exit Modified the handling of subscriber registration to prevent premature dropping of handles in `bootstrap_skill_runtime()`. This change ensures that the background tasks for subscribers remain active for the entire process lifecycle, enhancing event handling reliability. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(webhooks): use proper JSON serialization for error response bodies Hand-escaped JSON strings only handled double quotes, not backslashes, newlines, or other control chars. Replaced with serde_json serialization via an error_body() helper. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(webhooks): use proper JSON serialization for error response bodies Hand-escaped JSON strings only handled double quotes, not backslashes, newlines, or other control chars. Replaced with serde_json serialization via an error_body() helper. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-06 15:58:50 -07:00
3 people	45d15cd510	feat: text_input domain module for focused field read/insert/ghost text (#377) ... * feat: add text_input domain module for focused field read/insert/ghost text Thin orchestration layer over accessibility APIs — provides a clean RPC surface for reading, inserting, and previewing text in OS-focused input fields. Consumed by autocomplete and voice control. Five RPC methods: read_field, insert_text, show_ghost, dismiss_ghost, accept_ghost. Standalone CLI (`openhuman text-input`) and JSON-RPC server mode for testing. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: apply cargo fmt to text_input module Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-06 14:41:17 -07:00
3 people	88257d3231	feat(update): core sidecar self-update with version detection (#372) ... * update * update * Refactor code style for consistency and readability in core update module - Reformatted platform_triple function to improve readability by aligning braces. - Simplified async function calls and error handling in various places for better clarity. - Enhanced logging statements for improved observability during update processes. These changes enhance the maintainability of the codebase while ensuring consistent formatting across the module. * feat(update): periodic background update checker with config flag Add a periodic update scheduler that checks GitHub Releases for newer core binary versions on a configurable interval (default: 1 hour). Controlled by `[update]` config section with `enabled = true` by default. - New `UpdateConfig` in config schema (enabled, interval_minutes) - Env var overrides: OPENHUMAN_AUTO_UPDATE_ENABLED, OPENHUMAN_AUTO_UPDATE_INTERVAL_MINUTES - Background scheduler spawned at server startup in run_server() - Reports to health registry as "update_checker" component Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(update): address PR review — restart path, security, version tracking - CoreProcessHandle: add set_core_bin/effective_core_bin so ensure_running launches the newly staged binary instead of the original one - check_and_update_core: add `force` param — auto-check uses MINIMUM_CORE_VERSION, manual apply_core_update uses latest release (force=true) - Acquire restart_lock before download+staging to prevent concurrent updates; shutdown old process before staging; use unique temp filename - check_core_update Tauri command now queries GitHub for latest_version and returns update_available alongside outdated - Harden update_apply RPC: validate download URL is GitHub HTTPS, validate asset_name is safe filename starting with openhuman-core-, ignore caller staging_dir (always use safe default) - download_and_stage accepts target_version so installed_version reflects the staged release, not the running process - Add update.check and update.apply to about_app capability catalog - ops.rs: unwrap_or_default → unwrap_or_else with error context - tauriCommands.ts: convert to arrow-function exports, add latest_version and update_available to CoreUpdateStatus interface Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * format --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-06 14:21:13 -07:00
3 people	df503a23fd	Fix: decouple chat from local Ollama, route inference through Rust (#369) ... * refactor: streamline Conversations component by removing local response delivery logic - Eliminated the `deliverLocalResponse` function, which handled segmenting and dispatching local model responses, simplifying the message delivery process. - Updated socket status checks to remove unnecessary local model condition, ensuring clarity in connection handling. - Adjusted comments to reflect the new routing logic for cloud backend interactions, emphasizing that local models are now used only for supplementary features. * refactor: update chat service to utilize core RPC for message handling - Replaced Socket.IO message sending with core RPC calls for sending and canceling chat messages, enhancing the communication mechanism. - Improved error handling for socket connection checks, ensuring robust event routing. - Updated documentation to reflect the new RPC-based approach for chat message processing. * feat: enhance chat response handling with segmentation and emoji reactions - Introduced a new `ChatSegmentEvent` interface to support segmented message delivery, allowing for more natural chat interactions. - Updated the `Conversations` component to handle segmented responses and apply emoji reactions based on user messages. - Refactored the message delivery logic to improve clarity and maintainability, ensuring that responses are dispatched correctly based on segmentation. - Enhanced the chat service to emit segment events, facilitating a smoother user experience during conversations. * refactor: simplify Conversations component by removing unused imports and optimizing rendering logic - Removed unnecessary imports related to local model status and message segmentation, streamlining the codebase. - Updated the rendering logic to enhance clarity by simplifying conditions for displaying sending and delivering states in the chat interface. - Improved overall readability and maintainability of the Conversations component. * style: apply formatter fixes (prettier + cargo fmt) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * updated convos * fix test --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-06 12:25:10 -07:00
3 people	b589a29e77	Extract socket controller into dedicated domain module (#340) ... * refactor: remove global_engine usage in tests and instantiate AccessibilityEngine directly - Updated the test suite to eliminate the use of global_engine, replacing it with a direct instantiation of AccessibilityEngine. - This change enhances test isolation and clarity by ensuring that each test operates with its own instance of the engine, improving reliability and maintainability. * feat: add socket module for skill communication - Introduced a new `socket` module to facilitate communication between skills, enhancing the modularity and organization of the codebase. - Updated imports in `qjs_engine.rs` to reference the new `SocketManager` from the `socket` module, streamlining socket management for skill interactions. - Removed the deprecated `socket_manager` module from the skills module, improving clarity and reducing redundancy in the code structure. * feat: integrate socket controllers and schemas into core functionality - Added socket-related registered controllers and schemas to the core build functions, enhancing the communication capabilities within the OpenHuman framework. - Updated the `build_registered_controllers` and `build_declared_controller_schemas` functions to include socket components, ensuring comprehensive integration of the new socket module. * feat: enhance QuickJS skill runtime with socket manager integration - Updated the `bootstrap_skill_runtime` function to initialize and register the `SocketManager` globally, allowing RPC handlers to access socket functionalities. - Improved documentation to reflect the addition of socket management capabilities alongside the QuickJS skill runtime. - Cleaned up imports in `event_handlers.rs` to streamline the codebase. * refactor: remove socket manager integration from skill runtime - Removed the `SocketManager` integration from the `bootstrap_skill_runtime` function, simplifying the socket management process. - Eliminated the `socket_manager` field and related methods from the `RuntimeEngine` struct, streamlining the codebase. - Cleaned up unused MCP handlers and socket-related imports in the event handlers, enhancing code clarity and maintainability. * refactor: remove sync_tools calls from skill status handling - Eliminated unnecessary calls to `sync_tools()` in the `RuntimeEngine` during skill status changes, simplifying the skill lifecycle management. - This change enhances performance by reducing redundant synchronization operations during skill execution and shutdown processes. * feat: enhance socket event handling with improved logging - Added logging for incoming socket events to improve observability, including event name and data size. - Implemented detailed debug logging for event payloads, ensuring clarity on the data being processed. - Updated event handling logic to streamline routing for webhook requests and inbound channel messages, enhancing the overall responsiveness of the system. - Introduced logging for unhandled events to aid in debugging and monitoring. * feat: enhance socket management and auto-connect functionality - Updated the `bootstrap_skill_runtime` function to clone the `SocketManager` instance for global registration, ensuring proper socket management. - Introduced background tasks for auto-starting skills and auto-connecting to the backend using stored session tokens, improving startup efficiency and user experience. - Added detailed logging for session token checks and connection attempts, enhancing observability and debugging capabilities during socket operations. * feat: enhance skill selection and tool management in tests - Introduced a new `manifests_in_dir` function to retrieve skill manifests from a specified directory, improving skill discovery. - Added `select_skill_id` function to prioritize skill selection based on environment variables and preferred candidates, enhancing flexibility in test configurations. - Updated the test suite to utilize the new skill selection logic, ensuring more robust and configurable test scenarios. - Improved handling of tool selection with enhanced logging and fallback mechanisms for better debugging and usability. * format code:wq * feat: add Rust checks and formatting commands to package.json - Introduced new scripts for Rust checks and formatting in both the main and app package.json files. - Updated pre-push hook to include Rust compile checks, enhancing pre-push validation. - Modified existing format commands to integrate Rust formatting, ensuring consistency across codebases. * fix: improve formatting of the "Keep Screenshots" label description - Adjusted the formatting of the description text for better readability by breaking it into multiple lines within the `ScreenIntelligencePanel` component. * fix: install rustls crypto provider before WebSocket TLS connect The socket auto-connect was panicking with "Could not automatically determine the process-level CryptoProvider" because tokio-tungstenite uses rustls for wss:// but no crypto provider was installed. Install the ring provider before each connect attempt. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use dynamically selected skill in sync memory tests The tests hardcoded 'example-skill' which no longer exists in the skills directory. Now dynamically picks the first available skill (preferring server-ping) so tests work regardless of which skills are present. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-05 22:53:58 -07:00