mirror of
https://github.com/tinyhumansai/openhuman.git
synced 2026-07-27 21:08:00 +00:00
7175a2ba9cc6cde721b5a4b6fb520bf1f08c7e6d
1000
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Steven Enamakel
|
7175a2ba9c | update | ||
|
Steven Enamakel
|
97e0974c0c | Merge remote-tracking branch 'upstream/main' | ||
|
Steven Enamakel
|
6b1b2b9ba2 | update | ||
|
github-actions[bot]
|
3361ec040f | chore(release): v0.51.15 | ||
|
github-actions[bot]
|
4dd9da88a9 | chore(release): v0.51.14 | ||
|
github-actions[bot]
|
621f862095 | chore(release): v0.51.13 | ||
|
Steven Enamakel
|
d05b724fd3 | Merge remote-tracking branch 'upstream/main' | ||
|
github-actions[bot]
|
3ac6bc6a8a | chore(release): v0.51.12 | ||
|
Steven Enamakel
|
cb0cec0c70 | fix | ||
|
Steven Enamakel
|
39ce04a02d | Merge remote-tracking branch 'upstream/main' | ||
|
github-actions[bot]
|
c8c0f8bb5d | chore(release): v0.51.11 | ||
|
Steven Enamakel
|
8bda914576 | ifix buils | ||
|
github-actions[bot]
|
fbbdc07062 | chore(release): v0.51.10 | ||
|
github-actions[bot]
|
766ad5af48 | chore(release): v0.51.9 | ||
|
Steven EnamakelandGitHub
|
4afa751024 |
feat(memory): global singleton, CLI, graph extraction fixes & light storage (#383)
* feat(memory): add CLI support for memory commands - Introduced a new `memory` subcommand in the CLI for memory ingestion, graph inspection, and debugging. - Implemented various subcommands including `ingest`, `docs`, `graph`, `query`, and `namespaces` for comprehensive memory management. - Updated the CLI entry point to route the `memory` command appropriately, enhancing the command-line interface functionality. * refactor(memory_cli): streamline memory command ingestion and improve error handling - Simplified the ingestion process by removing unnecessary workspace directory creation and embedding logic. - Updated the ingestion function to utilize the `create_memory_client` for better client management. - Changed the limit parameter type from `usize` to `u32` for consistency and improved error handling in command arguments. - Enhanced logging for ingestion start to focus on model name only, removing redundant extraction mode information. * refactor(memory): implement global memory client singleton for improved resource management - Introduced a new `global.rs` module to manage a process-global memory client singleton, ensuring consistent access across subsystems. - Updated `create_memory_client` to utilize the global client, enhancing memory management and reducing resource contention. - Refactored various modules to replace local memory client instances with the global singleton, improving performance and reliability. - Adjusted CLI and screen intelligence components to leverage the global memory client for document persistence and ingestion operations. This refactor enhances the architecture by centralizing memory client management, leading to better resource utilization and simplified code structure. * feat(memory): add put_doc_light for screen-intelligence, skip vectors/graph Screen-intelligence captures are too frequent and ephemeral to justify vector embedding and GLiNER graph extraction per frame. Adds a lightweight storage path (put_doc_light) that persists the document row and markdown file without chunking, embedding, or graph extraction. Three-tier storage: - put_doc_light: DB + markdown only (screen-intelligence) - put_doc: DB + markdown + vectors + background graph (skill sync) - ingest_doc: full synchronous pipeline (CLI, debugging) * style: apply cargo fmt formatting * fix: add missing window_id field in AppContext test helper * fix(test): fall back to per-call MemoryClient when global not initialized In tests with isolated OPENHUMAN_WORKSPACE, the process-global singleton may not be initialized or may point at the wrong directory. Fall back to creating a client from Config (which respects env vars) when the global is not ready. * fix(test): use per-call MemoryClient for screen-intelligence persistence put_doc_light does no background work (no vectors, no graph), so a per-call client created from Config is safe and avoids the global singleton which may point at a different workspace in test suites. |
||
|
|
a98817917c |
feat(screen-intelligence): standalone server with OCR + vision pipeline (#382)
* feat(screen-intelligence): add new commands for diagnostics and vision processing - Introduced `doctor` command for system readiness diagnostics, checking permissions and platform support. - Added `vision` command to analyze frames and persist vision summaries. - Updated CLI usage documentation to reflect new command options and improved verbosity handling. - Enhanced the `run_server` function to provide detailed endpoint information for better user guidance. This update improves the functionality and usability of the screen intelligence CLI, enabling better diagnostics and vision processing capabilities. * feat(screen-intelligence): update CLI endpoints for status monitoring - Added `tower-http` dependency for enhanced HTTP capabilities. - Updated endpoint documentation in `run_server` to reflect changes from SSE to long-polling for status updates. - Renamed `/events` endpoint to `/watch` with a query parameter for interval control, improving clarity and usability. This update enhances the screen intelligence CLI by providing more flexible status monitoring options. * feat(screen-intelligence): integrate standalone server for screen intelligence - Added a new `server` module to handle the standalone screen intelligence server functionality. - Updated CLI commands to include an `--auto-start` option for initiating capture sessions on server boot. - Enhanced the `run_server` function to provide detailed endpoint information and improved logging for server status. - Integrated the screen intelligence engine with JSON-RPC and REST endpoints for better debugging and usability. This update significantly enhances the screen intelligence capabilities by allowing it to run independently and providing more flexible configuration options. * refactor(screen-intelligence): simplify CLI options and server configuration - Removed the `--port` and `--auto-start` options from the CLI for the `screen-intelligence run` command, streamlining the command usage. - Updated the server configuration to focus on session duration and logging, enhancing clarity and usability. - Adjusted the documentation to reflect the new command structure and improved logging details for the screen intelligence server. This refactor improves the user experience by simplifying command options and enhancing the clarity of server operations. * feat(screen-intelligence): enhance screenshot management and CLI options - Updated the CLI usage documentation to include the new `--keep` option for retaining screenshots after processing. - Modified the server configuration to support the `keep_screenshots` flag, allowing for immediate saving of screenshots to disk. - Enhanced the `run_server` function to reflect the updated configuration and logging details regarding screenshot retention. - Improved the capture logic to prioritize window ID for more reliable screenshot capturing on macOS. These changes improve the usability and functionality of the screen intelligence feature by providing better control over screenshot management. * refactor(accessibility): improve capture logic and window ID handling - Updated the capture mode logic to prioritize reliable window ID capture on macOS, falling back to fullscreen capture to avoid issues with region-based capture. - Refactored the method for resolving the frontmost window ID to utilize Swift for better performance and reliability, replacing the previous JavaScript-based approach. - Enhanced the frame processing in the accessibility engine to track processed timestamps, preventing re-analysis of the same screenshot and improving efficiency. - Adjusted the vision summary parsing to support both JSON and plain text formats, ensuring better compatibility and usability. These changes enhance the reliability and performance of the accessibility features, particularly in macOS environments. * feat(accessibility): integrate Apple Vision OCR and enhance LLM context analysis - Implemented Apple Vision OCR for extracting text from screenshots, improving accuracy and reducing hallucination. - Refactored the vision processing flow to include structured prompts for LLM context analysis, enhancing the quality of the output summary. - Updated the vision summary structure to include detailed fields such as APP, DOING, FOCUS, and MOOD, providing clearer insights into the captured content. - Improved error handling and logging for image processing and OCR operations, ensuring better reliability and debugging capabilities. These changes significantly enhance the accessibility engine's ability to analyze and summarize screen content effectively. * refactor(screen-intelligence): enhance capture logic and summary structure - Improved capture mode handling to prioritize window ID availability, with a fallback to bounds-based capture before defaulting to fullscreen. - Updated the vision summary structure to clearly separate synthesized summaries, visual context, and raw OCR text for better clarity and usability. - Enhanced logging to provide more informative output during capture operations, aiding in debugging and performance monitoring. These changes improve the reliability and clarity of the screen intelligence features, particularly in managing capture contexts and summarizing visual information. * refactor(accessibility): enhance capture logic to reject fullscreen fallback - Updated the screen capture logic to prevent falling back to fullscreen mode when no valid window ID or bounds are available, improving privacy and user intent. - Added detailed logging for cases where fullscreen capture is refused, aiding in debugging and understanding capture decisions. - Introduced tests to ensure that the new logic correctly rejects fullscreen capture under invalid conditions, enhancing reliability. These changes improve the accessibility engine's handling of screen captures, ensuring that only relevant content is captured. * feat(screen-intelligence): implement capture and processing workers for enhanced vision analysis - Introduced a dedicated `capture_worker` to manage screenshot capturing from the foreground context, ensuring efficient frame handling and immediate saving of screenshots when configured. - Added a `processing_worker` to analyze captured frames using Apple Vision OCR and LLM, synthesizing insights and persisting results to unified memory. - Refactored the `AccessibilityEngine` to integrate these workers, improving the overall architecture and separation of concerns in the screen intelligence module. - Enhanced logging throughout the capture and processing workflows for better debugging and performance monitoring. These changes significantly improve the screen intelligence capabilities by enabling real-time capture and analysis of visual data, enhancing user experience and functionality. * Merge remote-tracking branch 'upstream/main' into fix/screen-intgellignce-2 * refactor(screen-intelligence): improve type handling and visibility in capture and processing modules - Updated the calculation of baseline milliseconds in `capture_worker` to ensure proper type handling with floating-point division. - Made several fields in `SessionRuntime` and `EngineState` public for better accessibility within the crate. - Changed the visibility of the `analyze_frame` function in `processing_worker` to public within the crate, allowing it to be called from `engine.rs`. These changes enhance type safety and improve the modularity of the screen intelligence components, facilitating better integration and usage across the codebase. * refactor(screen-intelligence): update visibility and helper function usage in engine and processing modules - Changed the `SessionRuntime` struct to be public within the crate, enhancing accessibility. - Updated the `analyze_frame` function parameter to use a reference instead of a direct reference to the `AccessibilityEngine`, improving clarity. - Refactored calls to `persist_vision_summary` to utilize the helper function from the `super::helpers` module, promoting better organization and code reuse. These changes streamline the code structure and improve the modularity of the screen intelligence components. * feat(screen-intelligence): introduce input handling and autocomplete features - Added a new `input.rs` module to manage input actions, including keyboard/mouse automation and predictive text functionalities. - Implemented methods for handling input actions, generating autocomplete suggestions, and committing selected suggestions within the `AccessibilityEngine`. - Created a `state.rs` module to encapsulate engine state management, including session runtime and engine state structures. - Enhanced the `engine.rs` module to support session lifecycle management and integrate new input functionalities. - Introduced a `vision.rs` module for vision-related query methods, improving the overall architecture and modularity of the screen intelligence components. These changes significantly enhance user interaction capabilities and streamline the management of input actions and vision processing. * fix(screen-intelligence): update worker imports to use state module Workers imported AccessibilityEngine from engine.rs but it was moved to state.rs during the refactor. Fix the import paths. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(screen-intelligence): clean up formatting and improve readability in various modules - Reformatted print statements in `screen_intelligence_cli.rs` for better readability. - Simplified assertions in tests within `capture.rs` to enhance clarity. - Streamlined function definitions and calls in `focus.rs`, `engine.rs`, and `processing_worker.rs` for improved code organization. - Updated import statements in `mod.rs` and `server.rs` to maintain consistency and clarity. These changes enhance the overall readability and maintainability of the codebase, promoting better coding practices across the screen intelligence components. * refactor(screen-intelligence): update import paths for AccessibilityEngine and EngineState - Changed the import statements in `tests.rs` to source `AccessibilityEngine` and `EngineState` from the `state` module instead of the `engine` module. - This adjustment aligns with recent refactoring efforts to improve module organization and maintainability. These changes enhance code clarity and ensure consistency in module usage across the screen intelligence components. * fix(screen-intelligence): fix test compilation and assertions - Update test imports from engine to state module - Add mock env var path to processing_worker::analyze_frame for test support - Update parser tests for plain-text mode (non-JSON fallback) - Handle missing window_id gracefully in capture_scheduler test Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(screen-intelligence): enhance code clarity and performance in various modules - Updated command-line usage documentation in `screen_intelligence_cli.rs` to reflect new options. - Improved string handling for truncation in `run_start_session` and `run_vision` functions to use character counts instead of byte lengths, ensuring accurate truncation for multi-byte characters. - Added intentional blocking sleep in `resolve_frontmost_window_id` to minimize impact on the Tokio runtime during app switches. - Implemented a consistent default confidence score in vision processing and improved YAML escaping in `persist_vision_summary`. - Refactored session management in `capture_worker` and `engine` modules to reduce lock contention and improve performance during I/O operations. These changes enhance the overall performance, maintainability, and user experience of the screen intelligence components. * fix(tests): update confidence assertion in vision summary test - Adjusted the expected confidence value in the `parse_vision_missing_fields` test to reflect the new default confidence score of 0.8, ensuring consistency across JSON and plain-text branches. This change improves the accuracy of the test and aligns it with recent updates in the vision processing logic. * refactor(screen-intelligence): improve code formatting and readability in various modules - Enhanced string formatting for truncation in `run_vision` to improve clarity. - Reformatted variable declarations in `capture_worker` for better readability. - Updated YAML escaping logic in `helpers.rs` for consistency and clarity. These changes contribute to improved maintainability and readability of the codebase. * refactor(screen-intelligence): reorganize analyze_frame function for improved flow - Moved configuration validation to the beginning of the `analyze_frame` function to ensure proper setup before processing. - Reintroduced the Apple Vision OCR and Vision LLM steps in the correct order, enhancing the logical flow of the analysis process. These changes improve the readability and maintainability of the code, ensuring a clearer structure for future modifications. --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
|
8627cee960 |
Feat/overlay (#378)
* feat(autocomplete): add overlay TTL configuration to AutocompletePanel - Introduced `overlay_ttl_ms` parameter to the Autocomplete configuration, allowing users to set the overlay display duration. - Updated AutocompletePanel to include a new input field for adjusting the overlay TTL in milliseconds. - Enhanced parsing and saving logic to handle the new configuration parameter. - Added corresponding tests to ensure functionality and validate the new overlay TTL feature. This update improves user control over the autocomplete overlay behavior, enhancing the overall user experience. * Refactor accessibility code for improved readability and consistency - Simplified log statements in `precompile_helper_background` for better clarity. - Reformatted `detect_input_monitoring_permission` check in `keys.rs` for enhanced readability. - Rearranged imports in `mod.rs` to maintain consistent structure. - Improved formatting of `ElementBounds` initialization across multiple test cases in `overlay.rs` and `types.rs` for better visual alignment. - Enhanced test context creation in `types.rs` for improved clarity. These changes enhance code maintainability and readability across the accessibility module. * fix(overlay): parent core RPC, voice toggle, and debug for #342 - Pass OPENHUMAN_OVERLAY_PARENT_RPC_URL from sidecar spawn and strip inherited OPENHUMAN_CORE_PORT so the overlay no longer fights for the parent listen port. - Overlay UI uses HTTP JSON-RPC to the parent sidecar for globe, debug, and voice STT so state matches the main app; add parentCoreRpc helper mirroring legacy method aliases. - Skip embedded JSON-RPC server when parent URL is set; use OPENHUMAN_OVERLAY_EMBEDDED_CORE_PORT (default 7799) for standalone dev. - Fix screen intelligence status method name; add voice_status polling, STT section, collapsible debug summary, and connection banner when core is unreachable. - Voice capture switch gates the mic with real STT availability from voice_status. Closes #342 Made-with: Cursor * fix: address CodeRabbit review on overlay/autocomplete (PR #378) - helper: correlate JSON-RPC replies with monotonic request ids; discard mismatched lines until deadline (fixes stale response after timeout). - helper: serialize Swift compile via HELPER_COMPILE_LOCK (precompile vs first use). - Overlay Tab hint: pass tab_hint from Rust from accept_with_tab; Swift hides hint when empty. - keys: re-check Input Monitoring on an interval when denied so grant without restart works. - engine: use non-zero confidence placeholder (0.75) until inline_complete returns scores. - overlay dedupe: suppress identical badge only within 400ms, not for process lifetime. Co-authored-by: Code review feedback <noreply@github.com> Made-with: Cursor * fix(ci): resolve clippy and warning issues for Rust gates - Use match on anchor_bounds in autocomplete overlay (avoid unnecessary unwrap) - Drop unused test imports in registry_ops and rpc dispatch - Prefix unused notion_doc_id in subconscious integration test Made-with: Cursor * fix(overlay): improve parent RPC URL handling in App component - Updated the useEffect hook to manage the parent RPC URL more robustly by introducing a mounted flag to prevent state updates on unmounted components. - Added error handling to set the parent RPC URL to null in case of invocation failure, enhancing the reliability of the component's behavior. * feat(overlay): implement timeout handling for parent core RPC requests - Introduced a default timeout for parent core RPC requests, enhancing reliability by preventing indefinite waiting for responses. - Added an AbortController to manage request timeouts, throwing a specific error message when a timeout occurs. - Updated the `callParentCoreRpc` function to accept a customizable timeout parameter, improving flexibility for RPC calls. * fix(overlay): allow stopping active recording regardless of config state - Updated the main button handler in the App component to always permit stopping an active recording when the status is "listening", improving user experience and control over the recording process. - Removed redundant code that previously checked the status before stopping the recording, streamlining the logic. * feat(overlay): update Cargo.lock with new dependencies and versions - Added new packages including `alsa`, `alsa-sys`, `arboard`, `block`, `cocoa`, `core-foundation`, `core-graphics`, `coreaudio-rs`, `coreaudio-sys`, `cpal`, `crunchy`, and `dasp_sample` to enhance functionality and support for audio processing and system interactions. - Updated existing dependencies to their latest versions for improved performance and compatibility. - Modified the `show_overlay` function in `ops.rs` to include an additional parameter, enhancing the overlay display functionality. * feat(autocomplete): add overlay_ttl_ms parameter to Autocomplete interfaces - Introduced a new optional parameter `overlay_ttl_ms` to both `AutocompleteSetStyleParams` and `AutocompleteConfig` interfaces, allowing for customizable overlay timeout settings. - This enhancement improves the flexibility of the autocomplete feature by enabling developers to specify how long the overlay should remain visible. --------- Co-authored-by: Code review feedback <noreply@github.com> Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai> |
||
|
Steven EnamakelandGitHub
|
e8fd08c800 |
feat(memory): use timestamp-prefixed document IDs for chronological sorting (#381)
Replace random UUID document IDs with `{unix_seconds}_{8char_hex}` format
so memory documents sort chronologically by filename and ID.
|
||
|
Steven EnamakelandGitHub
|
3851d1ef67 |
fix(voice): anti-hallucination, clipboard paste, Fn key reliability (#380)
* fix(dictation): update hotkey default value and documentation - Changed the default global hotkey for dictation from "CmdOrCtrl+Shift+D" to "Fn" in both the configuration schema and the associated documentation. - Updated the hotkey parsing function to recognize "Fn" as a valid key, enhancing the flexibility of hotkey configurations. - Added a test case to ensure the "Fn" key can be parsed correctly, improving the robustness of the hotkey handling functionality. * fix(voice): update default activation mode and hotkey in configuration - Changed the default activation mode for voice and dictation from "tap" to "push" in the respective configuration schemas. - Updated the default hotkey for voice commands from "ctrl+shift+space" to "Fn" across various modules and documentation. - Adjusted related tests to reflect the new defaults, ensuring consistency in behavior and expectations. * feat(voice): integrate embedded global voice server startup - Added a new asynchronous function `start_if_enabled` to the voice server module, which initializes the embedded voice server based on configuration settings. - Updated the server run logic to check if the voice server should auto-start, enhancing the startup process for the core application. - Integrated the new server startup function into the main server run logic, ensuring the voice server is launched if enabled in the configuration. * feat(voice): add VoicePanel for managing voice server settings - Introduced a new `VoicePanel` component to handle voice server configurations, including startup options, hotkeys, and runtime controls. - Updated routing in the settings page to include the new voice settings section. - Enhanced the `useSettingsNavigation` hook to support navigation to the voice settings. - Added tests for the `VoicePanel` to ensure functionality and reliability of the voice server management features. * refactor(dictation): update documentation and improve component initialization - Revised comments in `DictationHotkeyManager` to clarify the component's mounting process within the app tree. - Removed unused imports and unnecessary state management from `ServiceBlockingGate`, streamlining the component's logic. - Updated tests for `ServiceBlockingGate` to reflect changes in behavior, ensuring accurate rendering of child components based on service status. - Enhanced the `Cargo.lock` file by updating dependencies to their latest versions for improved stability and security. * fix(voice): update default skip_cleanup setting and enhance VoicePanel options - Changed the default value of `skip_cleanup` in the voice server configuration from `false` to `true` to improve transcription handling. - Reordered options in the `VoicePanel` component to ensure "Natural cleanup" is displayed alongside "Verbatim transcription" for better user clarity. - Updated tests to reflect the new default settings and ensure proper functionality of the VoicePanel component. * feat(window): add window management commands for Tauri application - Introduced a new module `window.ts` containing functions for managing window visibility and state in a Tauri application. - Implemented commands to show, hide, toggle visibility, minimize, maximize, close, and set the title of the main window. - Added checks to ensure commands are only executed in a Tauri environment, enhancing compatibility with web contexts. * feat(tauriCommands): add comprehensive Tauri command modules - Introduced multiple new modules for Tauri commands, including `accessibility`, `autocomplete`, `config`, `conscious`, `core`, `cron`, `hardware`, `localAi`, and `window`. - Each module contains functions for managing specific functionalities such as accessibility permissions, autocomplete suggestions, configuration settings, and hardware interactions. - Implemented checks to ensure commands are executed only in a Tauri environment, enhancing compatibility and reliability. - This addition significantly expands the command capabilities of the Tauri application, providing a robust framework for future development. * feat(voice): enhance audio transcription with initial prompt support - Added functionality to transcribe audio with an optional initial prompt, allowing for vocabulary bias and improved conversational continuity. - Updated the `transcribe_pcm_f32` and `transcribe_wav_file` functions to accept an `initial_prompt` parameter, enhancing recognition of specific vocabulary. - Implemented peak RMS energy tracking during audio recording for silence detection, ensuring recordings below a defined threshold are skipped. - Enhanced the voice server configuration to include a silence threshold and custom dictionary for better transcription context. - Introduced methods to build and manage recent transcripts for improved continuity across consecutive recordings. - Updated tests to validate new features and ensure proper functionality of the transcription process. * feat(voice): enhance VoiceServerConfig with silence detection and custom dictionary - Added `silence_threshold` to the `VoiceServerConfig` for improved silence detection, allowing recordings with low RMS energy to be skipped. - Introduced `custom_dictionary` to bias transcription towards specific vocabulary, enhancing recognition of names and technical terms. - Updated the `voice_transcribe` and `voice_transcribe_bytes` functions to utilize the new `initial_prompt` parameter for better context during transcription. - Adjusted the `transcribe_pcm_i16` function to accept additional parameters for improved flexibility in handling audio input. * feat(voice): add silence threshold and custom dictionary features to VoicePanel - Implemented a new input for setting the silence threshold, allowing recordings with low RMS energy to be skipped. - Added functionality for a custom dictionary, enabling users to add specific vocabulary words to improve transcription accuracy. - Updated the VoiceServerSettings interface and related functions to support the new features, ensuring seamless integration with existing settings. - Enhanced the UI in the VoicePanel to facilitate user interaction with the new settings. * feat(voice): propagate silence threshold and custom dictionary to voice server command - Added `silence_threshold` and `custom_dictionary` parameters to the `run_voice_server_command` function, ensuring these settings are utilized during voice server operations. - Enhanced integration with the existing voice server configuration to support improved transcription accuracy and silence detection. * fix(tauriCommands): update import paths for coreRpcClient - Adjusted import paths for `callCoreRpc` in multiple Tauri command modules to ensure correct referencing from the updated directory structure. - This change enhances module organization and maintains consistency across the codebase. * feat(dependencies): update Cargo.lock and Cargo.toml for new packages and versions - Added new dependencies including `arboard`, `fax`, `fax_derive`, `gethostname`, `half`, `quick-error`, `tiff`, and `x11rb` to enhance functionality and support for clipboard operations, transcription improvements, and system interactions. - Updated existing dependencies to their latest versions for better performance and compatibility. - Modified `VoicePanel` to utilize the updated settings and ensure proper handling of voice server configurations. - Enhanced the text input mechanism to use clipboard-paste for improved reliability in text insertion. * fix(voice): update skip_cleanup default value and enhance logging - Changed the default value of `skip_cleanup` in `VoiceServerConfig` from `true` to `false` to align with expected behavior and improve transcription handling. - Added detailed logging in the transcription cleanup process to provide better insights into the LLM state and cleanup decisions. - Removed unused functions related to unreliable key releases in hotkey handling to simplify the codebase. - Updated tests to reflect the new default settings for `skip_cleanup` and ensure proper functionality across components. * style: apply linter formatting fixes * fix(voice): remove unused warn import in hotkey module * test(voice): add silence threshold and custom dictionary to VoicePanel tests - Updated tests for the VoicePanel component to include new parameters: `silence_threshold` and `custom_dictionary`. - Ensured that the tests reflect the latest configuration settings for improved transcription accuracy and functionality. |
||
|
|
11f718d8bc |
feat(voice): standalone voice dictation server with hotkey support (#368)
* feat: add standalone voice dictation server with hotkey support - Introduced a new `voice` subcommand to the CLI for running a standalone voice dictation server that listens for a hotkey, records audio, transcribes it using Whisper, and inserts the result into the active text field. - Implemented configuration options for the voice server, including hotkey combination, activation mode (tap or push), and an option to skip LLM post-processing. - Added audio capture functionality using the `cpal` crate and integrated hotkey listening with the `rdev` crate for global key event handling. - Enhanced the configuration schema to include voice server settings and updated the main configuration structure accordingly. - Updated relevant modules and tests to ensure consistent behavior and functionality across the application. This feature enhances user interaction by allowing voice dictation directly into any active text field, improving accessibility and usability. * feat: add voice dictation server with hotkey support - Introduced a standalone voice dictation server that listens for a configurable hotkey to start recording audio, transcribes it using whisper, and inserts the transcribed text into the active text field. - Added CLI support for the `voice` command, allowing users to manage the voice server's configuration, including hotkey and activation mode settings. - Implemented configuration structures for the voice server, including options for automatic start, hotkey combination, activation mode, and cleanup behavior. - Enhanced audio capture functionality using the `cpal` library for microphone input and integrated text insertion using the `enigo` library for simulating keyboard input. - Updated relevant modules and schemas to support the new voice server features, ensuring a cohesive integration within the OpenHuman platform. * refactor: streamline voice server command and enhance audio capture functionality - Updated the `run_voice_server_command` function to initialize the configuration with environment overrides instead of loading from a file, improving performance and flexibility. - Refactored the audio capture logic in `start_recording` to enhance thread management and error handling, ensuring a more robust audio stream setup. - Improved the handling of audio stream creation and playback, ensuring that all cpal objects are managed on the same thread as required, enhancing stability during recording operations. * fix: remove unused import in voice server module - Eliminated the `HotkeyListenerHandle` import from the `server.rs` file, streamlining the code and improving clarity by removing unnecessary dependencies. * feat(voice): auto-enable LLM cleanup when local model is ready The postprocessor now checks the local LLM state and automatically enables transcription cleanup when the model is downloaded and ready, even if not explicitly configured. Falls back gracefully to raw text when the LLM is unavailable. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: apply cargo fmt + prettier formatting Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(voice): dictation config, hotkey lifecycle, and WebSocket streaming (#332) Add the foundational infrastructure for voice dictation (EPIC #332): **Rust core:** - New `DictationConfig` schema with serde defaults and env var overrides (enabled, hotkey, activation_mode, llm_refinement, streaming, interval) - RPC controllers: `config_get_dictation_settings` / `config_update_dictation_settings` - WebSocket endpoint `/ws/dictation` for streaming PCM16 transcription with periodic partial inference and final LLM refinement - Microphone permission declaration (`NSMicrophoneUsageDescription`) in Tauri macOS bundle config **Frontend:** - `useDictationHotkey` hook: fetches config from core RPC, auto-registers global hotkey, listens for `dictation://toggle` events - `DictationHotkeyManager` headless component mounted in App.tsx - Fix voice RPC response type mismatch: voice handlers return flat results (no `{result, logs}` wrapper), so remove incorrect `CommandResponse<T>` wrapping from `openhumanVoiceStatus`, `openhumanVoiceTranscribe`, `openhumanVoiceTranscribeBytes`, and `openhumanVoiceTts` Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(tauri): remove invalid infoPlist config that breaks tauri dev The `infoPlist` field in tauri.conf.json expects a string path, not an inline object. Remove it for now — microphone permission will be added via a proper Info.plist supplement in the production build pipeline. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * style: apply Prettier formatting to dictation files Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * format files * feat(dictation): integrate dictation listener and event broadcasting - Added a global dictation hotkey listener that activates based on configuration. - Implemented a web channel bridge to handle dictation events and broadcast them to connected clients. - Updated the voice module to include the new dictation listener functionality. This enhances the voice dictation capabilities by ensuring real-time event handling and client communication. * update code * format * feat(voice): enhance voice server configuration and functionality - Updated `Cargo.toml` to mark voice-related dependencies as optional. - Introduced `VoiceActivationMode` enum for better control over voice server activation. - Refactored voice server command handling and dictation event broadcasting to support new features. - Added conditional compilation for voice features across various modules, ensuring they are only included when enabled. This commit improves the modularity and configurability of the voice server, allowing for more flexible integration and usage. * refactor: clean up whitespace and formatting in core and voice modules - Removed unnecessary blank lines in `cli.rs`, `jsonrpc.rs`, `schemas.rs`, and `socketio.rs` to improve code readability. - Adjusted import order in `mod.rs` for better organization. This commit enhances the overall code quality by ensuring consistent formatting across multiple files. * chore: update Dockerfile and test workflow to install additional system dependencies - Added installation of system dependencies (cmake, ALSA, X11) in the Dockerfile for improved build support. - Updated the GitHub Actions workflow to reflect the new dependencies, ensuring consistent environment setup for testing. This commit enhances the build environment by including necessary libraries for audio and GUI support. * format * fix claude * format --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: oxoxDev <nikhil@tinyhumans.ai> |
||
|
|
07b1df4f24 |
feat(event_bus): wire webhooks, channels & skills through the event bus (#379)
* feat(event_bus): enhance domain event handling across modules - Added new `DomainEvent` variants for channel and skill events, including `ChannelMessageReceived`, `ChannelMessageProcessed`, `ChannelConnected`, `ChannelDisconnected`, `SkillLoaded`, `SkillStopped`, and `SkillStartFailed`. - Implemented event publishing in the channels and skills modules to track message processing and skill lifecycle events. - Created dedicated event bus handler files for the skills and webhooks domains, preparing for future subscriber implementations. - Updated documentation in `CLAUDE.md` to reflect the new domain events and their usage. These changes improve the observability and modularity of the system by leveraging an event-driven architecture for cross-module communication. * feat(event_bus): implement channel and webhook event handling - Introduced `ChannelInboundSubscriber` to handle inbound channel messages, triggering the agent inference loop and sending replies via the backend REST API. - Added `WebhookRequestSubscriber` to manage incoming webhook requests, routing them to the appropriate skill and handling responses. - Updated the global event bus initialization in `bootstrap_skill_runtime` to register both channel and webhook subscribers. - Enhanced `DomainEvent` with new variants for channel inbound messages and webhook requests, improving event-driven communication across modules. These changes enhance the modularity and responsiveness of the system by leveraging an event-driven architecture for channel and webhook interactions. * refactor(event_bus): update domain event documentation and subscriber initialization - Revised the documentation in `CLAUDE.md` to provide a concise overview of domain events and their associated subscriber files, enhancing clarity for future development. - Updated the `start_channels` function to initialize `WebhookRequestSubscriber` and `ChannelInboundSubscriber`, ensuring proper event handling for webhooks and channel messages. - Streamlined the event bus subscriber registration process, reinforcing the modular architecture of the system. These changes improve the maintainability and usability of the event bus framework, facilitating better cross-module communication. * style: apply cargo fmt formatting Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(event_bus): remove duplicate subscriber registration in start_channels WebhookRequestSubscriber and ChannelInboundSubscriber were registered in both bootstrap_skill_runtime() and start_channels(), causing events to be handled twice when both paths run in the same process. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(event_bus): prevent subscriber handles from being dropped on function exit SubscriptionHandle::drop aborts the background task. Since bootstrap_skill_runtime() returns immediately after setup, the local handles were dropped, cancelling both subscribers. Use std::mem::forget to leak the handles so the tasks live for the entire process. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(event_bus): ensure subscriber handles persist beyond function exit Modified the handling of subscriber registration to prevent premature dropping of handles in `bootstrap_skill_runtime()`. This change ensures that the background tasks for subscribers remain active for the entire process lifecycle, enhancing event handling reliability. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(webhooks): use proper JSON serialization for error response bodies Hand-escaped JSON strings only handled double quotes, not backslashes, newlines, or other control chars. Replaced with serde_json serialization via an error_body() helper. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(webhooks): use proper JSON serialization for error response bodies Hand-escaped JSON strings only handled double quotes, not backslashes, newlines, or other control chars. Replaced with serde_json serialization via an error_body() helper. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Steven Enamakel
|
c5acfa90e0 | Refactor Docker CI image workflow to enhance security by replacing GitHub App token generation with GITHUB_TOKEN for authentication during Docker image pushes. | ||
|
|
5affb1e75f |
feat(integrations): add agent integration tools for Twilio, Google Places, and Parallel (#375)
* feat(integrations): add agent integration tools for Twilio, Google Places, and Parallel Add a new `src/openhuman/integrations/` module with 5 backend-proxied tools that give agents access to phone calls, location search, and web search/extraction. Each tool calls the backend API which handles external API keys, billing, rate limiting, and markup — no pricing logic on the client side. Tools added: - TwilioCallTool (scope: CLI/RPC only — requires explicit user action) - GooglePlacesSearchTool, GooglePlacesDetailsTool (scope: All) - ParallelSearchTool, ParallelExtractTool (scope: All) Includes IntegrationsConfig with per-integration toggles, shared IntegrationClient with pricing cache (fetched from backend GET /agent-integrations/pricing), ToolScope enum for future scope-based filtering, and 33 unit tests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(integrations): add agent integration tools for Twilio, Google Places, and Parallel Add a new `src/openhuman/integrations/` module with 5 backend-proxied tools that give agents access to phone calls, location search, and web search/extraction. Each tool calls the backend API which handles external API keys, billing, rate limiting, and markup — no pricing logic on the client side. Tools added: - TwilioCallTool (scope: CLI/RPC only — requires explicit user action) - GooglePlacesSearchTool, GooglePlacesDetailsTool (scope: All) - ParallelSearchTool, ParallelExtractTool (scope: All) Includes IntegrationsConfig with per-integration toggles, shared IntegrationClient with pricing cache (fetched from backend GET /agent-integrations/pricing), ToolScope enum for future scope-based filtering, and 33 unit tests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix code * format code * Refactor tool integration code to improve clarity and error handling - Updated `tool_loop.rs` to cache the result of `find_tool` for efficiency and clarity in logging. - Removed unused `scope` methods from `GooglePlacesSearchTool`, `ParallelSearchTool`, and `TwilioCallTool`, simplifying the codebase. - Modified `BackendResponse` structure in `mod.rs` to use `Option<T>` for `data`, enhancing error handling when no data is returned from the backend. - Updated related tests to reflect changes in the response structure and ensure proper handling of cases with no data. These changes enhance code maintainability and improve the robustness of the integration tools. * Remove comments * Add shared HTTP client for integration tools - Introduced `IntegrationClient` to manage backend interactions, including methods for POST and GET requests with error handling. - Implemented caching for pricing information fetched from the backend. - Created a new `types` module to define shared types such as `BackendResponse` and `IntegrationPricing`, enhancing code organization and clarity. - Updated `mod.rs` to include the new client and types, ensuring proper module structure. These changes improve the maintainability and functionality of the integration tools by providing a robust client for backend communication. --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Steven Enamakel
|
5c7c104085 | Added production environment specification to Docker CI image workflow for enhanced deployment clarity. | ||
|
Steven Enamakel
|
5b3136306b | Merge remote-tracking branch 'upstream/main' | ||
|
Steven Enamakel
|
521fc7805d |
Enhance GitHub Actions workflow for Docker CI image
- Added a step to generate a GitHub App token using the tibdex/github-app-token action, improving authentication for Docker image pushes. - Updated the Docker push step to use the generated token instead of the default GITHUB_TOKEN, enhancing security and access control during the CI process. |
||
|
|
45d15cd510 |
feat: text_input domain module for focused field read/insert/ghost text (#377)
* feat: add text_input domain module for focused field read/insert/ghost text Thin orchestration layer over accessibility APIs — provides a clean RPC surface for reading, inserting, and previewing text in OS-focused input fields. Consumed by autocomplete and voice control. Five RPC methods: read_field, insert_text, show_ghost, dismiss_ghost, accept_ghost. Standalone CLI (`openhuman text-input`) and JSON-RPC server mode for testing. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: apply cargo fmt to text_input module Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Steven Enamakel
|
23212f8b76 | update dicker | ||
|
Steven EnamakelandGitHub
|
bfe4918a85 |
Enhance E2E testing environment by updating Dockerfile and workflows (#376)
- Updated the Dockerfile to include additional system dependencies for E2E testing, such as xvfb, dbus, and webkit2gtk-driver, along with the installation of tauri-driver. - Modified the docker-compose.yml to use the shared CI image for E2E tests, streamlining the build process and ensuring consistency across environments. - Deprecated the previous E2E Dockerfile, consolidating the setup into the CI image for improved maintainability and reduced redundancy. - Adjusted the GitHub Actions workflow to reflect changes in the Dockerfile and ensure proper context for building the image. |
||
|
|
88257d3231 |
feat(update): core sidecar self-update with version detection (#372)
* update * update * Refactor code style for consistency and readability in core update module - Reformatted platform_triple function to improve readability by aligning braces. - Simplified async function calls and error handling in various places for better clarity. - Enhanced logging statements for improved observability during update processes. These changes enhance the maintainability of the codebase while ensuring consistent formatting across the module. * feat(update): periodic background update checker with config flag Add a periodic update scheduler that checks GitHub Releases for newer core binary versions on a configurable interval (default: 1 hour). Controlled by `[update]` config section with `enabled = true` by default. - New `UpdateConfig` in config schema (enabled, interval_minutes) - Env var overrides: OPENHUMAN_AUTO_UPDATE_ENABLED, OPENHUMAN_AUTO_UPDATE_INTERVAL_MINUTES - Background scheduler spawned at server startup in run_server() - Reports to health registry as "update_checker" component Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(update): address PR review — restart path, security, version tracking - CoreProcessHandle: add set_core_bin/effective_core_bin so ensure_running launches the newly staged binary instead of the original one - check_and_update_core: add `force` param — auto-check uses MINIMUM_CORE_VERSION, manual apply_core_update uses latest release (force=true) - Acquire restart_lock before download+staging to prevent concurrent updates; shutdown old process before staging; use unique temp filename - check_core_update Tauri command now queries GitHub for latest_version and returns update_available alongside outdated - Harden update_apply RPC: validate download URL is GitHub HTTPS, validate asset_name is safe filename starting with openhuman-core-, ignore caller staging_dir (always use safe default) - download_and_stage accepts target_version so installed_version reflects the staged release, not the running process - Add update.check and update.apply to about_app capability catalog - ops.rs: unwrap_or_default → unwrap_or_else with error context - tauriCommands.ts: convert to arrow-function exports, add latest_version and update_available to CoreUpdateStatus interface Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * format --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
|
264bfe4081 |
feat(local_ai): sentiment analysis, GIF decision & Tenor search (#373)
* feat(local_ai): add sentiment analysis, GIF decision, and Tenor search Extend the local model with two new capabilities: - Emotion/sentiment analysis (joy/sadness/anger/etc + valence + confidence) via a lightweight prompt, designed to run periodically (~hourly) - GIF decision + Tenor search: local model decides when a GIF response fits, generates a search query, and proxies through the backend's new Tenor API (POST /agent-integrations/tenor/search) New RPC endpoints: - openhuman.local_ai_analyze_sentiment - openhuman.local_ai_should_send_gif - openhuman.local_ai_tenor_search Frontend integrates with cadence-based invocation: - Reactions: every message (unchanged) - GIF decisions: every ~7 messages - Sentiment analysis: every ~1 hour Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: apply formatter fixes Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * merge g Please enter the commit message for your changes. Lines starting --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
|
a36535330b |
feat: add typed event bus for cross-module decoupling (#374)
* Add event bus integration to channels module - Introduced an `event_bus` module to facilitate event-driven communication within the channels. - Updated `ChannelRuntimeContext` to include an `event_bus` field for managing events. - Enhanced the `start_channels` function to initialize the global event bus and register a tracing subscriber for logging domain events. - Modified the cron scheduler to publish delivery requests as events, decoupling the delivery logic from specific channel implementations. - Updated tests to ensure proper initialization and usage of the event bus across various contexts. This change improves the modularity and scalability of the channels system by leveraging an event-driven architecture. * refactor: move CronDeliverySubscriber to cron/bus.rs Each domain owns its event bus handlers — move the delivery subscriber from channels/cron_delivery.rs into cron/bus.rs so the cron module contains both its publisher and subscriber logic. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update Dockerfile to install system dependencies and refactor event bus logging - Added installation of system dependencies (cmake, ALSA, X11) in the Dockerfile to support build requirements. - Refactored event bus logging in various files for improved readability by consolidating multi-line log statements into single lines. * chore: update Dockerfile to include webkit2gtk-driver installation - Added installation of webkit2gtk-driver in the Dockerfile to support additional system dependencies. - Removed redundant apt-get update command to streamline the installation process. * docs: add event bus usage guide to CLAUDE.md Documents the event bus pattern, core types, global access, and the convention for adding domain events and subscribers so future modules follow the established design. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update Dockerfile to streamline Node.js installation - Removed redundant cleanup command after installing yarn in the Dockerfile, simplifying the installation process. * chore: update Dockerfile to fix yarn installation command - Removed unnecessary trailing whitespace in the yarn installation command in the Dockerfile, ensuring cleaner code and consistency. * chore: consolidate system dependencies installation in Dockerfile - Merged the installation of webkit2gtk-driver with other system dependencies in the Dockerfile to streamline the setup process and reduce the number of RUN commands. * refactor: enforce EventBus as a singleton - Make EventBus::create() pub(crate) — only tests can construct isolated instances; production code must use the global singleton - Add subscribe_global() convenience for subscribing from any module - Remove event_bus field from ChannelRuntimeContext — all modules use init_global/publish_global/subscribe_global instead of passing instances - Update CLAUDE.md to document the singleton API as the only way to use the event bus Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address PR review — panic safety, init_global, Dockerfiles, tests - Wrap handler.handle() in catch_unwind so a panicking subscriber does not silently kill its task — logs the panic and continues the loop - Call init_global() in cron scheduler::run() so delivery events are not silently dropped when the scheduler starts before start_channels() - Export DEFAULT_CAPACITY and use it everywhere instead of hard-coded 256 - Add ALSA/X11/input dev libraries to e2e/Dockerfile to mirror CI runner - Remove duplicate cmake install from .github/Dockerfile - Add 4 unit tests for CronDeliverySubscriber (ignore non-delivery, dispatch to channel, missing channel, send failure) - Replace scheduler announce-mode test with one that verifies event payload is actually received by a subscriber Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
|
200db04fc7 |
feat: scope user data to per-user directories (#370)
* feat(config): add user ID retrieval and workspace scoping for authenticated users - Implemented `read_authenticated_user_id` to extract the user's ID from `auth-profiles.json`, avoiding a dependency cycle with the credentials module. - Introduced `maybe_scope_workspace_to_user` to create user-specific workspace directories based on the authenticated user ID, ensuring isolated workspace data. - Updated the configuration loading process to call `maybe_scope_workspace_to_user`, enhancing user data management. - Added unit tests for the new functionality, ensuring correct behavior in various scenarios. This change improves user experience by providing personalized workspace management based on authentication status. * feat(config): enhance user management with active user state handling - Added functions to manage the active user state, including `read_active_user_id`, `write_active_user_id`, and `clear_active_user`, allowing for user-specific configuration and workspace isolation. - Introduced `default_root_openhuman_dir` to standardize the retrieval of the root directory for user data. - Updated configuration loading to support user-scoped directories, improving the overall user experience by ensuring personalized settings and workspace management. This change enhances the OpenHuman platform by enabling better user data management and isolation. * feat(credentials): enhance user directory management during session storage - Added logic to create and activate user-scoped directories based on the resolved user ID when storing session data, ensuring credentials are saved in the correct location. - Implemented error handling for directory creation and active user ID writing, with appropriate logging for failures. - Updated the configuration loading process to reflect the newly activated user directory, improving user-specific settings management. - Enhanced the `get_data_dir` function to return user-scoped directories if an active user is set, streamlining data access. This change improves user experience by ensuring that session data is correctly organized and accessible based on user context. * refactor(tests): update user ID handling and improve test coverage - Renamed and refactored tests to better reflect functionality, focusing on active user ID management. - Removed the `write_auth_profiles` helper function and replaced it with direct calls to `write_active_user_id` for clarity. - Enhanced tests to cover scenarios for reading and clearing active user IDs, ensuring accurate behavior in user-specific configurations. - Added a new test for building user directory paths, improving overall test coverage for user management features. This change streamlines the testing process and enhances the clarity of user ID handling in the configuration schema. * refactor(paths): streamline model and binary path resolution - Introduced a new `shared_root_dir` function to centralize the logic for determining the shared root openhuman directory, improving code clarity and reducing duplication. - Updated `workspace_ollama_dir` and `workspace_local_models_dir` functions to utilize the new shared root directory, ensuring consistent path resolution for user-specific and shared resources. - Enhanced the `model_artifact_path` function to leverage the new directory structure, improving the organization of model artifacts. This refactor enhances maintainability and clarity in the path management for local AI resources. * style: apply cargo fmt formatting Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(paths): streamline directory management for model artifacts - Updated the `model_artifact_path` function to utilize a new `shared_root_dir` function, which centralizes the logic for determining the root openhuman directory. - Enhanced the `config_root_dir` function to improve clarity and maintainability. - Adjusted the `workspace_ollama_dir` and `workspace_local_models_dir` functions to leverage the new shared directory logic, ensuring consistent path resolution across the application. These changes improve the organization of directory management and enhance the overall clarity of the codebase. --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
|
df503a23fd |
Fix: decouple chat from local Ollama, route inference through Rust (#369)
* refactor: streamline Conversations component by removing local response delivery logic - Eliminated the `deliverLocalResponse` function, which handled segmenting and dispatching local model responses, simplifying the message delivery process. - Updated socket status checks to remove unnecessary local model condition, ensuring clarity in connection handling. - Adjusted comments to reflect the new routing logic for cloud backend interactions, emphasizing that local models are now used only for supplementary features. * refactor: update chat service to utilize core RPC for message handling - Replaced Socket.IO message sending with core RPC calls for sending and canceling chat messages, enhancing the communication mechanism. - Improved error handling for socket connection checks, ensuring robust event routing. - Updated documentation to reflect the new RPC-based approach for chat message processing. * feat: enhance chat response handling with segmentation and emoji reactions - Introduced a new `ChatSegmentEvent` interface to support segmented message delivery, allowing for more natural chat interactions. - Updated the `Conversations` component to handle segmented responses and apply emoji reactions based on user messages. - Refactored the message delivery logic to improve clarity and maintainability, ensuring that responses are dispatched correctly based on segmentation. - Enhanced the chat service to emit segment events, facilitating a smoother user experience during conversations. * refactor: simplify Conversations component by removing unused imports and optimizing rendering logic - Removed unnecessary imports related to local model status and message segmentation, streamlining the codebase. - Updated the rendering logic to enhance clarity by simplifying conditions for displaying sending and delivering states in the chat interface. - Improved overall readability and maintainability of the Conversations component. * style: apply formatter fixes (prettier + cargo fmt) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * updated convos * fix test --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
|
9b73d00b2e |
fix(agent): inject skill tools into agent registry and unify ToolResult (#341) (#360)
* fix(agent): inject skill tools into agent registry and unify ToolResult (#341) Skill tools (Notion, Gmail, etc.) from the QuickJS runtime were invisible to the agent's tool loop — the LLM could never call them. This change: 1. Adds SkillToolBridge to wrap skill ToolDefinitions as Tool trait impls and inject them into the agent's tool registry via collect_skill_tools(). 2. Unifies ToolResult across built-in and skill tools — both now use the MCP-spec type (content blocks + is_error) from skills::types, eliminating the need for result conversion between the two systems. 3. Adds convenience constructors (ToolResult::success/error/json) and accessor methods (output/text) to simplify all tool implementations. 4. Adds diagnostic logging in the tool loop for tool registry contents and tool lookup results. Verified: agent_chat RPC successfully called Notion list-all-pages tool and returned real workspace data through the agent loop. Closes #341 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(tools): address CodeRabbit review feedback on PR #360 - cron_run: preserve execution details (result_output) on failure - delegate: fix mismatched bracket in agent response header format string - run_linter: include lint diagnostics in error output, not just exit code - run_tests: include test output in error result, not just exit code - skill_bridge: prevent namespaced tool-name collisions with dedup + __ delimiter validation; sanitize runtime errors in model-facing output Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: apply cargo fmt to skill_bridge.rs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
|
faa881c5f1 |
Feat/343 screen intelligence e2e tests (#359)
* test(screen_intelligence): add E2E pipeline proof tests for #343 Add layered test coverage proving the full capture → vision → memory pipeline: screenshot save/cleanup disk paths, VisionSummary serde roundtrip, JSON-RPC shape tests for status and vision_recent endpoints. - tests/screen_intelligence_vision_e2e.rs: save_screenshot_to_disk creates a PNG and keep_screenshots=false cleanup removes it; VisionSummary struct serializes/persists/is queryable end-to-end; platform support table + macOS checklist added to module doc - tests/json_rpc_e2e.rs: screen_intelligence_status shape test (platform_supported, session.active, permissions.screen_recording); vision_recent returns empty summaries without an active session - src/openhuman/screen_intelligence/tests.rs: save_screenshot_to_disk unit tests for the write path and the no-image-ref error path Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(screen_intelligence): enhance vision summary persistence and error handling - Added new fields to track vision persistence count, last persisted key, and last persist error in SessionRuntime and SessionStatus. - Implemented error handling for vision summary persistence, ensuring errors are logged and state is updated accordingly. - Introduced a new method to analyze a frame and persist the summary, improving the vision processing pipeline. - Updated tests to validate the new functionality and ensure proper behavior with mocked vision outputs. This commit improves the robustness of the screen intelligence pipeline by enhancing the tracking and handling of vision summary persistence. --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
YellowSnnowmannandGitHub
|
1acc916244 |
feat: extract and load user working memory from skill sync payloads (#357)
* feat: implement user working memory extraction from skill sync payloads - Added functionality to enable the extraction of user working memory from successful skill syncs, allowing for persistent storage of user preferences, goals, constraints, and entities. - Introduced a new configuration option in to toggle working memory extraction. - Created comprehensive documentation on the working memory extraction process, detailing its implementation and privacy considerations. - Updated memory loading logic to include working memory entries in the context provided to agents, enhancing personalization capabilities. - Enhanced logging for memory extraction processes to improve observability and debugging. This feature enhances the user experience by allowing skills to maintain context across interactions, improving the overall effectiveness of the OpenHuman platform. * docs: update architecture documentation to include user working memory integration * refactor: centralize working memory constants and enhance extraction logic - Moved `WORKING_MEMORY_KEY_PREFIX` and `WORKING_MEMORY_LIMIT` constants to `memory_context.rs` for better organization and accessibility. - Updated `MemoryLoader` to utilize these constants, improving code clarity. - Enhanced working memory extraction logic in `MemoryWriteJob` to conditionally persist user working-memory documents based on the job type. - Improved logging for memory extraction processes to provide clearer insights during execution. - Adjusted tests to ensure consistent behavior with the new working memory extraction logic. * chore: update OpenHuman version to 0.51.8 and refactor JSON-RPC test for clarity - Bumped the OpenHuman version in Cargo.lock from 0.51.6 to 0.51.8. - Refactored the JSON-RPC end-to-end test to improve readability by encapsulating the result assertion logic within a block, enhancing clarity in the flow of data handling. |
||
|
|
b8ae9674b3 |
fix(memory): graph query returns namespace data and add sync e2e tests (#344) (#363)
* fix(memory): graph query returns namespace data and add sync e2e tests (#344) The knowledge graph UI showed empty because graph_query(None) only queried the graph_global table, while ingestion writes to graph_namespace. Now graph_query(None) queries both tables via graph_query_all(), merging results. Changes: - Added graph_query_all() in unified graph store to query across all namespaces - MemoryClient::graph_query(None) now uses graph_query_all() instead of graph_query_global() - MemoryWorkspace passes selectedNamespace to the RPC call - Added diagnostic logging in ingestion pipeline (RelEx model availability, extraction counts) - Added debug logging in tauriCommands for unexpected response shapes - Added 2 integration tests proving document sync populates the graph (ignored by default for CI, run with --ignored) Closes #344 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: remove trailing comma for Prettier compliance Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
|
48d937cf6b |
refactor(intelligence): move memory workspace to Settings > Developer Options (#364)
* fix(settings): normalize padding across all settings panels Remove max-w-md mx-auto constraints from BillingPanel, RecoveryPhrasePanel, TeamPanel, TeamMembersPanel, TeamInvitesPanel, and TeamManagementPanel so all panels use consistent p-4 padding matching ConnectionsPanel and the other baseline panels. * refactor(intelligence): move memory workspace to Settings > Developer Options Move Memory, Memory Graph, Intelligence Insights, Activity Heatmap, and File Management from the Intelligence page into a new "Memory Data" panel under Settings > Developer Options. The Intelligence page now shows only the search bar and actionable items, keeping it focused for end users. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(intelligence): move memory workspace to Settings > Developer Options Move Memory, Memory Graph, Intelligence Insights, Activity Heatmap, and File Management from the Intelligence page into a new "Memory Data" panel under Settings > Developer Options. The Intelligence page now shows only the search bar and actionable items, keeping it focused for end users. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Cyrus GrayandGitHub
|
a2dff715f0 |
fix(settings): normalize padding across all settings panels (#362)
Remove max-w-md mx-auto constraints from BillingPanel, RecoveryPhrasePanel, TeamPanel, TeamMembersPanel, TeamInvitesPanel, and TeamManagementPanel so all panels use consistent p-4 padding matching ConnectionsPanel and the other baseline panels. |
||
|
Cyrus GrayandGitHub
|
c036b75b7c |
feat(nav): enable Home tab in bottom toolbar (#361)
Uncomment the pre-defined Home tab entry in BottomTabBar so it appears as the first item in the bottom navigation. Route, active-state logic, and page component were already wired — only the tab array entry was commented out. Closes #356 |
||
|
YellowSnnowmannandGitHub
|
0c14aea96a |
Improve autocomplete observability and runtime controls in settings + JSON-RPC (#308)
* Enhance AutocompletePanel logging and functionality - Introduced MAX_LOG_ENTRIES constant to limit log entries to 200. - Updated log formatting to include timestamps with milliseconds for better precision. - Added UI logging for various actions (e.g., saving settings, starting/stopping autocomplete) to improve traceability. - Enhanced error handling in refreshStatus, acceptSuggestion, and other functions to log specific failure messages. - Added unit tests for AutocompletePanel to ensure functionality and logging behavior. This update improves the overall user experience by providing clearer logs and better error handling in the Autocomplete feature. * Refactor accessibility and autocomplete components for improved error handling and logging - Updated to log role changes as debug information instead of returning an error, allowing for more flexible handling of role fluctuations. - Increased the timeout for autocomplete refresh operations from 15 seconds to 120 seconds to accommodate longer processing times, enhancing reliability. - Improved error handling in the autocomplete engine to preserve previous suggestions and provide clearer error messages when operations are aborted. These changes enhance the user experience by providing better logging and more robust handling of focus and autocomplete functionalities. * Enhance AutocompletePanel functionality and improve error handling - Refactored loadHistory to return an empty array when Tauri is not available, improving error handling. - Introduced waitForAcceptedHistoryEntry to ensure the history is loaded before accepting suggestions, enhancing user experience. - Updated the accept suggestion logic to use the new waitForAcceptedHistoryEntry function, ensuring the correct suggestion is applied. - Modified tests to reflect changes in the accept suggestion API, ensuring accurate functionality. These changes improve the reliability and responsiveness of the autocomplete feature. |
||
|
|
24b892439d |
feat(channels): Discord server/channel wiring and picker (#289) (#349)
* feat(channels): add channel_id to DiscordConfig schema Add optional channel_id field to DiscordConfig for restricting the bot to a specific Discord channel, matching the pattern used by SlackConfig and MattermostConfig. Refs: #289 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(channels): add channel_id field to Discord definition Expose channel_id as an optional field in the Discord BotToken auth mode so users can specify a default channel for outbound messages via the UI. Refs: #289 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(channels): create Discord API helper module for guild/channel discovery Refactor discord.rs into discord/ folder module and add api.rs with: - list_bot_guilds: GET /users/@me/guilds - list_guild_channels: GET /guilds/{id}/channels (filtered to text channels) - check_channel_permissions: compute bot permissions from roles + overwrites Includes unit tests for type serialization and permission bit constants. Refs: #289 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(channels): add Discord RPC handlers for guild/channel discovery Add three new RPC endpoints: - openhuman.channels_discord_list_guilds: list servers the bot is in - openhuman.channels_discord_list_channels: list text channels in a guild - openhuman.channels_discord_check_permissions: validate bot permissions These retrieve the stored Discord bot token from credentials and call the Discord REST API directly from the Rust core. Refs: #289 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(channels): wire channel_id into Discord provider for channel filtering Add channel_id field to DiscordChannel struct and update all construction sites. When channel_id is set, the listen loop only processes messages from that specific channel, enabling server+channel-scoped operation. Refs: #289 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(channels): add Discord guild/channel API types and RPC methods Add TypeScript types for DiscordGuild, DiscordTextChannel, and BotPermissionCheck. Wire up three new RPC methods in channelConnectionsApi for listing guilds, listing channels, and checking bot permissions. Refs: #289 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(channels): create DiscordServerChannelPicker component Add server and channel selection UI that loads guilds and channels from the Discord API via the Rust core RPC. Includes permission checking with visual feedback for missing permissions. Refs: #289 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(channels): integrate server/channel picker into DiscordConfig Show DiscordServerChannelPicker below the connect buttons when the bot_token connection is active. Guild and channel selections flow back into the credential field values for persistence. Refs: #289 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test(channels): add unit tests for Discord channel_id and config serde Add tests for: - DiscordConfig TOML/JSON deserialization with and without channel_id - channel_id field storage on DiscordChannel struct - Config roundtrip serialization Refs: #289 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test(channels): add Vitest tests for DiscordServerChannelPicker Test guild loading, rendering, and placeholder states with mocked RPC responses. Verifies the component renders heading, loads guilds from the mock, and shows the select placeholder. Refs: #289 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(ci): resolve discord module conflict and format drift * fix(reviews): address CodeRabbit Discord picker and permission issues --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
github-actions[bot]
|
32c583aafd | chore(release): v0.51.8 v0.51.8 | ||
|
github-actions[bot]
|
36e5736814 | chore(release): v0.51.7 | ||
|
|
b589a29e77 |
Extract socket controller into dedicated domain module (#340)
* refactor: remove global_engine usage in tests and instantiate AccessibilityEngine directly - Updated the test suite to eliminate the use of global_engine, replacing it with a direct instantiation of AccessibilityEngine. - This change enhances test isolation and clarity by ensuring that each test operates with its own instance of the engine, improving reliability and maintainability. * feat: add socket module for skill communication - Introduced a new `socket` module to facilitate communication between skills, enhancing the modularity and organization of the codebase. - Updated imports in `qjs_engine.rs` to reference the new `SocketManager` from the `socket` module, streamlining socket management for skill interactions. - Removed the deprecated `socket_manager` module from the skills module, improving clarity and reducing redundancy in the code structure. * feat: integrate socket controllers and schemas into core functionality - Added socket-related registered controllers and schemas to the core build functions, enhancing the communication capabilities within the OpenHuman framework. - Updated the `build_registered_controllers` and `build_declared_controller_schemas` functions to include socket components, ensuring comprehensive integration of the new socket module. * feat: enhance QuickJS skill runtime with socket manager integration - Updated the `bootstrap_skill_runtime` function to initialize and register the `SocketManager` globally, allowing RPC handlers to access socket functionalities. - Improved documentation to reflect the addition of socket management capabilities alongside the QuickJS skill runtime. - Cleaned up imports in `event_handlers.rs` to streamline the codebase. * refactor: remove socket manager integration from skill runtime - Removed the `SocketManager` integration from the `bootstrap_skill_runtime` function, simplifying the socket management process. - Eliminated the `socket_manager` field and related methods from the `RuntimeEngine` struct, streamlining the codebase. - Cleaned up unused MCP handlers and socket-related imports in the event handlers, enhancing code clarity and maintainability. * refactor: remove sync_tools calls from skill status handling - Eliminated unnecessary calls to `sync_tools()` in the `RuntimeEngine` during skill status changes, simplifying the skill lifecycle management. - This change enhances performance by reducing redundant synchronization operations during skill execution and shutdown processes. * feat: enhance socket event handling with improved logging - Added logging for incoming socket events to improve observability, including event name and data size. - Implemented detailed debug logging for event payloads, ensuring clarity on the data being processed. - Updated event handling logic to streamline routing for webhook requests and inbound channel messages, enhancing the overall responsiveness of the system. - Introduced logging for unhandled events to aid in debugging and monitoring. * feat: enhance socket management and auto-connect functionality - Updated the `bootstrap_skill_runtime` function to clone the `SocketManager` instance for global registration, ensuring proper socket management. - Introduced background tasks for auto-starting skills and auto-connecting to the backend using stored session tokens, improving startup efficiency and user experience. - Added detailed logging for session token checks and connection attempts, enhancing observability and debugging capabilities during socket operations. * feat: enhance skill selection and tool management in tests - Introduced a new `manifests_in_dir` function to retrieve skill manifests from a specified directory, improving skill discovery. - Added `select_skill_id` function to prioritize skill selection based on environment variables and preferred candidates, enhancing flexibility in test configurations. - Updated the test suite to utilize the new skill selection logic, ensuring more robust and configurable test scenarios. - Improved handling of tool selection with enhanced logging and fallback mechanisms for better debugging and usability. * format code:wq * feat: add Rust checks and formatting commands to package.json - Introduced new scripts for Rust checks and formatting in both the main and app package.json files. - Updated pre-push hook to include Rust compile checks, enhancing pre-push validation. - Modified existing format commands to integrate Rust formatting, ensuring consistency across codebases. * fix: improve formatting of the "Keep Screenshots" label description - Adjusted the formatting of the description text for better readability by breaking it into multiple lines within the `ScreenIntelligencePanel` component. * fix: install rustls crypto provider before WebSocket TLS connect The socket auto-connect was panicking with "Could not automatically determine the process-level CryptoProvider" because tokio-tungstenite uses rustls for wss:// but no crypto provider was installed. Install the ring provider before each connect attempt. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use dynamically selected skill in sync memory tests The tests hardcoded 'example-skill' which no longer exists in the skills directory. Now dynamically picks the first available skill (preferring server-ping) so tests work regardless of which skills are present. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
|
bb1f2c3c8d |
feat: screen intelligence pipeline + CLI + keep_screenshots config (#339)
* feat: add keep_screenshots functionality to Screen Intelligence settings - Introduced a new `keep_screenshots` option in the Screen Intelligence settings, allowing users to save captured screenshots to the workspace instead of deleting them after processing. - Updated relevant components and tests to reflect this new feature, ensuring consistent behavior across the application. - Enhanced the user interface to include a checkbox for the `keep_screenshots` setting, improving user experience and configurability. * feat: add `openhuman screen-intelligence` CLI for standalone testing Adds a dedicated CLI (mirroring `openhuman skills`) to run the screen intelligence capture + vision pipeline without the full desktop app. Subcommands: run, status, capture, start, stop. Makes save_screenshot_to_disk public so the CLI capture --keep flag works. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add keep_screenshots to ScreenIntelligencePanel test assertion Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Steven EnamakelandGitHub
|
8757bb9f15 |
Feat/docs (#331)
* Enhance documentation and structure across core modules - Added comprehensive module-level documentation for `lib.rs`, `main.rs`, `cli.rs`, `jsonrpc.rs`, `mod.rs`, `config.rs`, and `skills.rs`, detailing their functionalities and responsibilities within the OpenHuman platform. - Improved function-level documentation to clarify arguments, return types, and error handling for key functions such as `run_core_from_args`, `rpc_handler`, and `run_from_cli_args`. - Enhanced comments throughout the code to improve readability and maintainability, ensuring a clearer understanding of the codebase for future development. * Enhance documentation for RuntimeEngine and SkillRegistry - Improved struct and function-level documentation in `qjs_engine.rs` to clarify the lifecycle management of JavaScript-based skills, including skill discovery, startup, and shutdown processes. - Updated comments in `skill_registry.rs` to provide clearer descriptions of the registry's role in managing active skills and their communication channels. - Added detailed explanations for key methods, enhancing understanding of their functionality and usage within the OpenHuman platform. * Enhance documentation across skill modules - Improved module-level and struct-level documentation in `manifest.rs`, `ops.rs`, `socket_manager.rs`, and `types.rs` to clarify the purpose and functionality of various components within the skills system. - Added detailed comments for key structs and functions, including `SkillSetup`, `Skill`, and `SocketManager`, enhancing understanding of their roles in skill management and communication. - Updated descriptions for enums and message types to provide clearer context for their usage in the skill lifecycle and internal messaging. * Enhance documentation across skills modules - Added comprehensive module-level documentation for `registry_ops.rs`, `registry_types.rs`, `schemas.rs`, and `qjs_skill_instance` modules, detailing their functionalities and responsibilities within the OpenHuman skills system. - Improved function-level comments to clarify the purpose and usage of key functions, including `registry_fetch`, `registry_search`, and `skill_install`. - Enhanced descriptions for structs and enums, providing clearer context for their roles in skill management and execution. - Updated comments throughout the code to improve readability and maintainability, ensuring a clearer understanding of the codebase for future development. * Enhance documentation for QuickJS and IndexedDB modules - Improved module-level documentation in `quickjs_libs/mod.rs` to clarify the purpose and functionality of the QuickJS runtime support. - Enhanced comments in `storage.rs` to provide detailed descriptions of the IndexedDB storage layer, including its features and compatibility with the browser's IndexedDB API. - Updated function-level comments to improve clarity on database connection management and object store operations, ensuring better understanding for future development. * Enhance documentation across memory modules - Added comprehensive module-level documentation for `mod.rs`, `ops.rs`, `rpc_models.rs`, `schemas.rs`, and `traits.rs`, detailing their functionalities within the OpenHuman memory system. - Improved function-level comments to clarify the purpose and usage of key functions, enhancing understanding of memory operations, RPC handling, and data structures. - Updated comments throughout the code to improve readability and maintainability, ensuring a clearer understanding of the memory system for future development. * Enhance documentation for memory modules - Updated module-level documentation in `chunker.rs`, `embeddings.rs`, and `relex.rs` to provide clear descriptions of their functionalities within the OpenHuman memory system. - Improved function-level comments to clarify the purpose and usage of key functions, enhancing understanding of markdown chunking, embedding providers, and relation extraction processes. - Added detailed explanations for structs and enums, ensuring better context for their roles in memory operations and data handling. * Enhance documentation across memory modules - Updated module-level documentation in `ingestion_queue.rs`, `ingestion.rs`, `response_cache.rs`, `client.rs`, and `mod.rs` to provide clearer descriptions of their functionalities within the OpenHuman memory system. - Improved function-level comments to clarify the purpose and usage of key functions, enhancing understanding of document ingestion, caching mechanisms, and memory client interactions. - Added detailed explanations for structs and enums, ensuring better context for their roles in memory operations and data handling. * Enhance documentation for memory store modules - Added comprehensive module-level documentation for `factories.rs` and `memory_trait.rs`, detailing their functionalities within the OpenHuman memory system. - Improved function-level comments to clarify the purpose and usage of key functions, enhancing understanding of memory instance creation and management. - Included detailed explanations for the `UnifiedMemory` implementation, ensuring better context for its role as a generic memory backend. * Refactor and enhance documentation across various modules - Removed unnecessary blank lines in `main.rs`, `chunker.rs`, `embeddings.rs`, `ingestion.rs`, `relex.rs`, `rpc_models.rs`, `memory_trait.rs`, `manifest.rs`, `mod.rs`, `ops.rs`, `qjs_engine.rs`, `socket_manager.rs`, and other files to improve code readability. - Improved comments and documentation in several modules to clarify the purpose and functionality of key components, enhancing overall understanding of the codebase. - Ensured consistent formatting and organization of comments throughout the code, contributing to better maintainability and clarity for future development. * fix import * format |
||
|
|
8aa5547af4 |
feat: Telegram managed login, channel messaging, and inbound agent loop (#338)
* feat: add managed Telegram login flow and API endpoints - Introduced a new managed Telegram login flow, allowing users to link their Telegram accounts via a deep link. - Added `telegram_login_start` and `telegram_login_check` functions to handle the creation of link tokens and status checks for user linking. - Updated the API with new endpoints for managing Telegram login, including detailed response structures for link token creation and status verification. - Enhanced the `.env.example` file to include a configuration option for the Telegram bot username, facilitating easier setup for developers. * refactor: update Telegram integration to use core RPC for login flow - Replaced the managed DM API with core RPC calls for initiating and checking Telegram login status. - Introduced new API methods `telegramLoginStart` and `telegramLoginCheck` to handle link token creation and verification. - Updated the TelegramConfig and MessagingPanel components to utilize the new login flow, enhancing the user experience and simplifying the codebase. - Adjusted tests to reflect changes in the login process and ensure proper functionality. * docs: update TODO list with new user interaction registration task for memory skill * feat: add ChannelSetupModal for configuring channel integrations - Introduced a new reusable modal component, ChannelSetupModal, for configuring channel integrations such as Telegram and Discord. - The modal can be opened from the Skills page or Settings, enhancing user experience by providing a centralized configuration interface. - Updated MessagingPanel and Skills components to integrate the new modal, allowing users to easily manage their channel settings. - Implemented channel-specific configuration components for better modularity and maintainability. * feat: add ChannelSetupModal for configuring channel integrations - Introduced a new reusable modal component, ChannelSetupModal, for configuring channel integrations such as Telegram and Discord. - The modal can be opened from the Skills page or Settings, enhancing user experience by providing a centralized configuration interface. - Updated MessagingPanel and Skills components to integrate the new modal, allowing users to easily manage their channel settings. - Implemented channel-specific configuration components for better modularity and maintainability. * style: update channel components for improved UI consistency - Refactored styles across various channel configuration components, including ChannelCapabilities, ChannelConfigPanel, ChannelFieldInput, ChannelSelector, DiscordConfig, TelegramConfig, and WebChannelConfig. - Enhanced color schemes and layout for better readability and visual appeal, ensuring a cohesive design across the application. - Updated status badge styles in definitions.ts to align with the new design standards. - Improved error message visibility and loading indicators in Channels page for a more user-friendly experience. * refactor: update WebChannelConfig and tests for improved clarity and functionality - Renamed the `definition` prop in WebChannelConfig to `_definition` for clarity. - Updated test cases in DiscordConfig and TelegramConfig to reflect changes in auth mode labels, ensuring accurate rendering of UI elements. - Adjusted the TelegramConfig test to fix the click event on the connect button, enhancing test reliability. - Modified channel definitions to streamline the managed DM auth mode, improving code organization and maintainability. * fix: enhance WebGL error handling in MeshGradient and RotatingTetrahedronCanvas components - Added error handling in the MeshGradient component to gracefully catch WebGL initialization failures, ensuring the app remains functional when the GPU context is unavailable. - Updated the RotatingTetrahedronCanvas component to verify WebGL context availability before initializing the renderer, preventing crashes and improving user experience. - Modified channel definitions to update auth mode labels for clarity and consistency across the application. * style: update RotatingTetrahedronCanvas colors and animation speed - Changed fill and edge colors in the RotatingTetrahedronCanvas component for improved visual appeal. - Adjusted opacity of the fill material to enhance transparency effects. - Introduced a speed multiplier for rotation animations, allowing for more dynamic visual effects. * refactor: improve WebGL error handling and cleanup in MeshGradient component - Removed unused canvas reference and WebGL context probing to streamline the MeshGradient component. - Enhanced error handling during shader compilation to throw an error when WebGL context may be lost, improving robustness. - Wrapped initialization logic in a try-catch block to gracefully handle failures, ensuring the gradient functionality is disabled if initialization fails. * refactor: simplify Telegram login flow by removing unused link token check - Removed the `check_channel_link_status` method from the `BackendOAuthClient`, streamlining the authentication process. - Updated `telegram_login_check` to directly fetch the user profile and determine link status based on the presence of `telegramId`, enhancing clarity and reducing complexity. - Adjusted logging to reflect the new flow and ensure accurate debugging information. * feat: enhance JSON-RPC method invocation with session expiration handling - Added logic to automatically clear stored session on receiving a 401 error from the backend, improving user experience by ensuring users are redirected to the login page when their session expires. - Introduced a helper function to identify session expiration errors, enhancing code clarity and maintainability. - Refactored the `invoke_method` function to incorporate this new error handling mechanism, ensuring robust session management during RPC calls. * style: update app-dotted-canvas background gradient for improved visibility - Changed the radial gradient background color in the app-dotted-canvas from rgba(0, 0, 0, 0.2) to rgba(0, 0, 0, 0.5), enhancing the contrast and overall visual appeal of the component. * feat: implement channel messaging API with end-to-end testing script - Added a new script `test-channel-messaging.sh` for end-to-end testing of message sending to Telegram via the backend API. - Implemented new backend API methods for sending messages, reactions, creating threads, updating threads, and listing threads in channels. - Enhanced the `BackendOAuthClient` with methods to handle channel messaging operations. - Updated controller schemas and handlers to support the new messaging functionalities, ensuring robust interaction with the backend. - Improved logging for better debugging and tracking of channel operations. * feat: add test-channel-receive script for real-time channel message listening - Introduced a new script `test-channel-receive.mjs` that connects to the backend Socket.IO server and listens for incoming channel messages. - Implemented session token retrieval and validation against the backend, ensuring secure message handling. - Added options for debugging and sending test messages, enhancing the script's usability for testing purposes. - Improved logging for better visibility of connection status and incoming messages. * feat: implement inbound channel message handling for real-time responses - Added functionality to handle inbound messages from channels (e.g., Telegram, Discord) by introducing the `handle_channel_inbound_message` function. - Implemented agent inference loop to process messages and send replies back to the originating channel via the REST API. - Enhanced logging for better tracking of message reception and response handling. - Integrated timeout handling to manage long-running requests effectively. * style: fix prettier formatting for channel components Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
|
09b627ccc9 |
Redesign billing: pay-as-you-go first-class + coupon redemption (#337)
* Refactor BillingPanel and introduce new billing components - Refactored the BillingPanel component to streamline its structure and improve readability by removing unused constants and functions. - Introduced new components: AutoRechargeSection, InferenceBudget, PayAsYouGoCard, and SubscriptionPlans to modularize billing functionalities. - Added coupon redemption functionality in the credits API, allowing users to redeem coupon codes for credits. - Enhanced the overall billing experience by improving the layout and organization of billing-related UI elements. * style: apply prettier formatting to billing components Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |