mirror of
https://github.com/tinyhumansai/openhuman.git
synced 2026-07-27 21:08:00 +00:00
88eb5ac46126dcb0fc265cd960fa172537f14383
1013
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Steven Enamakel
|
88eb5ac461 | Merge remote-tracking branch 'origin/main' | ||
|
Steven Enamakel
|
ca4eb39e9b | fix(oauth): enhance logging to include Notion-Version in fetch requests | ||
|
Mega MindandGitHub
|
5551e1e0aa |
Fix/365 enforce latest oauth gate from 351 (#421)
* feat: display app version in settings panel * fix(onboarding): auto-refresh accessibility state after grant (#351) * style(onboarding): apply formatter for issue #351 fix * fix(onboarding): ESLint + typed mock for ScreenPermissionsStep; clear flag in handler Consolidate tauriCommands imports and drop redundant mock cast. Handle granted accessibility in focus/visibility callback instead of a follow-up effect. Made-with: Cursor * fix(release): gate OAuth deep links on minimum app version (#365) - Add semver helpers and VITE_MINIMUM_SUPPORTED_APP_VERSION / download URL in app config - Block openhuman://oauth/success when desktop build is below minimum; enqueue error, open latest-release URL, dispatch oauth:stale-app - Pass new Vite env vars through release.yml and build-windows.yml Build frontend - Document policy in docs/RELEASE_POLICY.md; note vars in app/.env.example Closes #365 Made-with: Cursor * fix: import React in SkillSetupWizard component * fix: address CodeRabbit review (OAuth gate, semver, logging) - Anchor semver regex to full string; arrow-style exports; tests for bad inputs - Never throw from evaluateOAuthAppVersionGate; try/catch in deep link + omit raw URL from error logs - Document build-time vs runtime policy in config JSDoc and RELEASE_POLICY - Remove unused React import in SkillSetupWizard (tsc) Made-with: Cursor * fix: CodeRabbit — fail-closed OAuth gate, tauri-action Vite env, runbook - Block OAuth when minimum is set but getVersion fails or version is unparseable - Pass VITE_MINIMUM_* through tauri-action env (release + Windows) so bundles match yarn build - Expand RELEASE_POLICY: artifact retirement, dual workflow env note - Friendlier copy when current version is unknown Made-with: Cursor |
||
|
Mega MindandGitHub
|
eded18a703 |
fix: refresh accessibility onboarding state after macOS grant (#351) (#420)
* feat: display app version in settings panel * fix(onboarding): auto-refresh accessibility state after grant (#351) * style(onboarding): apply formatter for issue #351 fix * fix(onboarding): ESLint + typed mock for ScreenPermissionsStep; clear flag in handler Consolidate tauriCommands imports and drop redundant mock cast. Handle granted accessibility in focus/visibility callback instead of a follow-up effect. Made-with: Cursor |
||
|
Cyrus GrayandGitHub
|
9faef5582b |
feat(welcome): redesign sign-in page to match Figma card layout (#417)
* feat(welcome): redesign sign-in page to match Figma card layout Rebuild Welcome page using the same white-card design language as the Home page — centered container with shadow, border, and consistent spacing. Adds horizontal OAuth pill buttons (Google, GitHub, Twitter), "Or" divider, email input field, and "Continue with email" CTA. * fix(hooks): remove core crate from pre-push rust:check The root Cargo.toml includes whisper-rs-sys which fails to build on macOS due to -mcpu=native in its cmake/ggml layer. The Tauri shell crate builds fine and is the only Rust target the app ships, so scope rust:check to src-tauri only. |
||
|
|
91996a1dd0 |
fix(voice): reduce dictation hallucinations and improve Fn/focus reliability (#385) (#409)
* fix(voice): add per-segment confidence validation in whisper engine (#385) Reject whisper segments with avg token log-probability below -0.7 or entropy above 2.4. Return TranscriptionResult with confidence metadata instead of plain String. Update callers in speech.rs and streaming.rs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(voice): upgrade default STT model from tiny to base (#385) Base model produces significantly fewer hallucinations than tiny, especially in noisy/quiet conditions. User can still override via config. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(voice): add real-time silence gating in audio capture (#385) Gate sustained silence (>500ms) from being sent to whisper to prevent hallucinations. Maintain 100ms look-ahead ring buffer so speech onset after pauses is not clipped. Thresholds adapt to source sample rate. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(voice): fix Fn key timing race condition in hotkey event loop (#385) start_recording() blocks 1-7s on cpal device init but macOS fires Fn Release almost immediately, causing skipped cycles. Move recording start to spawn_blocking so the event loop stays responsive. Buffer Release events during setup and ensure minimum 1.5s recording duration when release arrives before recording handle is ready. Also includes: capture focused app on hotkey press, pass through pipeline for focus validation before paste. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(voice): validate and restore focus before paste, attempt regardless (#385) Add expected_app parameter to insert_text(). Before Cmd+V, validate focus via accessibility API and restore via AppleScript if shifted. Don't abort paste on focus validation failure — attempt insertion regardless so text is never silently lost. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(voice-ui): align voice server RPC response shape in settings panel (#385) * style(voice): apply rustfmt formatting in text_input * fix(voice): address CodeRabbit regressions in server, streaming, and settings polling --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
YellowSnnowmannandGitHub
|
fb8987bcad |
Improve inline autocomplete reliability, sanitization, and debug logging (#407)
* Enhance autocomplete functionality and logging. Increased debounce time for autocomplete suggestions and added minimum context character requirement. Improved inline suggestion handling with new cleanup logic for tab acceptance. Introduced a new logging option for autocomplete-only logs in CLI. Updated various components to support these changes, including sanitization and error handling in the autocomplete engine. * Add autocomplete CLI adapter for improved argument handling This commit introduces a new module, , which encapsulates the argument parsing and logging logic specific to the autocomplete namespace in the CLI. Key features include extraction of leading verbose flags, handling of the flag, and improved help message printing. The existing CLI command handling has been refactored to utilize this new adapter, enhancing code organization and maintainability. * Refactor inline completion sanitization and enhance context handling |
||
|
|
000b40bf43 |
fix(local_ai): Windows Ollama discovery + DirectML GPU acceleration for GLiNER RelEx (#416)
Ollama was not found on Windows because find_system_ollama_binary lacked common Windows install paths (%LOCALAPPDATA%\Programs\Ollama). The server spawn also silently swallowed errors, and the NSIS installer fallback didn't check system paths after install. GLiNER RelEx ONNX sessions were CPU-only — no execution providers were configured. Now offers DirectML (Windows), CoreML (macOS), and CUDA as GPU backends with automatic fallback. Updated the release to v0.5-onnx.2 with a DirectML-enabled onnxruntime.dll. Bundle completeness now requires the platform DLL and verifies checksums to trigger re-download on update. Changes: - Add Windows common paths to find_system_ollama_binary (install.rs) - Log and return spawn errors in start_and_wait_for_server (ollama_admin.rs) - Fall back to find_system_ollama_binary after Windows installer (ollama_admin.rs) - Add platform_execution_providers() with DirectML/CoreML/CUDA (relex.rs) - Require ORT DLL in bundle_complete check (relex.rs) - Verify platform DLL checksums in managed_bundle_complete (relex.rs) - Update release URL and SHA256 hashes for v0.5-onnx.2 (relex.rs) Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Cyrus GrayandGitHub
|
c3dd1370d7 |
update onboarding (#410)
* feat(onboarding): move Tools step from onboarding to Settings Remove the Tools toggle step from onboarding (5 → 4 steps) and add it as a dedicated Tools panel under Settings > AI & Skills. This reduces onboarding friction while keeping tool configuration accessible. * feat(onboarding): remove Local AI step from onboarding Remove the Local AI download/consent step, reducing onboarding from 4 to 3 steps (Welcome → Screen Permissions → Skills). Local AI setup is already accessible from Settings and auto-bootstraps from Home. |
||
|
Cyrus GrayandGitHub
|
db7eeee859 |
feat(onboarding): move Tools step from onboarding to Settings (#408)
Remove the Tools toggle step from onboarding (5 → 4 steps) and add it as a dedicated Tools panel under Settings > AI & Skills. This reduces onboarding friction while keeping tool configuration accessible. |
||
|
github-actions[bot]
|
17c9b39f74 | chore(release): v0.51.17 v0.51.17 | ||
|
Steven Enamakel
|
8b886df91b | Merge remote-tracking branch 'upstream/main' | ||
|
Steven Enamakel
|
7175a2ba9c | update | ||
|
github-actions[bot]
|
8d96f28f96 | chore(release): v0.51.16 | ||
|
Steven Enamakel
|
97e0974c0c | Merge remote-tracking branch 'upstream/main' | ||
|
Steven Enamakel
|
6b1b2b9ba2 | update | ||
|
github-actions[bot]
|
3361ec040f | chore(release): v0.51.15 | ||
|
github-actions[bot]
|
4dd9da88a9 | chore(release): v0.51.14 | ||
|
github-actions[bot]
|
621f862095 | chore(release): v0.51.13 | ||
|
Steven Enamakel
|
d05b724fd3 | Merge remote-tracking branch 'upstream/main' | ||
|
github-actions[bot]
|
3ac6bc6a8a | chore(release): v0.51.12 | ||
|
Steven Enamakel
|
cb0cec0c70 | fix | ||
|
Steven Enamakel
|
39ce04a02d | Merge remote-tracking branch 'upstream/main' | ||
|
github-actions[bot]
|
c8c0f8bb5d | chore(release): v0.51.11 | ||
|
Steven Enamakel
|
8bda914576 | ifix buils | ||
|
github-actions[bot]
|
fbbdc07062 | chore(release): v0.51.10 | ||
|
github-actions[bot]
|
766ad5af48 | chore(release): v0.51.9 | ||
|
Steven EnamakelandGitHub
|
4afa751024 |
feat(memory): global singleton, CLI, graph extraction fixes & light storage (#383)
* feat(memory): add CLI support for memory commands - Introduced a new `memory` subcommand in the CLI for memory ingestion, graph inspection, and debugging. - Implemented various subcommands including `ingest`, `docs`, `graph`, `query`, and `namespaces` for comprehensive memory management. - Updated the CLI entry point to route the `memory` command appropriately, enhancing the command-line interface functionality. * refactor(memory_cli): streamline memory command ingestion and improve error handling - Simplified the ingestion process by removing unnecessary workspace directory creation and embedding logic. - Updated the ingestion function to utilize the `create_memory_client` for better client management. - Changed the limit parameter type from `usize` to `u32` for consistency and improved error handling in command arguments. - Enhanced logging for ingestion start to focus on model name only, removing redundant extraction mode information. * refactor(memory): implement global memory client singleton for improved resource management - Introduced a new `global.rs` module to manage a process-global memory client singleton, ensuring consistent access across subsystems. - Updated `create_memory_client` to utilize the global client, enhancing memory management and reducing resource contention. - Refactored various modules to replace local memory client instances with the global singleton, improving performance and reliability. - Adjusted CLI and screen intelligence components to leverage the global memory client for document persistence and ingestion operations. This refactor enhances the architecture by centralizing memory client management, leading to better resource utilization and simplified code structure. * feat(memory): add put_doc_light for screen-intelligence, skip vectors/graph Screen-intelligence captures are too frequent and ephemeral to justify vector embedding and GLiNER graph extraction per frame. Adds a lightweight storage path (put_doc_light) that persists the document row and markdown file without chunking, embedding, or graph extraction. Three-tier storage: - put_doc_light: DB + markdown only (screen-intelligence) - put_doc: DB + markdown + vectors + background graph (skill sync) - ingest_doc: full synchronous pipeline (CLI, debugging) * style: apply cargo fmt formatting * fix: add missing window_id field in AppContext test helper * fix(test): fall back to per-call MemoryClient when global not initialized In tests with isolated OPENHUMAN_WORKSPACE, the process-global singleton may not be initialized or may point at the wrong directory. Fall back to creating a client from Config (which respects env vars) when the global is not ready. * fix(test): use per-call MemoryClient for screen-intelligence persistence put_doc_light does no background work (no vectors, no graph), so a per-call client created from Config is safe and avoids the global singleton which may point at a different workspace in test suites. |
||
|
|
a98817917c |
feat(screen-intelligence): standalone server with OCR + vision pipeline (#382)
* feat(screen-intelligence): add new commands for diagnostics and vision processing - Introduced `doctor` command for system readiness diagnostics, checking permissions and platform support. - Added `vision` command to analyze frames and persist vision summaries. - Updated CLI usage documentation to reflect new command options and improved verbosity handling. - Enhanced the `run_server` function to provide detailed endpoint information for better user guidance. This update improves the functionality and usability of the screen intelligence CLI, enabling better diagnostics and vision processing capabilities. * feat(screen-intelligence): update CLI endpoints for status monitoring - Added `tower-http` dependency for enhanced HTTP capabilities. - Updated endpoint documentation in `run_server` to reflect changes from SSE to long-polling for status updates. - Renamed `/events` endpoint to `/watch` with a query parameter for interval control, improving clarity and usability. This update enhances the screen intelligence CLI by providing more flexible status monitoring options. * feat(screen-intelligence): integrate standalone server for screen intelligence - Added a new `server` module to handle the standalone screen intelligence server functionality. - Updated CLI commands to include an `--auto-start` option for initiating capture sessions on server boot. - Enhanced the `run_server` function to provide detailed endpoint information and improved logging for server status. - Integrated the screen intelligence engine with JSON-RPC and REST endpoints for better debugging and usability. This update significantly enhances the screen intelligence capabilities by allowing it to run independently and providing more flexible configuration options. * refactor(screen-intelligence): simplify CLI options and server configuration - Removed the `--port` and `--auto-start` options from the CLI for the `screen-intelligence run` command, streamlining the command usage. - Updated the server configuration to focus on session duration and logging, enhancing clarity and usability. - Adjusted the documentation to reflect the new command structure and improved logging details for the screen intelligence server. This refactor improves the user experience by simplifying command options and enhancing the clarity of server operations. * feat(screen-intelligence): enhance screenshot management and CLI options - Updated the CLI usage documentation to include the new `--keep` option for retaining screenshots after processing. - Modified the server configuration to support the `keep_screenshots` flag, allowing for immediate saving of screenshots to disk. - Enhanced the `run_server` function to reflect the updated configuration and logging details regarding screenshot retention. - Improved the capture logic to prioritize window ID for more reliable screenshot capturing on macOS. These changes improve the usability and functionality of the screen intelligence feature by providing better control over screenshot management. * refactor(accessibility): improve capture logic and window ID handling - Updated the capture mode logic to prioritize reliable window ID capture on macOS, falling back to fullscreen capture to avoid issues with region-based capture. - Refactored the method for resolving the frontmost window ID to utilize Swift for better performance and reliability, replacing the previous JavaScript-based approach. - Enhanced the frame processing in the accessibility engine to track processed timestamps, preventing re-analysis of the same screenshot and improving efficiency. - Adjusted the vision summary parsing to support both JSON and plain text formats, ensuring better compatibility and usability. These changes enhance the reliability and performance of the accessibility features, particularly in macOS environments. * feat(accessibility): integrate Apple Vision OCR and enhance LLM context analysis - Implemented Apple Vision OCR for extracting text from screenshots, improving accuracy and reducing hallucination. - Refactored the vision processing flow to include structured prompts for LLM context analysis, enhancing the quality of the output summary. - Updated the vision summary structure to include detailed fields such as APP, DOING, FOCUS, and MOOD, providing clearer insights into the captured content. - Improved error handling and logging for image processing and OCR operations, ensuring better reliability and debugging capabilities. These changes significantly enhance the accessibility engine's ability to analyze and summarize screen content effectively. * refactor(screen-intelligence): enhance capture logic and summary structure - Improved capture mode handling to prioritize window ID availability, with a fallback to bounds-based capture before defaulting to fullscreen. - Updated the vision summary structure to clearly separate synthesized summaries, visual context, and raw OCR text for better clarity and usability. - Enhanced logging to provide more informative output during capture operations, aiding in debugging and performance monitoring. These changes improve the reliability and clarity of the screen intelligence features, particularly in managing capture contexts and summarizing visual information. * refactor(accessibility): enhance capture logic to reject fullscreen fallback - Updated the screen capture logic to prevent falling back to fullscreen mode when no valid window ID or bounds are available, improving privacy and user intent. - Added detailed logging for cases where fullscreen capture is refused, aiding in debugging and understanding capture decisions. - Introduced tests to ensure that the new logic correctly rejects fullscreen capture under invalid conditions, enhancing reliability. These changes improve the accessibility engine's handling of screen captures, ensuring that only relevant content is captured. * feat(screen-intelligence): implement capture and processing workers for enhanced vision analysis - Introduced a dedicated `capture_worker` to manage screenshot capturing from the foreground context, ensuring efficient frame handling and immediate saving of screenshots when configured. - Added a `processing_worker` to analyze captured frames using Apple Vision OCR and LLM, synthesizing insights and persisting results to unified memory. - Refactored the `AccessibilityEngine` to integrate these workers, improving the overall architecture and separation of concerns in the screen intelligence module. - Enhanced logging throughout the capture and processing workflows for better debugging and performance monitoring. These changes significantly improve the screen intelligence capabilities by enabling real-time capture and analysis of visual data, enhancing user experience and functionality. * Merge remote-tracking branch 'upstream/main' into fix/screen-intgellignce-2 * refactor(screen-intelligence): improve type handling and visibility in capture and processing modules - Updated the calculation of baseline milliseconds in `capture_worker` to ensure proper type handling with floating-point division. - Made several fields in `SessionRuntime` and `EngineState` public for better accessibility within the crate. - Changed the visibility of the `analyze_frame` function in `processing_worker` to public within the crate, allowing it to be called from `engine.rs`. These changes enhance type safety and improve the modularity of the screen intelligence components, facilitating better integration and usage across the codebase. * refactor(screen-intelligence): update visibility and helper function usage in engine and processing modules - Changed the `SessionRuntime` struct to be public within the crate, enhancing accessibility. - Updated the `analyze_frame` function parameter to use a reference instead of a direct reference to the `AccessibilityEngine`, improving clarity. - Refactored calls to `persist_vision_summary` to utilize the helper function from the `super::helpers` module, promoting better organization and code reuse. These changes streamline the code structure and improve the modularity of the screen intelligence components. * feat(screen-intelligence): introduce input handling and autocomplete features - Added a new `input.rs` module to manage input actions, including keyboard/mouse automation and predictive text functionalities. - Implemented methods for handling input actions, generating autocomplete suggestions, and committing selected suggestions within the `AccessibilityEngine`. - Created a `state.rs` module to encapsulate engine state management, including session runtime and engine state structures. - Enhanced the `engine.rs` module to support session lifecycle management and integrate new input functionalities. - Introduced a `vision.rs` module for vision-related query methods, improving the overall architecture and modularity of the screen intelligence components. These changes significantly enhance user interaction capabilities and streamline the management of input actions and vision processing. * fix(screen-intelligence): update worker imports to use state module Workers imported AccessibilityEngine from engine.rs but it was moved to state.rs during the refactor. Fix the import paths. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(screen-intelligence): clean up formatting and improve readability in various modules - Reformatted print statements in `screen_intelligence_cli.rs` for better readability. - Simplified assertions in tests within `capture.rs` to enhance clarity. - Streamlined function definitions and calls in `focus.rs`, `engine.rs`, and `processing_worker.rs` for improved code organization. - Updated import statements in `mod.rs` and `server.rs` to maintain consistency and clarity. These changes enhance the overall readability and maintainability of the codebase, promoting better coding practices across the screen intelligence components. * refactor(screen-intelligence): update import paths for AccessibilityEngine and EngineState - Changed the import statements in `tests.rs` to source `AccessibilityEngine` and `EngineState` from the `state` module instead of the `engine` module. - This adjustment aligns with recent refactoring efforts to improve module organization and maintainability. These changes enhance code clarity and ensure consistency in module usage across the screen intelligence components. * fix(screen-intelligence): fix test compilation and assertions - Update test imports from engine to state module - Add mock env var path to processing_worker::analyze_frame for test support - Update parser tests for plain-text mode (non-JSON fallback) - Handle missing window_id gracefully in capture_scheduler test Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(screen-intelligence): enhance code clarity and performance in various modules - Updated command-line usage documentation in `screen_intelligence_cli.rs` to reflect new options. - Improved string handling for truncation in `run_start_session` and `run_vision` functions to use character counts instead of byte lengths, ensuring accurate truncation for multi-byte characters. - Added intentional blocking sleep in `resolve_frontmost_window_id` to minimize impact on the Tokio runtime during app switches. - Implemented a consistent default confidence score in vision processing and improved YAML escaping in `persist_vision_summary`. - Refactored session management in `capture_worker` and `engine` modules to reduce lock contention and improve performance during I/O operations. These changes enhance the overall performance, maintainability, and user experience of the screen intelligence components. * fix(tests): update confidence assertion in vision summary test - Adjusted the expected confidence value in the `parse_vision_missing_fields` test to reflect the new default confidence score of 0.8, ensuring consistency across JSON and plain-text branches. This change improves the accuracy of the test and aligns it with recent updates in the vision processing logic. * refactor(screen-intelligence): improve code formatting and readability in various modules - Enhanced string formatting for truncation in `run_vision` to improve clarity. - Reformatted variable declarations in `capture_worker` for better readability. - Updated YAML escaping logic in `helpers.rs` for consistency and clarity. These changes contribute to improved maintainability and readability of the codebase. * refactor(screen-intelligence): reorganize analyze_frame function for improved flow - Moved configuration validation to the beginning of the `analyze_frame` function to ensure proper setup before processing. - Reintroduced the Apple Vision OCR and Vision LLM steps in the correct order, enhancing the logical flow of the analysis process. These changes improve the readability and maintainability of the code, ensuring a clearer structure for future modifications. --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
|
8627cee960 |
Feat/overlay (#378)
* feat(autocomplete): add overlay TTL configuration to AutocompletePanel - Introduced `overlay_ttl_ms` parameter to the Autocomplete configuration, allowing users to set the overlay display duration. - Updated AutocompletePanel to include a new input field for adjusting the overlay TTL in milliseconds. - Enhanced parsing and saving logic to handle the new configuration parameter. - Added corresponding tests to ensure functionality and validate the new overlay TTL feature. This update improves user control over the autocomplete overlay behavior, enhancing the overall user experience. * Refactor accessibility code for improved readability and consistency - Simplified log statements in `precompile_helper_background` for better clarity. - Reformatted `detect_input_monitoring_permission` check in `keys.rs` for enhanced readability. - Rearranged imports in `mod.rs` to maintain consistent structure. - Improved formatting of `ElementBounds` initialization across multiple test cases in `overlay.rs` and `types.rs` for better visual alignment. - Enhanced test context creation in `types.rs` for improved clarity. These changes enhance code maintainability and readability across the accessibility module. * fix(overlay): parent core RPC, voice toggle, and debug for #342 - Pass OPENHUMAN_OVERLAY_PARENT_RPC_URL from sidecar spawn and strip inherited OPENHUMAN_CORE_PORT so the overlay no longer fights for the parent listen port. - Overlay UI uses HTTP JSON-RPC to the parent sidecar for globe, debug, and voice STT so state matches the main app; add parentCoreRpc helper mirroring legacy method aliases. - Skip embedded JSON-RPC server when parent URL is set; use OPENHUMAN_OVERLAY_EMBEDDED_CORE_PORT (default 7799) for standalone dev. - Fix screen intelligence status method name; add voice_status polling, STT section, collapsible debug summary, and connection banner when core is unreachable. - Voice capture switch gates the mic with real STT availability from voice_status. Closes #342 Made-with: Cursor * fix: address CodeRabbit review on overlay/autocomplete (PR #378) - helper: correlate JSON-RPC replies with monotonic request ids; discard mismatched lines until deadline (fixes stale response after timeout). - helper: serialize Swift compile via HELPER_COMPILE_LOCK (precompile vs first use). - Overlay Tab hint: pass tab_hint from Rust from accept_with_tab; Swift hides hint when empty. - keys: re-check Input Monitoring on an interval when denied so grant without restart works. - engine: use non-zero confidence placeholder (0.75) until inline_complete returns scores. - overlay dedupe: suppress identical badge only within 400ms, not for process lifetime. Co-authored-by: Code review feedback <noreply@github.com> Made-with: Cursor * fix(ci): resolve clippy and warning issues for Rust gates - Use match on anchor_bounds in autocomplete overlay (avoid unnecessary unwrap) - Drop unused test imports in registry_ops and rpc dispatch - Prefix unused notion_doc_id in subconscious integration test Made-with: Cursor * fix(overlay): improve parent RPC URL handling in App component - Updated the useEffect hook to manage the parent RPC URL more robustly by introducing a mounted flag to prevent state updates on unmounted components. - Added error handling to set the parent RPC URL to null in case of invocation failure, enhancing the reliability of the component's behavior. * feat(overlay): implement timeout handling for parent core RPC requests - Introduced a default timeout for parent core RPC requests, enhancing reliability by preventing indefinite waiting for responses. - Added an AbortController to manage request timeouts, throwing a specific error message when a timeout occurs. - Updated the `callParentCoreRpc` function to accept a customizable timeout parameter, improving flexibility for RPC calls. * fix(overlay): allow stopping active recording regardless of config state - Updated the main button handler in the App component to always permit stopping an active recording when the status is "listening", improving user experience and control over the recording process. - Removed redundant code that previously checked the status before stopping the recording, streamlining the logic. * feat(overlay): update Cargo.lock with new dependencies and versions - Added new packages including `alsa`, `alsa-sys`, `arboard`, `block`, `cocoa`, `core-foundation`, `core-graphics`, `coreaudio-rs`, `coreaudio-sys`, `cpal`, `crunchy`, and `dasp_sample` to enhance functionality and support for audio processing and system interactions. - Updated existing dependencies to their latest versions for improved performance and compatibility. - Modified the `show_overlay` function in `ops.rs` to include an additional parameter, enhancing the overlay display functionality. * feat(autocomplete): add overlay_ttl_ms parameter to Autocomplete interfaces - Introduced a new optional parameter `overlay_ttl_ms` to both `AutocompleteSetStyleParams` and `AutocompleteConfig` interfaces, allowing for customizable overlay timeout settings. - This enhancement improves the flexibility of the autocomplete feature by enabling developers to specify how long the overlay should remain visible. --------- Co-authored-by: Code review feedback <noreply@github.com> Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai> |
||
|
Steven EnamakelandGitHub
|
e8fd08c800 |
feat(memory): use timestamp-prefixed document IDs for chronological sorting (#381)
Replace random UUID document IDs with `{unix_seconds}_{8char_hex}` format
so memory documents sort chronologically by filename and ID.
|
||
|
Steven EnamakelandGitHub
|
3851d1ef67 |
fix(voice): anti-hallucination, clipboard paste, Fn key reliability (#380)
* fix(dictation): update hotkey default value and documentation - Changed the default global hotkey for dictation from "CmdOrCtrl+Shift+D" to "Fn" in both the configuration schema and the associated documentation. - Updated the hotkey parsing function to recognize "Fn" as a valid key, enhancing the flexibility of hotkey configurations. - Added a test case to ensure the "Fn" key can be parsed correctly, improving the robustness of the hotkey handling functionality. * fix(voice): update default activation mode and hotkey in configuration - Changed the default activation mode for voice and dictation from "tap" to "push" in the respective configuration schemas. - Updated the default hotkey for voice commands from "ctrl+shift+space" to "Fn" across various modules and documentation. - Adjusted related tests to reflect the new defaults, ensuring consistency in behavior and expectations. * feat(voice): integrate embedded global voice server startup - Added a new asynchronous function `start_if_enabled` to the voice server module, which initializes the embedded voice server based on configuration settings. - Updated the server run logic to check if the voice server should auto-start, enhancing the startup process for the core application. - Integrated the new server startup function into the main server run logic, ensuring the voice server is launched if enabled in the configuration. * feat(voice): add VoicePanel for managing voice server settings - Introduced a new `VoicePanel` component to handle voice server configurations, including startup options, hotkeys, and runtime controls. - Updated routing in the settings page to include the new voice settings section. - Enhanced the `useSettingsNavigation` hook to support navigation to the voice settings. - Added tests for the `VoicePanel` to ensure functionality and reliability of the voice server management features. * refactor(dictation): update documentation and improve component initialization - Revised comments in `DictationHotkeyManager` to clarify the component's mounting process within the app tree. - Removed unused imports and unnecessary state management from `ServiceBlockingGate`, streamlining the component's logic. - Updated tests for `ServiceBlockingGate` to reflect changes in behavior, ensuring accurate rendering of child components based on service status. - Enhanced the `Cargo.lock` file by updating dependencies to their latest versions for improved stability and security. * fix(voice): update default skip_cleanup setting and enhance VoicePanel options - Changed the default value of `skip_cleanup` in the voice server configuration from `false` to `true` to improve transcription handling. - Reordered options in the `VoicePanel` component to ensure "Natural cleanup" is displayed alongside "Verbatim transcription" for better user clarity. - Updated tests to reflect the new default settings and ensure proper functionality of the VoicePanel component. * feat(window): add window management commands for Tauri application - Introduced a new module `window.ts` containing functions for managing window visibility and state in a Tauri application. - Implemented commands to show, hide, toggle visibility, minimize, maximize, close, and set the title of the main window. - Added checks to ensure commands are only executed in a Tauri environment, enhancing compatibility with web contexts. * feat(tauriCommands): add comprehensive Tauri command modules - Introduced multiple new modules for Tauri commands, including `accessibility`, `autocomplete`, `config`, `conscious`, `core`, `cron`, `hardware`, `localAi`, and `window`. - Each module contains functions for managing specific functionalities such as accessibility permissions, autocomplete suggestions, configuration settings, and hardware interactions. - Implemented checks to ensure commands are executed only in a Tauri environment, enhancing compatibility and reliability. - This addition significantly expands the command capabilities of the Tauri application, providing a robust framework for future development. * feat(voice): enhance audio transcription with initial prompt support - Added functionality to transcribe audio with an optional initial prompt, allowing for vocabulary bias and improved conversational continuity. - Updated the `transcribe_pcm_f32` and `transcribe_wav_file` functions to accept an `initial_prompt` parameter, enhancing recognition of specific vocabulary. - Implemented peak RMS energy tracking during audio recording for silence detection, ensuring recordings below a defined threshold are skipped. - Enhanced the voice server configuration to include a silence threshold and custom dictionary for better transcription context. - Introduced methods to build and manage recent transcripts for improved continuity across consecutive recordings. - Updated tests to validate new features and ensure proper functionality of the transcription process. * feat(voice): enhance VoiceServerConfig with silence detection and custom dictionary - Added `silence_threshold` to the `VoiceServerConfig` for improved silence detection, allowing recordings with low RMS energy to be skipped. - Introduced `custom_dictionary` to bias transcription towards specific vocabulary, enhancing recognition of names and technical terms. - Updated the `voice_transcribe` and `voice_transcribe_bytes` functions to utilize the new `initial_prompt` parameter for better context during transcription. - Adjusted the `transcribe_pcm_i16` function to accept additional parameters for improved flexibility in handling audio input. * feat(voice): add silence threshold and custom dictionary features to VoicePanel - Implemented a new input for setting the silence threshold, allowing recordings with low RMS energy to be skipped. - Added functionality for a custom dictionary, enabling users to add specific vocabulary words to improve transcription accuracy. - Updated the VoiceServerSettings interface and related functions to support the new features, ensuring seamless integration with existing settings. - Enhanced the UI in the VoicePanel to facilitate user interaction with the new settings. * feat(voice): propagate silence threshold and custom dictionary to voice server command - Added `silence_threshold` and `custom_dictionary` parameters to the `run_voice_server_command` function, ensuring these settings are utilized during voice server operations. - Enhanced integration with the existing voice server configuration to support improved transcription accuracy and silence detection. * fix(tauriCommands): update import paths for coreRpcClient - Adjusted import paths for `callCoreRpc` in multiple Tauri command modules to ensure correct referencing from the updated directory structure. - This change enhances module organization and maintains consistency across the codebase. * feat(dependencies): update Cargo.lock and Cargo.toml for new packages and versions - Added new dependencies including `arboard`, `fax`, `fax_derive`, `gethostname`, `half`, `quick-error`, `tiff`, and `x11rb` to enhance functionality and support for clipboard operations, transcription improvements, and system interactions. - Updated existing dependencies to their latest versions for better performance and compatibility. - Modified `VoicePanel` to utilize the updated settings and ensure proper handling of voice server configurations. - Enhanced the text input mechanism to use clipboard-paste for improved reliability in text insertion. * fix(voice): update skip_cleanup default value and enhance logging - Changed the default value of `skip_cleanup` in `VoiceServerConfig` from `true` to `false` to align with expected behavior and improve transcription handling. - Added detailed logging in the transcription cleanup process to provide better insights into the LLM state and cleanup decisions. - Removed unused functions related to unreliable key releases in hotkey handling to simplify the codebase. - Updated tests to reflect the new default settings for `skip_cleanup` and ensure proper functionality across components. * style: apply linter formatting fixes * fix(voice): remove unused warn import in hotkey module * test(voice): add silence threshold and custom dictionary to VoicePanel tests - Updated tests for the VoicePanel component to include new parameters: `silence_threshold` and `custom_dictionary`. - Ensured that the tests reflect the latest configuration settings for improved transcription accuracy and functionality. |
||
|
|
11f718d8bc |
feat(voice): standalone voice dictation server with hotkey support (#368)
* feat: add standalone voice dictation server with hotkey support - Introduced a new `voice` subcommand to the CLI for running a standalone voice dictation server that listens for a hotkey, records audio, transcribes it using Whisper, and inserts the result into the active text field. - Implemented configuration options for the voice server, including hotkey combination, activation mode (tap or push), and an option to skip LLM post-processing. - Added audio capture functionality using the `cpal` crate and integrated hotkey listening with the `rdev` crate for global key event handling. - Enhanced the configuration schema to include voice server settings and updated the main configuration structure accordingly. - Updated relevant modules and tests to ensure consistent behavior and functionality across the application. This feature enhances user interaction by allowing voice dictation directly into any active text field, improving accessibility and usability. * feat: add voice dictation server with hotkey support - Introduced a standalone voice dictation server that listens for a configurable hotkey to start recording audio, transcribes it using whisper, and inserts the transcribed text into the active text field. - Added CLI support for the `voice` command, allowing users to manage the voice server's configuration, including hotkey and activation mode settings. - Implemented configuration structures for the voice server, including options for automatic start, hotkey combination, activation mode, and cleanup behavior. - Enhanced audio capture functionality using the `cpal` library for microphone input and integrated text insertion using the `enigo` library for simulating keyboard input. - Updated relevant modules and schemas to support the new voice server features, ensuring a cohesive integration within the OpenHuman platform. * refactor: streamline voice server command and enhance audio capture functionality - Updated the `run_voice_server_command` function to initialize the configuration with environment overrides instead of loading from a file, improving performance and flexibility. - Refactored the audio capture logic in `start_recording` to enhance thread management and error handling, ensuring a more robust audio stream setup. - Improved the handling of audio stream creation and playback, ensuring that all cpal objects are managed on the same thread as required, enhancing stability during recording operations. * fix: remove unused import in voice server module - Eliminated the `HotkeyListenerHandle` import from the `server.rs` file, streamlining the code and improving clarity by removing unnecessary dependencies. * feat(voice): auto-enable LLM cleanup when local model is ready The postprocessor now checks the local LLM state and automatically enables transcription cleanup when the model is downloaded and ready, even if not explicitly configured. Falls back gracefully to raw text when the LLM is unavailable. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: apply cargo fmt + prettier formatting Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(voice): dictation config, hotkey lifecycle, and WebSocket streaming (#332) Add the foundational infrastructure for voice dictation (EPIC #332): **Rust core:** - New `DictationConfig` schema with serde defaults and env var overrides (enabled, hotkey, activation_mode, llm_refinement, streaming, interval) - RPC controllers: `config_get_dictation_settings` / `config_update_dictation_settings` - WebSocket endpoint `/ws/dictation` for streaming PCM16 transcription with periodic partial inference and final LLM refinement - Microphone permission declaration (`NSMicrophoneUsageDescription`) in Tauri macOS bundle config **Frontend:** - `useDictationHotkey` hook: fetches config from core RPC, auto-registers global hotkey, listens for `dictation://toggle` events - `DictationHotkeyManager` headless component mounted in App.tsx - Fix voice RPC response type mismatch: voice handlers return flat results (no `{result, logs}` wrapper), so remove incorrect `CommandResponse<T>` wrapping from `openhumanVoiceStatus`, `openhumanVoiceTranscribe`, `openhumanVoiceTranscribeBytes`, and `openhumanVoiceTts` Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(tauri): remove invalid infoPlist config that breaks tauri dev The `infoPlist` field in tauri.conf.json expects a string path, not an inline object. Remove it for now — microphone permission will be added via a proper Info.plist supplement in the production build pipeline. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * style: apply Prettier formatting to dictation files Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * format files * feat(dictation): integrate dictation listener and event broadcasting - Added a global dictation hotkey listener that activates based on configuration. - Implemented a web channel bridge to handle dictation events and broadcast them to connected clients. - Updated the voice module to include the new dictation listener functionality. This enhances the voice dictation capabilities by ensuring real-time event handling and client communication. * update code * format * feat(voice): enhance voice server configuration and functionality - Updated `Cargo.toml` to mark voice-related dependencies as optional. - Introduced `VoiceActivationMode` enum for better control over voice server activation. - Refactored voice server command handling and dictation event broadcasting to support new features. - Added conditional compilation for voice features across various modules, ensuring they are only included when enabled. This commit improves the modularity and configurability of the voice server, allowing for more flexible integration and usage. * refactor: clean up whitespace and formatting in core and voice modules - Removed unnecessary blank lines in `cli.rs`, `jsonrpc.rs`, `schemas.rs`, and `socketio.rs` to improve code readability. - Adjusted import order in `mod.rs` for better organization. This commit enhances the overall code quality by ensuring consistent formatting across multiple files. * chore: update Dockerfile and test workflow to install additional system dependencies - Added installation of system dependencies (cmake, ALSA, X11) in the Dockerfile for improved build support. - Updated the GitHub Actions workflow to reflect the new dependencies, ensuring consistent environment setup for testing. This commit enhances the build environment by including necessary libraries for audio and GUI support. * format * fix claude * format --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: oxoxDev <nikhil@tinyhumans.ai> |
||
|
|
07b1df4f24 |
feat(event_bus): wire webhooks, channels & skills through the event bus (#379)
* feat(event_bus): enhance domain event handling across modules - Added new `DomainEvent` variants for channel and skill events, including `ChannelMessageReceived`, `ChannelMessageProcessed`, `ChannelConnected`, `ChannelDisconnected`, `SkillLoaded`, `SkillStopped`, and `SkillStartFailed`. - Implemented event publishing in the channels and skills modules to track message processing and skill lifecycle events. - Created dedicated event bus handler files for the skills and webhooks domains, preparing for future subscriber implementations. - Updated documentation in `CLAUDE.md` to reflect the new domain events and their usage. These changes improve the observability and modularity of the system by leveraging an event-driven architecture for cross-module communication. * feat(event_bus): implement channel and webhook event handling - Introduced `ChannelInboundSubscriber` to handle inbound channel messages, triggering the agent inference loop and sending replies via the backend REST API. - Added `WebhookRequestSubscriber` to manage incoming webhook requests, routing them to the appropriate skill and handling responses. - Updated the global event bus initialization in `bootstrap_skill_runtime` to register both channel and webhook subscribers. - Enhanced `DomainEvent` with new variants for channel inbound messages and webhook requests, improving event-driven communication across modules. These changes enhance the modularity and responsiveness of the system by leveraging an event-driven architecture for channel and webhook interactions. * refactor(event_bus): update domain event documentation and subscriber initialization - Revised the documentation in `CLAUDE.md` to provide a concise overview of domain events and their associated subscriber files, enhancing clarity for future development. - Updated the `start_channels` function to initialize `WebhookRequestSubscriber` and `ChannelInboundSubscriber`, ensuring proper event handling for webhooks and channel messages. - Streamlined the event bus subscriber registration process, reinforcing the modular architecture of the system. These changes improve the maintainability and usability of the event bus framework, facilitating better cross-module communication. * style: apply cargo fmt formatting Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(event_bus): remove duplicate subscriber registration in start_channels WebhookRequestSubscriber and ChannelInboundSubscriber were registered in both bootstrap_skill_runtime() and start_channels(), causing events to be handled twice when both paths run in the same process. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(event_bus): prevent subscriber handles from being dropped on function exit SubscriptionHandle::drop aborts the background task. Since bootstrap_skill_runtime() returns immediately after setup, the local handles were dropped, cancelling both subscribers. Use std::mem::forget to leak the handles so the tasks live for the entire process. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(event_bus): ensure subscriber handles persist beyond function exit Modified the handling of subscriber registration to prevent premature dropping of handles in `bootstrap_skill_runtime()`. This change ensures that the background tasks for subscribers remain active for the entire process lifecycle, enhancing event handling reliability. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(webhooks): use proper JSON serialization for error response bodies Hand-escaped JSON strings only handled double quotes, not backslashes, newlines, or other control chars. Replaced with serde_json serialization via an error_body() helper. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(webhooks): use proper JSON serialization for error response bodies Hand-escaped JSON strings only handled double quotes, not backslashes, newlines, or other control chars. Replaced with serde_json serialization via an error_body() helper. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Steven Enamakel
|
c5acfa90e0 | Refactor Docker CI image workflow to enhance security by replacing GitHub App token generation with GITHUB_TOKEN for authentication during Docker image pushes. | ||
|
|
5affb1e75f |
feat(integrations): add agent integration tools for Twilio, Google Places, and Parallel (#375)
* feat(integrations): add agent integration tools for Twilio, Google Places, and Parallel Add a new `src/openhuman/integrations/` module with 5 backend-proxied tools that give agents access to phone calls, location search, and web search/extraction. Each tool calls the backend API which handles external API keys, billing, rate limiting, and markup — no pricing logic on the client side. Tools added: - TwilioCallTool (scope: CLI/RPC only — requires explicit user action) - GooglePlacesSearchTool, GooglePlacesDetailsTool (scope: All) - ParallelSearchTool, ParallelExtractTool (scope: All) Includes IntegrationsConfig with per-integration toggles, shared IntegrationClient with pricing cache (fetched from backend GET /agent-integrations/pricing), ToolScope enum for future scope-based filtering, and 33 unit tests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(integrations): add agent integration tools for Twilio, Google Places, and Parallel Add a new `src/openhuman/integrations/` module with 5 backend-proxied tools that give agents access to phone calls, location search, and web search/extraction. Each tool calls the backend API which handles external API keys, billing, rate limiting, and markup — no pricing logic on the client side. Tools added: - TwilioCallTool (scope: CLI/RPC only — requires explicit user action) - GooglePlacesSearchTool, GooglePlacesDetailsTool (scope: All) - ParallelSearchTool, ParallelExtractTool (scope: All) Includes IntegrationsConfig with per-integration toggles, shared IntegrationClient with pricing cache (fetched from backend GET /agent-integrations/pricing), ToolScope enum for future scope-based filtering, and 33 unit tests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix code * format code * Refactor tool integration code to improve clarity and error handling - Updated `tool_loop.rs` to cache the result of `find_tool` for efficiency and clarity in logging. - Removed unused `scope` methods from `GooglePlacesSearchTool`, `ParallelSearchTool`, and `TwilioCallTool`, simplifying the codebase. - Modified `BackendResponse` structure in `mod.rs` to use `Option<T>` for `data`, enhancing error handling when no data is returned from the backend. - Updated related tests to reflect changes in the response structure and ensure proper handling of cases with no data. These changes enhance code maintainability and improve the robustness of the integration tools. * Remove comments * Add shared HTTP client for integration tools - Introduced `IntegrationClient` to manage backend interactions, including methods for POST and GET requests with error handling. - Implemented caching for pricing information fetched from the backend. - Created a new `types` module to define shared types such as `BackendResponse` and `IntegrationPricing`, enhancing code organization and clarity. - Updated `mod.rs` to include the new client and types, ensuring proper module structure. These changes improve the maintainability and functionality of the integration tools by providing a robust client for backend communication. --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Steven Enamakel
|
5c7c104085 | Added production environment specification to Docker CI image workflow for enhanced deployment clarity. | ||
|
Steven Enamakel
|
5b3136306b | Merge remote-tracking branch 'upstream/main' | ||
|
Steven Enamakel
|
521fc7805d |
Enhance GitHub Actions workflow for Docker CI image
- Added a step to generate a GitHub App token using the tibdex/github-app-token action, improving authentication for Docker image pushes. - Updated the Docker push step to use the generated token instead of the default GITHUB_TOKEN, enhancing security and access control during the CI process. |
||
|
|
45d15cd510 |
feat: text_input domain module for focused field read/insert/ghost text (#377)
* feat: add text_input domain module for focused field read/insert/ghost text Thin orchestration layer over accessibility APIs — provides a clean RPC surface for reading, inserting, and previewing text in OS-focused input fields. Consumed by autocomplete and voice control. Five RPC methods: read_field, insert_text, show_ghost, dismiss_ghost, accept_ghost. Standalone CLI (`openhuman text-input`) and JSON-RPC server mode for testing. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: apply cargo fmt to text_input module Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Steven Enamakel
|
23212f8b76 | update dicker | ||
|
Steven EnamakelandGitHub
|
bfe4918a85 |
Enhance E2E testing environment by updating Dockerfile and workflows (#376)
- Updated the Dockerfile to include additional system dependencies for E2E testing, such as xvfb, dbus, and webkit2gtk-driver, along with the installation of tauri-driver. - Modified the docker-compose.yml to use the shared CI image for E2E tests, streamlining the build process and ensuring consistency across environments. - Deprecated the previous E2E Dockerfile, consolidating the setup into the CI image for improved maintainability and reduced redundancy. - Adjusted the GitHub Actions workflow to reflect changes in the Dockerfile and ensure proper context for building the image. |
||
|
|
88257d3231 |
feat(update): core sidecar self-update with version detection (#372)
* update * update * Refactor code style for consistency and readability in core update module - Reformatted platform_triple function to improve readability by aligning braces. - Simplified async function calls and error handling in various places for better clarity. - Enhanced logging statements for improved observability during update processes. These changes enhance the maintainability of the codebase while ensuring consistent formatting across the module. * feat(update): periodic background update checker with config flag Add a periodic update scheduler that checks GitHub Releases for newer core binary versions on a configurable interval (default: 1 hour). Controlled by `[update]` config section with `enabled = true` by default. - New `UpdateConfig` in config schema (enabled, interval_minutes) - Env var overrides: OPENHUMAN_AUTO_UPDATE_ENABLED, OPENHUMAN_AUTO_UPDATE_INTERVAL_MINUTES - Background scheduler spawned at server startup in run_server() - Reports to health registry as "update_checker" component Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(update): address PR review — restart path, security, version tracking - CoreProcessHandle: add set_core_bin/effective_core_bin so ensure_running launches the newly staged binary instead of the original one - check_and_update_core: add `force` param — auto-check uses MINIMUM_CORE_VERSION, manual apply_core_update uses latest release (force=true) - Acquire restart_lock before download+staging to prevent concurrent updates; shutdown old process before staging; use unique temp filename - check_core_update Tauri command now queries GitHub for latest_version and returns update_available alongside outdated - Harden update_apply RPC: validate download URL is GitHub HTTPS, validate asset_name is safe filename starting with openhuman-core-, ignore caller staging_dir (always use safe default) - download_and_stage accepts target_version so installed_version reflects the staged release, not the running process - Add update.check and update.apply to about_app capability catalog - ops.rs: unwrap_or_default → unwrap_or_else with error context - tauriCommands.ts: convert to arrow-function exports, add latest_version and update_available to CoreUpdateStatus interface Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * format --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
|
264bfe4081 |
feat(local_ai): sentiment analysis, GIF decision & Tenor search (#373)
* feat(local_ai): add sentiment analysis, GIF decision, and Tenor search Extend the local model with two new capabilities: - Emotion/sentiment analysis (joy/sadness/anger/etc + valence + confidence) via a lightweight prompt, designed to run periodically (~hourly) - GIF decision + Tenor search: local model decides when a GIF response fits, generates a search query, and proxies through the backend's new Tenor API (POST /agent-integrations/tenor/search) New RPC endpoints: - openhuman.local_ai_analyze_sentiment - openhuman.local_ai_should_send_gif - openhuman.local_ai_tenor_search Frontend integrates with cadence-based invocation: - Reactions: every message (unchanged) - GIF decisions: every ~7 messages - Sentiment analysis: every ~1 hour Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: apply formatter fixes Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * merge g Please enter the commit message for your changes. Lines starting --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
|
a36535330b |
feat: add typed event bus for cross-module decoupling (#374)
* Add event bus integration to channels module - Introduced an `event_bus` module to facilitate event-driven communication within the channels. - Updated `ChannelRuntimeContext` to include an `event_bus` field for managing events. - Enhanced the `start_channels` function to initialize the global event bus and register a tracing subscriber for logging domain events. - Modified the cron scheduler to publish delivery requests as events, decoupling the delivery logic from specific channel implementations. - Updated tests to ensure proper initialization and usage of the event bus across various contexts. This change improves the modularity and scalability of the channels system by leveraging an event-driven architecture. * refactor: move CronDeliverySubscriber to cron/bus.rs Each domain owns its event bus handlers — move the delivery subscriber from channels/cron_delivery.rs into cron/bus.rs so the cron module contains both its publisher and subscriber logic. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update Dockerfile to install system dependencies and refactor event bus logging - Added installation of system dependencies (cmake, ALSA, X11) in the Dockerfile to support build requirements. - Refactored event bus logging in various files for improved readability by consolidating multi-line log statements into single lines. * chore: update Dockerfile to include webkit2gtk-driver installation - Added installation of webkit2gtk-driver in the Dockerfile to support additional system dependencies. - Removed redundant apt-get update command to streamline the installation process. * docs: add event bus usage guide to CLAUDE.md Documents the event bus pattern, core types, global access, and the convention for adding domain events and subscribers so future modules follow the established design. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update Dockerfile to streamline Node.js installation - Removed redundant cleanup command after installing yarn in the Dockerfile, simplifying the installation process. * chore: update Dockerfile to fix yarn installation command - Removed unnecessary trailing whitespace in the yarn installation command in the Dockerfile, ensuring cleaner code and consistency. * chore: consolidate system dependencies installation in Dockerfile - Merged the installation of webkit2gtk-driver with other system dependencies in the Dockerfile to streamline the setup process and reduce the number of RUN commands. * refactor: enforce EventBus as a singleton - Make EventBus::create() pub(crate) — only tests can construct isolated instances; production code must use the global singleton - Add subscribe_global() convenience for subscribing from any module - Remove event_bus field from ChannelRuntimeContext — all modules use init_global/publish_global/subscribe_global instead of passing instances - Update CLAUDE.md to document the singleton API as the only way to use the event bus Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address PR review — panic safety, init_global, Dockerfiles, tests - Wrap handler.handle() in catch_unwind so a panicking subscriber does not silently kill its task — logs the panic and continues the loop - Call init_global() in cron scheduler::run() so delivery events are not silently dropped when the scheduler starts before start_channels() - Export DEFAULT_CAPACITY and use it everywhere instead of hard-coded 256 - Add ALSA/X11/input dev libraries to e2e/Dockerfile to mirror CI runner - Remove duplicate cmake install from .github/Dockerfile - Add 4 unit tests for CronDeliverySubscriber (ignore non-delivery, dispatch to channel, missing channel, send failure) - Replace scheduler announce-mode test with one that verifies event payload is actually received by a subscriber Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
|
200db04fc7 |
feat: scope user data to per-user directories (#370)
* feat(config): add user ID retrieval and workspace scoping for authenticated users - Implemented `read_authenticated_user_id` to extract the user's ID from `auth-profiles.json`, avoiding a dependency cycle with the credentials module. - Introduced `maybe_scope_workspace_to_user` to create user-specific workspace directories based on the authenticated user ID, ensuring isolated workspace data. - Updated the configuration loading process to call `maybe_scope_workspace_to_user`, enhancing user data management. - Added unit tests for the new functionality, ensuring correct behavior in various scenarios. This change improves user experience by providing personalized workspace management based on authentication status. * feat(config): enhance user management with active user state handling - Added functions to manage the active user state, including `read_active_user_id`, `write_active_user_id`, and `clear_active_user`, allowing for user-specific configuration and workspace isolation. - Introduced `default_root_openhuman_dir` to standardize the retrieval of the root directory for user data. - Updated configuration loading to support user-scoped directories, improving the overall user experience by ensuring personalized settings and workspace management. This change enhances the OpenHuman platform by enabling better user data management and isolation. * feat(credentials): enhance user directory management during session storage - Added logic to create and activate user-scoped directories based on the resolved user ID when storing session data, ensuring credentials are saved in the correct location. - Implemented error handling for directory creation and active user ID writing, with appropriate logging for failures. - Updated the configuration loading process to reflect the newly activated user directory, improving user-specific settings management. - Enhanced the `get_data_dir` function to return user-scoped directories if an active user is set, streamlining data access. This change improves user experience by ensuring that session data is correctly organized and accessible based on user context. * refactor(tests): update user ID handling and improve test coverage - Renamed and refactored tests to better reflect functionality, focusing on active user ID management. - Removed the `write_auth_profiles` helper function and replaced it with direct calls to `write_active_user_id` for clarity. - Enhanced tests to cover scenarios for reading and clearing active user IDs, ensuring accurate behavior in user-specific configurations. - Added a new test for building user directory paths, improving overall test coverage for user management features. This change streamlines the testing process and enhances the clarity of user ID handling in the configuration schema. * refactor(paths): streamline model and binary path resolution - Introduced a new `shared_root_dir` function to centralize the logic for determining the shared root openhuman directory, improving code clarity and reducing duplication. - Updated `workspace_ollama_dir` and `workspace_local_models_dir` functions to utilize the new shared root directory, ensuring consistent path resolution for user-specific and shared resources. - Enhanced the `model_artifact_path` function to leverage the new directory structure, improving the organization of model artifacts. This refactor enhances maintainability and clarity in the path management for local AI resources. * style: apply cargo fmt formatting Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(paths): streamline directory management for model artifacts - Updated the `model_artifact_path` function to utilize a new `shared_root_dir` function, which centralizes the logic for determining the root openhuman directory. - Enhanced the `config_root_dir` function to improve clarity and maintainability. - Adjusted the `workspace_ollama_dir` and `workspace_local_models_dir` functions to leverage the new shared directory logic, ensuring consistent path resolution across the application. These changes improve the organization of directory management and enhance the overall clarity of the codebase. --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
|
df503a23fd |
Fix: decouple chat from local Ollama, route inference through Rust (#369)
* refactor: streamline Conversations component by removing local response delivery logic - Eliminated the `deliverLocalResponse` function, which handled segmenting and dispatching local model responses, simplifying the message delivery process. - Updated socket status checks to remove unnecessary local model condition, ensuring clarity in connection handling. - Adjusted comments to reflect the new routing logic for cloud backend interactions, emphasizing that local models are now used only for supplementary features. * refactor: update chat service to utilize core RPC for message handling - Replaced Socket.IO message sending with core RPC calls for sending and canceling chat messages, enhancing the communication mechanism. - Improved error handling for socket connection checks, ensuring robust event routing. - Updated documentation to reflect the new RPC-based approach for chat message processing. * feat: enhance chat response handling with segmentation and emoji reactions - Introduced a new `ChatSegmentEvent` interface to support segmented message delivery, allowing for more natural chat interactions. - Updated the `Conversations` component to handle segmented responses and apply emoji reactions based on user messages. - Refactored the message delivery logic to improve clarity and maintainability, ensuring that responses are dispatched correctly based on segmentation. - Enhanced the chat service to emit segment events, facilitating a smoother user experience during conversations. * refactor: simplify Conversations component by removing unused imports and optimizing rendering logic - Removed unnecessary imports related to local model status and message segmentation, streamlining the codebase. - Updated the rendering logic to enhance clarity by simplifying conditions for displaying sending and delivering states in the chat interface. - Improved overall readability and maintainability of the Conversations component. * style: apply formatter fixes (prettier + cargo fmt) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * updated convos * fix test --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
|
9b73d00b2e |
fix(agent): inject skill tools into agent registry and unify ToolResult (#341) (#360)
* fix(agent): inject skill tools into agent registry and unify ToolResult (#341) Skill tools (Notion, Gmail, etc.) from the QuickJS runtime were invisible to the agent's tool loop — the LLM could never call them. This change: 1. Adds SkillToolBridge to wrap skill ToolDefinitions as Tool trait impls and inject them into the agent's tool registry via collect_skill_tools(). 2. Unifies ToolResult across built-in and skill tools — both now use the MCP-spec type (content blocks + is_error) from skills::types, eliminating the need for result conversion between the two systems. 3. Adds convenience constructors (ToolResult::success/error/json) and accessor methods (output/text) to simplify all tool implementations. 4. Adds diagnostic logging in the tool loop for tool registry contents and tool lookup results. Verified: agent_chat RPC successfully called Notion list-all-pages tool and returned real workspace data through the agent loop. Closes #341 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(tools): address CodeRabbit review feedback on PR #360 - cron_run: preserve execution details (result_output) on failure - delegate: fix mismatched bracket in agent response header format string - run_linter: include lint diagnostics in error output, not just exit code - run_tests: include test output in error result, not just exit code - skill_bridge: prevent namespaced tool-name collisions with dedup + __ delimiter validation; sanitize runtime errors in model-facing output Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: apply cargo fmt to skill_bridge.rs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
|
faa881c5f1 |
Feat/343 screen intelligence e2e tests (#359)
* test(screen_intelligence): add E2E pipeline proof tests for #343 Add layered test coverage proving the full capture → vision → memory pipeline: screenshot save/cleanup disk paths, VisionSummary serde roundtrip, JSON-RPC shape tests for status and vision_recent endpoints. - tests/screen_intelligence_vision_e2e.rs: save_screenshot_to_disk creates a PNG and keep_screenshots=false cleanup removes it; VisionSummary struct serializes/persists/is queryable end-to-end; platform support table + macOS checklist added to module doc - tests/json_rpc_e2e.rs: screen_intelligence_status shape test (platform_supported, session.active, permissions.screen_recording); vision_recent returns empty summaries without an active session - src/openhuman/screen_intelligence/tests.rs: save_screenshot_to_disk unit tests for the write path and the no-image-ref error path Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(screen_intelligence): enhance vision summary persistence and error handling - Added new fields to track vision persistence count, last persisted key, and last persist error in SessionRuntime and SessionStatus. - Implemented error handling for vision summary persistence, ensuring errors are logged and state is updated accordingly. - Introduced a new method to analyze a frame and persist the summary, improving the vision processing pipeline. - Updated tests to validate the new functionality and ensure proper behavior with mocked vision outputs. This commit improves the robustness of the screen intelligence pipeline by enhancing the tracking and handling of vision summary persistence. --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
YellowSnnowmannandGitHub
|
1acc916244 |
feat: extract and load user working memory from skill sync payloads (#357)
* feat: implement user working memory extraction from skill sync payloads - Added functionality to enable the extraction of user working memory from successful skill syncs, allowing for persistent storage of user preferences, goals, constraints, and entities. - Introduced a new configuration option in to toggle working memory extraction. - Created comprehensive documentation on the working memory extraction process, detailing its implementation and privacy considerations. - Updated memory loading logic to include working memory entries in the context provided to agents, enhancing personalization capabilities. - Enhanced logging for memory extraction processes to improve observability and debugging. This feature enhances the user experience by allowing skills to maintain context across interactions, improving the overall effectiveness of the OpenHuman platform. * docs: update architecture documentation to include user working memory integration * refactor: centralize working memory constants and enhance extraction logic - Moved `WORKING_MEMORY_KEY_PREFIX` and `WORKING_MEMORY_LIMIT` constants to `memory_context.rs` for better organization and accessibility. - Updated `MemoryLoader` to utilize these constants, improving code clarity. - Enhanced working memory extraction logic in `MemoryWriteJob` to conditionally persist user working-memory documents based on the job type. - Improved logging for memory extraction processes to provide clearer insights during execution. - Adjusted tests to ensure consistent behavior with the new working memory extraction logic. * chore: update OpenHuman version to 0.51.8 and refactor JSON-RPC test for clarity - Bumped the OpenHuman version in Cargo.lock from 0.51.6 to 0.51.8. - Refactored the JSON-RPC end-to-end test to improve readability by encapsulating the result assertion logic within a block, enhancing clarity in the flow of data handling. |