* refactor(voice): move standalone CLI adapter into voice domain
Move the blocking `openhuman voice` / `openhuman dictate` dictation-server
subcommand out of `src/core/cli.rs` and into a new `src/openhuman/voice/cli.rs`
owned by the voice domain. The core CLI dispatcher now routes to
`voice::cli::run_standalone_subcommand` and no longer imports voice internals.
Why this shape: the standalone server blocks forever on the hotkey listener,
which doesn't fit the controller registry's request/response contract. A
domain-owned CLI adapter is the right home for long-lived operational
commands; the controller registry stays for RPC-style capabilities.
Establishes the exposure rule documented in PLAN.md for the rest of the
backend overhaul.
- src/openhuman/voice/cli.rs: new 118-line adapter (flag parsing, tokio
runtime, run_standalone call)
- src/openhuman/voice/mod.rs: declare `pub mod cli`
- src/core/cli.rs: delete 92-line run_voice_server_command, dispatch only
- PLAN.md: new scope doc + exposure rule
* refactor(text_input): move CLI adapter into domain
Move src/core/text_input_cli.rs to src/openhuman/text_input/cli.rs and
drop the stale `pub mod text_input_cli` from src/core/mod.rs. Core CLI
dispatcher now routes `text-input` to the domain-owned adapter.
The moved file has a mixed shape that validates the exposure rule from
the previous voice commit:
- `run` starts a long-lived HTTP JSON-RPC server (domain cli.rs shape)
- `read / insert / ghost / dismiss` are short-lived UX wrappers around
`text_input::rpc::*` (pretty-print, flag parsing — CLI affordance, not
transport duplication)
Both shapes legitimately belong in the domain. The controller registry
stays for transport-agnostic capabilities; CLI UX wrappers live next to
their domain RPC.
- git mv src/core/text_input_cli.rs src/openhuman/text_input/cli.rs
- src/openhuman/text_input/mod.rs: declare `pub mod cli`
- src/core/mod.rs: drop `pub mod text_input_cli`
- src/core/cli.rs: update dispatch path
* refactor(tree_summarizer): move CLI adapter into domain
git mv src/core/tree_summarizer_cli.rs src/openhuman/tree_summarizer/cli.rs
and drop the stale `pub mod tree_summarizer_cli` from src/core/mod.rs.
Core CLI dispatcher now routes `tree-summarizer` to the domain-owned
adapter.
All five subcommands (ingest / run / query / status / rebuild) are short
-lived UX wrappers over registry handlers that already exist in
tree_summarizer/schemas.rs — no long-running server, no business logic
duplicated. Straight relocation keeps transport generic.
- git mv src/core/tree_summarizer_cli.rs src/openhuman/tree_summarizer/cli.rs
- src/openhuman/tree_summarizer/mod.rs: declare `pub mod cli`
- src/core/mod.rs: drop `pub mod tree_summarizer_cli`
- src/core/cli.rs: update dispatch path
* refactor(screen_intelligence): move + split CLI adapter into domain
Move src/core/screen_intelligence_cli.rs (699 lines, 7 subcommands) into
a domain-owned cli/ submodule, split by responsibility so no single file
exceeds the project's 500-line soft limit.
Layout:
screen_intelligence/cli/mod.rs (204) dispatch + shared helpers
(CliOpts, parse_opts,
bootstrap_engine, logging,
print_help)
screen_intelligence/cli/server.rs (79) run_server (long-running
capture+vision loop)
screen_intelligence/cli/session.rs (150) status / start / stop
screen_intelligence/cli/capture.rs (173) capture / vision (inspect pair)
screen_intelligence/cli/doctor.rs (107) readiness diagnostics
Classification before moving:
- no business-logic duplication vs schemas.rs handlers
- heavy fns (doctor 103, capture 97, start 89) were 80%+ pretty-printing
and CLI-side orchestration (polling loop, flag-driven save) — legit
CLI UX, not extracted domain logic
- shared helpers stayed pub(super) inside mod.rs; no new domain APIs
introduced
Core transport changes:
- delete src/core/screen_intelligence_cli.rs
- drop `pub mod screen_intelligence_cli` from src/core/mod.rs
- src/core/cli.rs dispatch routes to domain cli::
After this commit, src/core/cli.rs contains only dispatch lines for all
four migrated domains (voice, text_input, tree_summarizer,
screen_intelligence) — no embedded domain logic.
* refactor(screen_intelligence): address external review — extract capture save + tighten cli visibility
Follow-ups from codex + gemini review of the branch:
1. Extract CaptureFrame construction + disk save from the CLI.
`capture.rs --keep` was building a CaptureFrame inline (stamping
timestamps, copying context fields) and calling
`AccessibilityEngine::save_screenshot_to_disk` directly. Added
`AccessibilityEngine::save_capture_test_result(workspace_dir,
&CaptureTestResult, reason) -> Option<Result<PathBuf, String>>`
so the CLI only passes the result through and picks a reason
label. Domain owns the frame shape, not the CLI.
2. Tighten CLI module visibility across all four migrated domains.
`pub mod cli` -> `pub(crate) mod cli` and
`pub fn run_*_command` -> `pub(crate) fn` for voice, text_input,
tree_summarizer, screen_intelligence. Only src/core/cli.rs needs
to see these; they're transport plumbing, not domain public API.
No behavior change.
* style: cargo fmt
* address review comments from coderabbitai
- screen_intelligence/cli/server: use effective keep_screenshots (opts.keep
|| config flag) for both SiServerConfig and status output, so the server
behaves consistently with what the CLI reports.
- voice/cli: surface config load errors with a warning instead of silently
falling back to Config::default(); reject unknown --mode values instead
of silently coercing them to Push.
- PLAN.md: narrow the Phase 4 enforcement grep so it forbids domain
imports/non-dispatcher references in the transport layer while still
allowing crate::openhuman::<domain>::cli::run_*(...) dispatch calls,
which this PR legitimately introduces.
* style: cargo fmt voice cli
* style: cargo fmt + bump tauri-cef vendor
* fix(screen_intelligence): narrow save_capture_test_result to pub(crate) (addresses @coderabbitai nitpick on engine.rs:486)
AccessibilityEngine::save_capture_test_result is only called from the
domain's own cli/capture.rs (crate-internal). Making it pub leaks it
into the public API surface unnecessarily — tighten to pub(crate) to
match the visibility-tightening done across the rest of this PR.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* update
* chore: update .gitignore to include scheduled_tasks.lock in app/.claude
* ran formatter
---------
Co-authored-by: Jwalin Shah <jshah1331@gmail.com>
Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(about_app): add capability privacy metadata
Adds optional `CapabilityPrivacy { leaves_device, data_kind, destinations }`
to the about_app capability catalog so the in-app Privacy surface can be
backend-backed instead of hand-maintained. Twelve representative capabilities
are annotated for the first audited set (raw/local, derived/backend,
credentials, diagnostics, model download); remaining entries default to
None and are simply not surfaced. Wire format stays backward compatible
via skip_serializing_if.
* feat(settings): drive privacy panel from about_app capabilities
Replaces the hand-maintained privacy rows with data fetched from
openhuman.about_app_list. Only capabilities that ship privacy metadata
are rendered; loading and RPC failure both degrade gracefully and the
analytics toggle plus explanatory copy remain intact. Adds a small typed
client (utils/tauriCommands/aboutApp.ts) and focused vitest coverage for
render, omission of unannotated entries, and RPC failure.
---------
Co-authored-by: Jwalin Shah <jshah1331@gmail.com>
* feat(memory_recipes): introduce memory recipes module for source-type-specific ingestion
- Added a new `memory_recipes` module that includes functionality for handling various source types, such as email and chat, with dedicated recipes for ingestion.
- Implemented core types and the `Recipe` trait to facilitate the ingestion process, allowing for structured handling of raw content and metadata.
- Created specific recipes for email and chat, enhancing the ability to process and normalize content before storing it in memory.
- Updated the core controller registration to include new endpoints for memory recipes, enabling JSON-RPC interactions for ingesting content and listing available recipes.
- Enhanced task management by integrating task storage and query helpers, allowing for better tracking and management of action items extracted during ingestion.
These changes significantly improve the ingestion capabilities of the application, providing a robust framework for handling diverse content types.
* refactor: remove API key references from configuration and schema
- Eliminated the `api_key` field from various configuration schemas and related structures, including `Config`, `ModelSettingsPatch`, and `ComposioConfig`, to streamline the configuration process.
- Updated the handling of API keys in the application to rely solely on session JWTs, enhancing security and simplifying the authentication flow.
- Adjusted related documentation and comments to reflect the removal of the API key, ensuring clarity in the new authentication approach.
- Refactored multiple components and functions to remove dependencies on the API key, improving code maintainability and reducing complexity.
These changes significantly enhance the security posture of the application by removing reliance on static API keys and promoting the use of session-based authentication.
* chore: update .env.example and documentation for API key removal
- Removed references to `OPENHUMAN_API_KEY` from the `.env.example` file to reflect recent changes in the authentication approach.
- Updated documentation in `AGENTS.md` and `CLAUDE.md` to clarify that `OPENHUMAN_API_URL` now overrides `config.api_url`, ensuring consistency across configuration references.
- Bumped the version in `Cargo.lock` to 0.52.23 to align with the latest changes.
These updates enhance clarity in configuration management and documentation following the removal of static API keys.
* fix: address config pruning review comments
* feat(tests): add validation tests for SearchResponse deserialization
- Introduced new tests to ensure that the SearchResponse struct correctly rejects JSON inputs missing required fields: searchId, results, and costUsd.
- Updated DelegateTool instantiation to remove the unused fallback_credential parameter, simplifying the constructor and related tests.
These changes enhance the robustness of the SearchResponse handling and improve code clarity by removing unnecessary parameters.
* refactor: remove api_key params from provider factory signatures
The OpenHuman backend now only uses the app-session JWT for auth —
the api_key parameter was ignored (as _api_key) after the config
pruning. Drop it from the signatures entirely instead of leaving
dead None arguments at every call site.
Also removes the unused ChannelContext.api_key field.
* refactor: remove api_key/composio_key params from factory signatures
Every caller was passing None after the config pruning:
- create_embedding_provider / create_memory* — the api_key param
was only threaded to the openai / custom embedding branches, but
no caller has ever supplied a non-None value since the pruning.
Embeddings use an empty key; the param is gone.
- all_tools / all_tools_with_runtime — both composio_key and
composio_entity_id only fed the ComposioTool registration block,
which was already dead after the JWT migration. Drop both args
and the block; the modern path is all_composio_agent_tools via
build_composio_client(config).
* style: apply cargo fmt
* Enhance draft message handling in streaming edits
- Introduced a new `draft_sent` field in the `StreamingState` struct to track when a draft message has been posted, decoupling the existence of a draft from the ability to edit it.
- Updated the `flush_streaming_edit` function to set `draft_sent` upon posting a draft, ensuring that duplicate messages are not sent if the backend fails to return an ID.
- Modified the `finalize_channel_reply` function to prevent sending a fresh message if a draft was already posted without an ID, improving user experience by avoiding duplicate bubbles.
These changes enhance the robustness of message handling during streaming edits, ensuring a smoother interaction for users.
* feat(onboarding): enhance LocalAIStep for low-RAM device handling
- Added logic to determine if the device is below the RAM threshold, defaulting to a cloud AI fallback if necessary.
- Introduced a new UI for low-RAM devices, informing users about the cloud mode and providing options to continue with cloud AI or force-enable local AI.
- Updated the `ensureRecommendedLocalAiPresetIfNeeded` function to return a `recommend_disabled` flag based on device RAM.
- Enhanced tests to cover new cloud fallback UI and local AI consent handling.
These changes improve the onboarding experience by providing clearer options based on device capabilities.
* style: apply prettier formatting to LocalAIStep
* feat(local_ai): unlock model selection + add Disabled/cloud fallback option
- Remove MVP ceiling that clamped selection to the 2-4 GB tier, in bootstrap
and in the apply_preset RPC — users can now pick any tier.
- Add special "disabled" tier string to apply_preset that toggles
config.local_ai.enabled = false so the app uses the cloud summarizer.
- Presets RPC now returns local_ai_enabled so the UI can render the
currently-active state correctly.
- Rewrite DeviceCapabilitySection: tiers are now clickable buttons that
call apply_preset; adds a "Disabled — Cloud fallback" card at the top
marked "Recommended" on low-RAM devices and "Active" when enabled=false.
- Remove the MVP message copy from the model tier panel.
- Remove unread-count badge from the bottom tab bar.
* refactor(onboarding): streamline LocalAIStep and update local AI preset handling
- Removed the `ensureRecommendedLocalAiPresetIfNeeded` function call from `LocalAIStep`, replacing it with `openhumanLocalAiPresets` to directly fetch preset information.
- Updated the logic in `ensureRecommendedLocalAiPresetIfNeeded` to ensure the recommended tier is applied correctly based on user consent, improving clarity in the local AI setup process.
- Enhanced tests to mock the new `openhumanLocalAiPresets` function, ensuring coverage for low-RAM device scenarios and local AI consent handling.
- Updated documentation in the `schemas.rs` file to reflect changes in the data structure returned by the presets RPC.
These changes improve the onboarding experience by providing a more direct and efficient method for managing local AI presets based on device capabilities.
* test(LocalAIStep): improve mock implementation for local AI presets
- Refactored the mock for `openhumanLocalAiPresets` to enhance readability and maintainability.
- Ensured the mock returns consistent device information and preset details for testing scenarios.
- This change supports better test coverage and clarity in the LocalAIStep component's behavior during onboarding.
* fix(local_ai): preserve hadSelectedTier semantics in bootstrap return
hadSelectedTier / selectedTier represent the incoming state ("was a tier
already selected before this call"), not the post-apply state. Setting
both to the just-applied tier broke the existing localAiBootstrap
contract and the unit test that encodes it.
The Rust-side persistence fix is independent: removing the
recommend_disabled short-circuit ensures openhumanLocalAiApplyPreset
actually writes the selected tier to disk on the opt-in path, so
config_with_recommended_tier_if_unselected() honors the user's choice.
* feat: enhance local AI asset state management and error handling
- Updated the Home component to improve readiness checks for local AI assets, ensuring a more robust UI experience.
- Refactored the LocalAiService to provide clearer state management for STT and TTS models, including handling on-demand downloads and error logging.
- Enhanced the voice status function to accurately reflect the availability of transcription backends, improving overall system reliability.
- Introduced warnings for on-demand model downloads to inform users about potential delays in functionality.
* feat: implement polling for voice server status in OverlayApp
- Added a polling mechanism to periodically check the voice server status every 2 seconds, ensuring the overlay remains in sync with the server state.
- Introduced logic to activate or dismiss the speech-to-text (STT) mode based on the server's recording or transcribing state, enhancing user experience and responsiveness.
- Utilized the existing `callCoreRpc` function to fetch server status, improving reliability in state management during brief disconnects or reconnections.
* feat: refine prompt handling in LocalAiService
- Updated the prompt construction logic in `LocalAiService` to enhance clarity and functionality.
- When `no_think` is set, the system prompt now includes a directive for the model to respond with only the final answer, improving response accuracy.
- Refactored the prompt and system parameters to ensure they are correctly formatted and passed to the `OllamaGenerateRequest`, enhancing overall request handling.
* feat: enhance STT readiness checks in VoicePanel and useVoiceSkillStatus
- Updated the STT readiness logic in `VoicePanel` to account for both 'ready' and 'ondemand' states, improving the accuracy of readiness assessments.
- Refined the `useVoiceSkillStatus` hook to ensure it only blocks when the local AI's STT state is explicitly 'missing', enhancing overall system reliability and user experience.
* feat(composio): implement ComposeIO trigger history component and hook
- Added `ComposeioTriggerHistory` component to display a list of ComposeIO trigger events with formatted timestamps and payloads.
- Introduced `useComposeioTriggerHistory` hook to manage fetching and state of trigger history entries, including error handling and loading states.
- Updated `Webhooks` page to integrate the new component and hook, replacing previous webhook activity display with ComposeIO trigger history.
- Created utility functions for formatting timestamps and payloads for better readability in the UI.
- Established backend support for fetching trigger history through new Tauri commands, ensuring robust data handling and storage.
* style: apply repo formatting fixes
* feat: add fs2 dependency and enhance ComposeIO trigger history handling
- Introduced the `fs2` crate to manage file locking, improving the reliability of file operations in the ComposeIO trigger history.
- Updated `ComposioTriggerHistoryStore` to utilize exclusive file locks during archive writing, ensuring data integrity.
- Enhanced error handling for file operations, providing clearer logging for failures related to file access and locking.
- Refactored the initialization logic for global trigger history to prevent duplicate setups, improving overall stability.
* refactor: remove quickjs skills runtime
* style: apply repo formatting
* refactor: clean up error reporting and connection handling
- Removed the 'skill' source option from the error report structure to streamline error reporting.
- Refactored the ConnectionsPanel component to simplify connection status badge rendering and improve clarity.
- Updated the CronJobsPanel to enhance logging for cron job loading processes.
- Adjusted SkillCard component to use a more consistent type for icons.
- Deleted outdated end-to-end tests for Gmail and Notion skills, improving test suite maintainability.
* fix: remove unnecessary ESLint disable comment in Conversations component
- Cleaned up the Conversations component by removing the ESLint disable comment for exhaustive dependencies in the useEffect hook, improving code clarity and maintainability.
* fix: remove unnecessary whitespace in Conversations component
- Eliminated an extra line of whitespace in the Conversations component, enhancing code readability and maintainability.
* refactor: streamline SkillCard imports for improved clarity
- Combined import statements in the SkillCard component to enhance code readability and maintainability.
* feat(config): introduce pre-login user directory structure
- Added support for a pre-login user directory to encapsulate configuration, memory, and state before any user logs in. This ensures that all initial data is scoped under a dedicated user directory (`users/local`), preventing direct writes to the root `.openhuman` path.
- Implemented the `pre_login_user_dir` function to return the appropriate path for the pre-login user.
- Updated configuration loading logic to defer disk state creation until the first successful login, enhancing user data management and isolation.
- Added tests to verify the correct behavior of the pre-login directory structure.
* refactor: remove hardware-related components and streamline service management
- Deleted hardware configuration and related tools from the codebase, including `HardwareConfig`, `HardwareTransport`, and associated memory management tools.
- Introduced a new `service.ts` module for managing service and daemon commands, consolidating service-related functionalities.
- Updated import paths across the application to reflect the removal of hardware references and the addition of the new service management module.
- Refactored the `build_system_prompt` function to remove hardware access instructions, focusing on action instructions instead.
- Cleaned up the Cargo.toml and Cargo.lock files by removing unused dependencies related to hardware management.
* chore: apply formatting and tauri lockfile sync
* refactor(tests): extract config file writing logic into a reusable function
- Introduced a `write_config_file` function to encapsulate the logic for creating directories and writing configuration files, improving code reuse and readability.
- Updated test cases to utilize the new function for writing configuration files, ensuring consistency and reducing duplication.
- Added handling for pre-login user directory structure to ensure configuration is correctly written to the appropriate paths.
* feat(voice): sync overlay orb with chat voice button state (#487)
The overlay orb already reacts to hotkey-based dictation via Socket.IO
events, but the chat "Start Talking" button used local React state only.
Add a new RPC method `openhuman.overlay_stt_notify` that the chat button
calls at each voice state transition, which publishes to the existing
DICTATION_BUS / TRANSCRIPTION_BUS broadcast channels — so the overlay
reflects recording/transcribing/idle from both input paths with zero
changes to the Socket.IO bridge or overlay event handlers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(voice): address CI formatting and CodeRabbit review feedback
- Run cargo fmt and prettier to fix formatting violations
- Use typed enum OverlaySttState instead of raw String for state param
(serde rejects invalid states at deserialization, eliminating the
unknown state branch)
- Require `text` field for transcription_done state (return error if
missing instead of silently ignoring)
- Replace raw transcript logging with metadata-only (has_text, text_len)
to avoid logging sensitive user speech content
- Use "Voice input active" aria-label (covers recording + linger phases)
- Convert notifyOverlaySttState to arrow function with async/await per
repo TS conventions
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* refactor(accessibility): remove device control and predictive input features from accessibility settings
- Updated accessibility-related components and tests to eliminate device control and predictive input features.
- Adjusted AccessibilityPanel and ScreenIntelligencePanel to reflect the removal of these features.
- Modified related tests to ensure consistency with the updated accessibility status structure.
- Cleaned up accessibility session parameters and state management to focus solely on screen monitoring.
* refactor(accessibility): streamline featureOverrides state initialization
- Simplified the initialization of featureOverrides state in AccessibilityPanel and ScreenIntelligencePanel components for better readability.
- Consolidated parameter definitions in startAccessibilitySession to enhance clarity and maintainability.
- Removed unnecessary re-exports in the screen_intelligence engine module to clean up the codebase.
* chore(dependencies): update OpenHuman to version 0.51.19 in Cargo.lock
* chore(dependencies): update OpenHuman version to 0.51.19 in Cargo.lock
* feat(restart): implement core process restart functionality
- Added a new `SystemRestartRequested` event to the `DomainEvent` enum to handle restart requests.
- Introduced a `RestartSubscriber` that listens for restart events and manages the process respawn.
- Created a `service_restart` function to publish restart requests via the event bus.
- Updated service schemas to include a new `restart` controller with parameters for source and reason.
- Enhanced documentation to reflect changes in behavior and added necessary code comments.
* feat(accessibility): add last restart summary to Screen Intelligence Panel
- Introduced `lastRestartSummary` to the accessibility state and updated relevant components to display the last successful core restart information.
- Modified `PermissionsSection` and `ScreenIntelligencePanel` to include the new summary.
- Updated tests to validate the display of the last restart summary and ensure proper state management during core restarts.
- Refactored accessibility slice to handle the new restart summary in state updates.
* feat(core): enhance startup process with restart delay and subscriber registration
- Added a call to apply startup restart delay from environment variables in `run_core_from_args`.
- Updated the `bootstrap_skill_runtime` function to register a `RestartSubscriber` for handling restart requests, ensuring consistent respawn logic across triggers.
- Introduced a new `core_process` field in the `AccessibilityEngine` to track the core process status, including its PID and start time.
- Implemented a helper function to capture the core process start time using `OnceLock` for efficient initialization.
* feat(screen-intelligence): refactor accessibility state management and UI components
- Replaced direct Redux state access with a new `useScreenIntelligenceState` hook across multiple components, including `AccessibilityPanel`, `ScreenIntelligencePanel`, and their respective subcomponents.
- Streamlined permission and session handling by consolidating related functions and removing unnecessary dispatch calls.
- Updated tests to mock the new state management approach, ensuring consistent behavior and validation of UI elements.
- Removed the `SessionAndVisionSection` component to simplify the structure and improve maintainability.
- Introduced a new API file for screen intelligence to encapsulate related functionality and improve code organization.
* refactor(tests): clean up and optimize test files for accessibility and screen intelligence panels
- Removed redundant imports and streamlined the structure of test files for `AccessibilityPanel` and `ScreenIntelligencePanel`.
- Consolidated core process state initialization in test mocks for better readability.
- Updated dependency imports and ensured consistent mocking of state management hooks across tests.
- Enhanced the `ScreenPermissionsStep` component by improving the dependency array in the useEffect hook for better performance.
* refactor(store): remove unused authentication and user management code
- Deleted the `UserProvider`, `authSlice`, `authSelectors`, `userSlice`, `teamSlice`, and related test files to streamline the codebase.
- This cleanup enhances maintainability by removing legacy code that is no longer in use.
- Updated the store configuration to reflect the removal of these slices and ensure proper state management.
* refactor(webhooks): reorganize types and remove legacy state management
- Moved `TunnelRegistration` and `WebhookActivityEntry` types to a new `types.ts` file for better organization.
- Updated imports in `TunnelList` and `WebhookActivity` components to reference the new types location.
- Refactored `useWebhooks` hook to eliminate Redux state management in favor of local state, enhancing performance and reducing complexity.
- Removed unused `aiSlice`, `inviteSlice`, and `webhooksSlice` along with their associated tests to streamline the codebase.
* refactor(daemon): migrate state management from Redux to a custom store
- Introduced a new `store.ts` file to manage daemon state, replacing the previous Redux slice.
- Updated components and hooks to utilize the new state management approach, enhancing performance and reducing complexity.
- Removed the legacy `daemonSlice` and associated Redux logic, streamlining the codebase.
- Adjusted imports in various components and hooks to reference the new store structure.
* refactor(screen-intelligence): integrate core state management and enhance status handling
- Replaced direct state management in `useScreenIntelligenceState` with a new core state approach, utilizing `useCoreState` for improved performance and consistency.
- Updated status fetching and permission handling to leverage the core state snapshot, streamlining the logic and reducing redundant API calls.
- Introduced a new `CoreRuntimeSnapshot` interface to encapsulate runtime statuses, including screen intelligence, local AI, autocomplete, and service states.
- Adjusted related components and hooks to align with the new state management structure, enhancing maintainability and readability.
- Updated tests to validate the new runtime state structure and ensure proper functionality across the application.
* refactor(components): reorganize imports and streamline function formatting
- Moved the import of `Tunnel` and `tunnelsApi` in `TunnelList.tsx` for better organization.
- Reformatted function definitions in `store.ts`, `useDaemonHealth.ts`, `useDaemonLifecycle.ts`, `useWebhooks.ts` for improved readability.
- Cleaned up the structure of test files in `coreRpcClient.test.ts` by consolidating object properties for clarity.
- These changes enhance code maintainability and readability across the application.
* test(screen-intelligence): fix duplicate hook imports
* fix(tests): update ScreenIntelligenceDebugPanel test to use baseState for refresh status and vision calls
* refactor(invites): simplify error message rendering in Invites component
- Consolidated the conditional rendering of the load error message in the Invites component for improved readability.
- This change enhances the clarity of the code without altering functionality.
* refactor(daemon): streamline state management and function definitions
- Removed the `healthTimeoutId` from the `DaemonUserState` interface and related functions to simplify state management.
- Converted several functions in `store.ts` to arrow function syntax for consistency and improved readability.
- Updated the `Invites` component to handle asynchronous loading and error states more effectively, ensuring that in-flight requests are properly managed.
- Refactored the `CoreStateProvider` to enhance the refresh logic and prevent multiple simultaneous refreshes.
- Introduced a new `register_domain_subscribers` function in `jsonrpc.rs` to centralize event bus subscriber registration, improving code organization and maintainability.
* fix: add debug logging, atomic restart guard, and idempotent subscriber registration
- CoreStateProvider: add namespaced debug logger for polling failure diagnostics
- service/bus.rs: add AtomicBool gate to prevent duplicate restart spawns
- service/bus.rs: use OnceLock for idempotent RestartSubscriber registration
- Invites.tsx: add debug log in loadInviteCodes catch block
* style: apply prettier formatting to CoreStateProvider
* fix: sanitize error logging, serialize refresh, and demote restart logs
- CoreStateProvider: sanitize error objects in poll failure logs to avoid
leaking tokens/headers
- CoreStateProvider: move in-flight guard into refresh() via shared promise
so all callers (poll, updateLocalState, storeSessionToken) are serialized
- CoreStateProvider: log refreshTeams errors instead of swallowing them
- service/bus.rs: demote duplicate-restart log to debug, omit reason from
log output to avoid free-form text emission
* style: apply cargo fmt to service/bus.rs
* fix(auth): update RPC method names for authentication calls
Refactor authentication-related RPC method names to use underscores instead of dots for consistency. Updated methods include `get_state`, `get_session_token`, `clear_session`, and `store_session`.
chore: update OpenHuman version to 0.51.19
style: standardize string formatting in quickjs_libs/bootstrap.js and other files
- Replace single quotes with double quotes for string literals in various functions.
- Ensure consistent formatting across console logging and error handling.
fix(config): improve token retrieval logic in ops_core.rs
- Enhance the logic for retrieving the active session token from the credentials store, accommodating user-specific directories.
* test: align auth and OAuth assertions with current behavior
Update stale test expectations for underscore-style auth RPC methods and light-theme OAuth button classes, and make the bypass-login E2E assertion resilient to the current auth persistence model.
Made-with: Cursor
* chore: apply formatter output for login flow spec
Include Prettier formatting adjustments produced by the pre-push hook so the branch can pass repository push checks cleanly.
Made-with: Cursor
* feat(subconscious): stabilize heartbeat + subconscious loop (#392)
- Enable heartbeat by default (enabled=true, inference_enabled=true, 5min interval)
- Seed system tasks on engine init, not first tick
- SQLite-backed task/log/escalation persistence
- Overlap guard with generation counter — stale ticks are cancelled
- Single log entry per task per tick, updated in place (in_progress → act/noop/escalate/failed/cancelled)
- Rate-limit retry (429 only) for agentic-v1 cloud model calls
- Approval gate: unsolicited write actions on read-only tasks require user approval
- Analysis-only mode for agentic-v1 on read-only escalations
- Non-blocking status RPC — reads from DB, never blocks on engine mutex
- Frontend: system vs user task distinction, toggle switches, expandable activity log
- Frontend: 3s auto-poll on Subconscious tab, skill-related escalation navigation
- Consecutive failure counter in status (resets on success)
- last_tick_at only advances on successful evaluation
- Missing LLM evaluation fallback — unevaluated tasks default to noop
- Docs: subconscious.md architecture guide, memory-sync-functions.md reference
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* style: fix Prettier formatting for subconscious frontend files
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* ci: retrigger checks
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(heartbeat): use disabled config in run_returns_immediately_when_disabled test
HeartbeatConfig::default() has enabled: true, so run() entered the infinite
loop and never returned — hanging the test (and CI) indefinitely.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(subconscious): remove HEARTBEAT.md task import, use SQLite as sole task source
Tasks are now managed exclusively in SQLite via the Subconscious UI.
HEARTBEAT.md is retained for instructions/context only, not as a task list.
Situation report now reads pending tasks from SQLite instead of HEARTBEAT.md.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* style: cargo fmt on subconscious engine
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(settings): add 'Use Vision Model' option to Screen Intelligence Panel
- Introduced a new checkbox in the Screen Intelligence Panel to toggle the use of a vision model for richer context extraction from screenshots.
- Updated state management to handle the new option and integrated it into the configuration and processing logic.
- Adjusted related tests and configurations to support the new feature, ensuring compatibility across the application.
* feat(cli): add --no-vision-model option for screen intelligence
- Introduced a new command-line option `--no-vision-model` to allow users to skip the vision model and use OCR and text LLM only.
- Updated the CLI options parsing to handle the new flag and modified the bootstrap logic to respect this setting.
- Enhanced usage documentation to reflect the new option and its alias `--ocr-only` for clarity.
* fix(screen-intelligence): read use_vision_model from engine runtime config
The processing worker was reading use_vision_model from the persisted
config file (Config::load_or_init), so the CLI --no-vision-model flag
had no effect. Now reads from the engine's in-memory runtime config
which the CLI correctly overrides via apply_config(). Also moves image
compression before OCR pass.
* fix: add use_vision_model to test fixtures and fix rustfmt
Add the new use_vision_model field to all AccessibilityConfig test
fixtures so TypeScript compilation passes. Also includes rustfmt
auto-fix for screen_intelligence_cli.rs.
* fix(voice): add per-segment confidence validation in whisper engine (#385)
Reject whisper segments with avg token log-probability below -0.7 or
entropy above 2.4. Return TranscriptionResult with confidence metadata
instead of plain String. Update callers in speech.rs and streaming.rs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(voice): upgrade default STT model from tiny to base (#385)
Base model produces significantly fewer hallucinations than tiny,
especially in noisy/quiet conditions. User can still override via config.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(voice): add real-time silence gating in audio capture (#385)
Gate sustained silence (>500ms) from being sent to whisper to prevent
hallucinations. Maintain 100ms look-ahead ring buffer so speech onset
after pauses is not clipped. Thresholds adapt to source sample rate.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(voice): fix Fn key timing race condition in hotkey event loop (#385)
start_recording() blocks 1-7s on cpal device init but macOS fires Fn
Release almost immediately, causing skipped cycles. Move recording
start to spawn_blocking so the event loop stays responsive. Buffer
Release events during setup and ensure minimum 1.5s recording duration
when release arrives before recording handle is ready.
Also includes: capture focused app on hotkey press, pass through
pipeline for focus validation before paste.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(voice): validate and restore focus before paste, attempt regardless (#385)
Add expected_app parameter to insert_text(). Before Cmd+V, validate
focus via accessibility API and restore via AppleScript if shifted.
Don't abort paste on focus validation failure — attempt insertion
regardless so text is never silently lost.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(voice-ui): align voice server RPC response shape in settings panel (#385)
* style(voice): apply rustfmt formatting in text_input
* fix(voice): address CodeRabbit regressions in server, streaming, and settings polling
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat(autocomplete): add overlay TTL configuration to AutocompletePanel
- Introduced `overlay_ttl_ms` parameter to the Autocomplete configuration, allowing users to set the overlay display duration.
- Updated AutocompletePanel to include a new input field for adjusting the overlay TTL in milliseconds.
- Enhanced parsing and saving logic to handle the new configuration parameter.
- Added corresponding tests to ensure functionality and validate the new overlay TTL feature.
This update improves user control over the autocomplete overlay behavior, enhancing the overall user experience.
* Refactor accessibility code for improved readability and consistency
- Simplified log statements in `precompile_helper_background` for better clarity.
- Reformatted `detect_input_monitoring_permission` check in `keys.rs` for enhanced readability.
- Rearranged imports in `mod.rs` to maintain consistent structure.
- Improved formatting of `ElementBounds` initialization across multiple test cases in `overlay.rs` and `types.rs` for better visual alignment.
- Enhanced test context creation in `types.rs` for improved clarity.
These changes enhance code maintainability and readability across the accessibility module.
* fix(overlay): parent core RPC, voice toggle, and debug for #342
- Pass OPENHUMAN_OVERLAY_PARENT_RPC_URL from sidecar spawn and strip inherited
OPENHUMAN_CORE_PORT so the overlay no longer fights for the parent listen port.
- Overlay UI uses HTTP JSON-RPC to the parent sidecar for globe, debug, and voice
STT so state matches the main app; add parentCoreRpc helper mirroring legacy
method aliases.
- Skip embedded JSON-RPC server when parent URL is set; use
OPENHUMAN_OVERLAY_EMBEDDED_CORE_PORT (default 7799) for standalone dev.
- Fix screen intelligence status method name; add voice_status polling, STT
section, collapsible debug summary, and connection banner when core is unreachable.
- Voice capture switch gates the mic with real STT availability from voice_status.
Closes#342
Made-with: Cursor
* fix: address CodeRabbit review on overlay/autocomplete (PR #378)
- helper: correlate JSON-RPC replies with monotonic request ids; discard mismatched
lines until deadline (fixes stale response after timeout).
- helper: serialize Swift compile via HELPER_COMPILE_LOCK (precompile vs first use).
- Overlay Tab hint: pass tab_hint from Rust from accept_with_tab; Swift hides hint when empty.
- keys: re-check Input Monitoring on an interval when denied so grant without restart works.
- engine: use non-zero confidence placeholder (0.75) until inline_complete returns scores.
- overlay dedupe: suppress identical badge only within 400ms, not for process lifetime.
Co-authored-by: Code review feedback <noreply@github.com>
Made-with: Cursor
* fix(ci): resolve clippy and warning issues for Rust gates
- Use match on anchor_bounds in autocomplete overlay (avoid unnecessary unwrap)
- Drop unused test imports in registry_ops and rpc dispatch
- Prefix unused notion_doc_id in subconscious integration test
Made-with: Cursor
* fix(overlay): improve parent RPC URL handling in App component
- Updated the useEffect hook to manage the parent RPC URL more robustly by introducing a mounted flag to prevent state updates on unmounted components.
- Added error handling to set the parent RPC URL to null in case of invocation failure, enhancing the reliability of the component's behavior.
* feat(overlay): implement timeout handling for parent core RPC requests
- Introduced a default timeout for parent core RPC requests, enhancing reliability by preventing indefinite waiting for responses.
- Added an AbortController to manage request timeouts, throwing a specific error message when a timeout occurs.
- Updated the `callParentCoreRpc` function to accept a customizable timeout parameter, improving flexibility for RPC calls.
* fix(overlay): allow stopping active recording regardless of config state
- Updated the main button handler in the App component to always permit stopping an active recording when the status is "listening", improving user experience and control over the recording process.
- Removed redundant code that previously checked the status before stopping the recording, streamlining the logic.
* feat(overlay): update Cargo.lock with new dependencies and versions
- Added new packages including `alsa`, `alsa-sys`, `arboard`, `block`, `cocoa`, `core-foundation`, `core-graphics`, `coreaudio-rs`, `coreaudio-sys`, `cpal`, `crunchy`, and `dasp_sample` to enhance functionality and support for audio processing and system interactions.
- Updated existing dependencies to their latest versions for improved performance and compatibility.
- Modified the `show_overlay` function in `ops.rs` to include an additional parameter, enhancing the overlay display functionality.
* feat(autocomplete): add overlay_ttl_ms parameter to Autocomplete interfaces
- Introduced a new optional parameter `overlay_ttl_ms` to both `AutocompleteSetStyleParams` and `AutocompleteConfig` interfaces, allowing for customizable overlay timeout settings.
- This enhancement improves the flexibility of the autocomplete feature by enabling developers to specify how long the overlay should remain visible.
---------
Co-authored-by: Code review feedback <noreply@github.com>
Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai>
* fix(dictation): update hotkey default value and documentation
- Changed the default global hotkey for dictation from "CmdOrCtrl+Shift+D" to "Fn" in both the configuration schema and the associated documentation.
- Updated the hotkey parsing function to recognize "Fn" as a valid key, enhancing the flexibility of hotkey configurations.
- Added a test case to ensure the "Fn" key can be parsed correctly, improving the robustness of the hotkey handling functionality.
* fix(voice): update default activation mode and hotkey in configuration
- Changed the default activation mode for voice and dictation from "tap" to "push" in the respective configuration schemas.
- Updated the default hotkey for voice commands from "ctrl+shift+space" to "Fn" across various modules and documentation.
- Adjusted related tests to reflect the new defaults, ensuring consistency in behavior and expectations.
* feat(voice): integrate embedded global voice server startup
- Added a new asynchronous function `start_if_enabled` to the voice server module, which initializes the embedded voice server based on configuration settings.
- Updated the server run logic to check if the voice server should auto-start, enhancing the startup process for the core application.
- Integrated the new server startup function into the main server run logic, ensuring the voice server is launched if enabled in the configuration.
* feat(voice): add VoicePanel for managing voice server settings
- Introduced a new `VoicePanel` component to handle voice server configurations, including startup options, hotkeys, and runtime controls.
- Updated routing in the settings page to include the new voice settings section.
- Enhanced the `useSettingsNavigation` hook to support navigation to the voice settings.
- Added tests for the `VoicePanel` to ensure functionality and reliability of the voice server management features.
* refactor(dictation): update documentation and improve component initialization
- Revised comments in `DictationHotkeyManager` to clarify the component's mounting process within the app tree.
- Removed unused imports and unnecessary state management from `ServiceBlockingGate`, streamlining the component's logic.
- Updated tests for `ServiceBlockingGate` to reflect changes in behavior, ensuring accurate rendering of child components based on service status.
- Enhanced the `Cargo.lock` file by updating dependencies to their latest versions for improved stability and security.
* fix(voice): update default skip_cleanup setting and enhance VoicePanel options
- Changed the default value of `skip_cleanup` in the voice server configuration from `false` to `true` to improve transcription handling.
- Reordered options in the `VoicePanel` component to ensure "Natural cleanup" is displayed alongside "Verbatim transcription" for better user clarity.
- Updated tests to reflect the new default settings and ensure proper functionality of the VoicePanel component.
* feat(window): add window management commands for Tauri application
- Introduced a new module `window.ts` containing functions for managing window visibility and state in a Tauri application.
- Implemented commands to show, hide, toggle visibility, minimize, maximize, close, and set the title of the main window.
- Added checks to ensure commands are only executed in a Tauri environment, enhancing compatibility with web contexts.
* feat(tauriCommands): add comprehensive Tauri command modules
- Introduced multiple new modules for Tauri commands, including `accessibility`, `autocomplete`, `config`, `conscious`, `core`, `cron`, `hardware`, `localAi`, and `window`.
- Each module contains functions for managing specific functionalities such as accessibility permissions, autocomplete suggestions, configuration settings, and hardware interactions.
- Implemented checks to ensure commands are executed only in a Tauri environment, enhancing compatibility and reliability.
- This addition significantly expands the command capabilities of the Tauri application, providing a robust framework for future development.
* feat(voice): enhance audio transcription with initial prompt support
- Added functionality to transcribe audio with an optional initial prompt, allowing for vocabulary bias and improved conversational continuity.
- Updated the `transcribe_pcm_f32` and `transcribe_wav_file` functions to accept an `initial_prompt` parameter, enhancing recognition of specific vocabulary.
- Implemented peak RMS energy tracking during audio recording for silence detection, ensuring recordings below a defined threshold are skipped.
- Enhanced the voice server configuration to include a silence threshold and custom dictionary for better transcription context.
- Introduced methods to build and manage recent transcripts for improved continuity across consecutive recordings.
- Updated tests to validate new features and ensure proper functionality of the transcription process.
* feat(voice): enhance VoiceServerConfig with silence detection and custom dictionary
- Added `silence_threshold` to the `VoiceServerConfig` for improved silence detection, allowing recordings with low RMS energy to be skipped.
- Introduced `custom_dictionary` to bias transcription towards specific vocabulary, enhancing recognition of names and technical terms.
- Updated the `voice_transcribe` and `voice_transcribe_bytes` functions to utilize the new `initial_prompt` parameter for better context during transcription.
- Adjusted the `transcribe_pcm_i16` function to accept additional parameters for improved flexibility in handling audio input.
* feat(voice): add silence threshold and custom dictionary features to VoicePanel
- Implemented a new input for setting the silence threshold, allowing recordings with low RMS energy to be skipped.
- Added functionality for a custom dictionary, enabling users to add specific vocabulary words to improve transcription accuracy.
- Updated the VoiceServerSettings interface and related functions to support the new features, ensuring seamless integration with existing settings.
- Enhanced the UI in the VoicePanel to facilitate user interaction with the new settings.
* feat(voice): propagate silence threshold and custom dictionary to voice server command
- Added `silence_threshold` and `custom_dictionary` parameters to the `run_voice_server_command` function, ensuring these settings are utilized during voice server operations.
- Enhanced integration with the existing voice server configuration to support improved transcription accuracy and silence detection.
* fix(tauriCommands): update import paths for coreRpcClient
- Adjusted import paths for `callCoreRpc` in multiple Tauri command modules to ensure correct referencing from the updated directory structure.
- This change enhances module organization and maintains consistency across the codebase.
* feat(dependencies): update Cargo.lock and Cargo.toml for new packages and versions
- Added new dependencies including `arboard`, `fax`, `fax_derive`, `gethostname`, `half`, `quick-error`, `tiff`, and `x11rb` to enhance functionality and support for clipboard operations, transcription improvements, and system interactions.
- Updated existing dependencies to their latest versions for better performance and compatibility.
- Modified `VoicePanel` to utilize the updated settings and ensure proper handling of voice server configurations.
- Enhanced the text input mechanism to use clipboard-paste for improved reliability in text insertion.
* fix(voice): update skip_cleanup default value and enhance logging
- Changed the default value of `skip_cleanup` in `VoiceServerConfig` from `true` to `false` to align with expected behavior and improve transcription handling.
- Added detailed logging in the transcription cleanup process to provide better insights into the LLM state and cleanup decisions.
- Removed unused functions related to unreliable key releases in hotkey handling to simplify the codebase.
- Updated tests to reflect the new default settings for `skip_cleanup` and ensure proper functionality across components.
* style: apply linter formatting fixes
* fix(voice): remove unused warn import in hotkey module
* test(voice): add silence threshold and custom dictionary to VoicePanel tests
- Updated tests for the VoicePanel component to include new parameters: `silence_threshold` and `custom_dictionary`.
- Ensured that the tests reflect the latest configuration settings for improved transcription accuracy and functionality.
