777c98ef7c
* refactor(tauriCommands): streamline RPC method calls for skill management - Simplified the syntax of RPC method calls in the `tauriCommands.ts` file by removing unnecessary line breaks, enhancing code readability. - Updated descriptions in the `schemas.rs` file to maintain consistent formatting for RPC method descriptions, improving documentation clarity. * chore(release): bump OpenHuman version to 0.49.23 in Cargo.lock * feat(skills): add skill status management in desktopDeepLinkListener - Introduced a new action `setSkillStatus` to manage the skill's connection status. - Updated the OAuth deep link handling to dispatch the skill status as 'ready' upon successful connection, improving state management for skills. * feat(screen_intelligence): implement capture mode for screenshots - Added a `CaptureMode` enum to differentiate between windowed and fullscreen screenshot captures. - Enhanced the `capture_screen_image_ref_for_context` function to determine capture mode based on window bounds, with fallback to fullscreen. - Implemented logic to downscale screenshots exceeding size limits when captured in fullscreen mode. - Introduced a new `parse_foreground_output` function to parse application context from AppleScript output, improving context retrieval for screenshots. * chore(build): update Tauri configuration to remove updater artifacts creation - Modified the TAURI_CONFIG_OVERRIDE to exclude the creation of updater artifacts during the build process, streamlining the build workflow. * feat(accessibility): enhance accessibility state and add capture test functionality - Introduced new properties `captureTestResult` and `isCaptureTestRunning` to the `AccessibilityState` interface for managing capture test states. - Added `CaptureTestResult` and `CaptureTestContextInfo` interfaces to define the structure of capture test results. - Implemented `openhumanScreenIntelligenceCaptureTest` function to initiate screen intelligence capture tests, improving accessibility features. * feat(debug): add Screen Intelligence Debug Panel for enhanced diagnostics - Introduced a new `ScreenIntelligenceDebugPanel` component to display accessibility status, session information, and capture test results. - Integrated the debug panel into the existing `ScreenIntelligencePanel`, allowing users to expand and collapse the debug section. - Updated accessibility state management to include `captureTestResult` and `isCaptureTestRunning` for improved testing feedback. - Enhanced test setup to accommodate new debug functionalities. * feat(screen_intelligence): implement custom hook for screen intelligence items - Added `useScreenIntelligenceItems` hook to fetch and manage screen intelligence items from the accessibility state. - Integrated the hook into the `Intelligence` component, combining items from both memory and screen intelligence sources. - Enhanced loading state management to reflect the combined loading status of both data sources. * test(screen_intelligence): add unit tests for useScreenIntelligenceItems mapping and confidenceToPriority functions - Introduced tests for the mapping logic of AccessibilityVisionSummary to ActionableItem, ensuring correct transformation of properties. - Added tests for confidenceToPriority function to validate priority assignment based on confidence levels. - Included edge cases such as handling empty arrays, null app names, and long actionable notes truncation. * feat(mnemonic): update mnemonic handling to support variable word counts - Refactored mnemonic import logic to accommodate BIP39 phrase lengths (12, 15, 18, 21, 24 words). - Updated state initialization and validation to dynamically adjust based on the allowed word counts. - Enhanced user prompts to reflect the new word count options for recovery phrases. - Adjusted focus handling for input fields to improve user experience during phrase entry. * refactor(skills): migrate skill state management to RPC-based hooks - Replaced Redux-based skill state management with RPC-backed hooks for improved performance and reactivity. - Introduced `useSkillSnapshot` and `useAllSkillSnapshots` hooks to fetch skill states directly from the Rust core. - Updated components to utilize the new hooks, enhancing the overall architecture and reducing dependency on Redux. - Added event listeners to trigger state updates in response to skill state changes, ensuring real-time updates across the application. * fix(lint): merge duplicate tauriCommands import in accessibilitySlice test Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(tests): remove eslint-disable comment from screen intelligence E2E test file --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2.6 KiB
todo
-
allow skills to be downloaded from the web
-
allow skills to be written as text formatted files like SKILL.md
-
skills need to specific via JSON-rpc the state changes they make to their state and data files in memory
-
skills need to be able to download custom mcp servers from the web
-
integrate the payments flow properly, skip the connect account page and goto the home page
[] - allow for new skills to be coded on their own [] - allow for multiple instances of a skill to be loaded [] - add a local model that can read through the screen and also go through voice using an API like whisper [] - add a screener recorder that goes through the intefaces in the screen and locally summarizes what is happening and brings more assitance to the user [] clean up the core so that we can run it as a binary on a server or as docker
[x] Separate the binary from the tauri codebase [] Integrate our custom memory engine into core - sanil [] Integrate our skills registry into core - steve [x] Integrate accessibility service installation [] Add as a step and setting in the UI - cyrus [x] Remove mentions of zeroclaw from the codebaes [x] Integrate local LLM into core [x] Handle process/deamon properly [x] install the linux philosophy of few modules that do their own thing really well sort of.. [x] Remove android / ios support from the codebase. [x] e2e test to check if daemon and sidecar loading works properly [x] Find a better way to structure the cargo files [x] fix all the rust and cargo issues [] Add icon and app name to the various permission settings - mithil [] add self update based on github release. create a update action on the cli - aniketh [] for each skill show information on how much data has been synced locally and information on how much syncs have happened so far etc.. - mithil/elvin [x] redo the docs once everything is done. [x] remove unwanted feature flags from the rust binary [] fix the config properly - mithil [] Allow for Migrating from OpenClaw - steve done - to be tested [] allow users to choose which version of LLM model they'd like to choose based on their CPU. better ram and gpu means higher parameter model can be used. - mithil [x] in the client side app, make console.log follow a logger style logging where there's a namespace for every logger (like python) - steve [ ] - currently we bundle tauri in the openhumany rust core but that shouldn't really have to be there. it can be completely removed. [] allow skills to be debuggged from the UI (we shuold try to call various tools or see state from the UI itself)
--- e2e tests to write up
- connecting a channel like telegram/discord works properly