Refactor core-polled app state and screen intelligence status (#464)

* refactor(accessibility): remove device control and predictive input features from accessibility settings

- Updated accessibility-related components and tests to eliminate device control and predictive input features.
- Adjusted AccessibilityPanel and ScreenIntelligencePanel to reflect the removal of these features.
- Modified related tests to ensure consistency with the updated accessibility status structure.
- Cleaned up accessibility session parameters and state management to focus solely on screen monitoring.

* refactor(accessibility): streamline featureOverrides state initialization

- Simplified the initialization of featureOverrides state in AccessibilityPanel and ScreenIntelligencePanel components for better readability.
- Consolidated parameter definitions in startAccessibilitySession to enhance clarity and maintainability.
- Removed unnecessary re-exports in the screen_intelligence engine module to clean up the codebase.

* chore(dependencies): update OpenHuman to version 0.51.19 in Cargo.lock

* chore(dependencies): update OpenHuman version to 0.51.19 in Cargo.lock

* feat(restart): implement core process restart functionality

- Added a new `SystemRestartRequested` event to the `DomainEvent` enum to handle restart requests.
- Introduced a `RestartSubscriber` that listens for restart events and manages the process respawn.
- Created a `service_restart` function to publish restart requests via the event bus.
- Updated service schemas to include a new `restart` controller with parameters for source and reason.
- Enhanced documentation to reflect changes in behavior and added necessary code comments.

* feat(accessibility): add last restart summary to Screen Intelligence Panel

- Introduced `lastRestartSummary` to the accessibility state and updated relevant components to display the last successful core restart information.
- Modified `PermissionsSection` and `ScreenIntelligencePanel` to include the new summary.
- Updated tests to validate the display of the last restart summary and ensure proper state management during core restarts.
- Refactored accessibility slice to handle the new restart summary in state updates.

* feat(core): enhance startup process with restart delay and subscriber registration

- Added a call to apply startup restart delay from environment variables in `run_core_from_args`.
- Updated the `bootstrap_skill_runtime` function to register a `RestartSubscriber` for handling restart requests, ensuring consistent respawn logic across triggers.
- Introduced a new `core_process` field in the `AccessibilityEngine` to track the core process status, including its PID and start time.
- Implemented a helper function to capture the core process start time using `OnceLock` for efficient initialization.

* feat(screen-intelligence): refactor accessibility state management and UI components

- Replaced direct Redux state access with a new `useScreenIntelligenceState` hook across multiple components, including `AccessibilityPanel`, `ScreenIntelligencePanel`, and their respective subcomponents.
- Streamlined permission and session handling by consolidating related functions and removing unnecessary dispatch calls.
- Updated tests to mock the new state management approach, ensuring consistent behavior and validation of UI elements.
- Removed the `SessionAndVisionSection` component to simplify the structure and improve maintainability.
- Introduced a new API file for screen intelligence to encapsulate related functionality and improve code organization.

* refactor(tests): clean up and optimize test files for accessibility and screen intelligence panels

- Removed redundant imports and streamlined the structure of test files for `AccessibilityPanel` and `ScreenIntelligencePanel`.
- Consolidated core process state initialization in test mocks for better readability.
- Updated dependency imports and ensured consistent mocking of state management hooks across tests.
- Enhanced the `ScreenPermissionsStep` component by improving the dependency array in the useEffect hook for better performance.

* refactor(store): remove unused authentication and user management code

- Deleted the `UserProvider`, `authSlice`, `authSelectors`, `userSlice`, `teamSlice`, and related test files to streamline the codebase.
- This cleanup enhances maintainability by removing legacy code that is no longer in use.
- Updated the store configuration to reflect the removal of these slices and ensure proper state management.

* refactor(webhooks): reorganize types and remove legacy state management

- Moved `TunnelRegistration` and `WebhookActivityEntry` types to a new `types.ts` file for better organization.
- Updated imports in `TunnelList` and `WebhookActivity` components to reference the new types location.
- Refactored `useWebhooks` hook to eliminate Redux state management in favor of local state, enhancing performance and reducing complexity.
- Removed unused `aiSlice`, `inviteSlice`, and `webhooksSlice` along with their associated tests to streamline the codebase.

* refactor(daemon): migrate state management from Redux to a custom store

- Introduced a new `store.ts` file to manage daemon state, replacing the previous Redux slice.
- Updated components and hooks to utilize the new state management approach, enhancing performance and reducing complexity.
- Removed the legacy `daemonSlice` and associated Redux logic, streamlining the codebase.
- Adjusted imports in various components and hooks to reference the new store structure.

* refactor(screen-intelligence): integrate core state management and enhance status handling

- Replaced direct state management in `useScreenIntelligenceState` with a new core state approach, utilizing `useCoreState` for improved performance and consistency.
- Updated status fetching and permission handling to leverage the core state snapshot, streamlining the logic and reducing redundant API calls.
- Introduced a new `CoreRuntimeSnapshot` interface to encapsulate runtime statuses, including screen intelligence, local AI, autocomplete, and service states.
- Adjusted related components and hooks to align with the new state management structure, enhancing maintainability and readability.
- Updated tests to validate the new runtime state structure and ensure proper functionality across the application.

* refactor(components): reorganize imports and streamline function formatting

- Moved the import of `Tunnel` and `tunnelsApi` in `TunnelList.tsx` for better organization.
- Reformatted function definitions in `store.ts`, `useDaemonHealth.ts`, `useDaemonLifecycle.ts`, `useWebhooks.ts` for improved readability.
- Cleaned up the structure of test files in `coreRpcClient.test.ts` by consolidating object properties for clarity.
- These changes enhance code maintainability and readability across the application.

* test(screen-intelligence): fix duplicate hook imports

* fix(tests): update ScreenIntelligenceDebugPanel test to use baseState for refresh status and vision calls

* refactor(invites): simplify error message rendering in Invites component

- Consolidated the conditional rendering of the load error message in the Invites component for improved readability.
- This change enhances the clarity of the code without altering functionality.

* refactor(daemon): streamline state management and function definitions

- Removed the `healthTimeoutId` from the `DaemonUserState` interface and related functions to simplify state management.
- Converted several functions in `store.ts` to arrow function syntax for consistency and improved readability.
- Updated the `Invites` component to handle asynchronous loading and error states more effectively, ensuring that in-flight requests are properly managed.
- Refactored the `CoreStateProvider` to enhance the refresh logic and prevent multiple simultaneous refreshes.
- Introduced a new `register_domain_subscribers` function in `jsonrpc.rs` to centralize event bus subscriber registration, improving code organization and maintainability.

* fix: add debug logging, atomic restart guard, and idempotent subscriber registration

- CoreStateProvider: add namespaced debug logger for polling failure diagnostics
- service/bus.rs: add AtomicBool gate to prevent duplicate restart spawns
- service/bus.rs: use OnceLock for idempotent RestartSubscriber registration
- Invites.tsx: add debug log in loadInviteCodes catch block

* style: apply prettier formatting to CoreStateProvider

* fix: sanitize error logging, serialize refresh, and demote restart logs

- CoreStateProvider: sanitize error objects in poll failure logs to avoid
  leaking tokens/headers
- CoreStateProvider: move in-flight guard into refresh() via shared promise
  so all callers (poll, updateLocalState, storeSessionToken) are serialized
- CoreStateProvider: log refreshTeams errors instead of swallowing them
- service/bus.rs: demote duplicate-restart log to debug, omit reason from
  log output to avoid free-form text emission

* style: apply cargo fmt to service/bus.rs
This commit is contained in:
Steven Enamakel
2026-04-09 15:51:27 -07:00
committed by GitHub
parent d66ee0d4de
commit acc6246e59
72 changed files with 2135 additions and 3670 deletions
+2
View File
@@ -265,6 +265,7 @@ Deep link plugin is registered where supported; behavior is platform-specific (s
- **`src/openhuman/` module layout**: **New** functionality must live in a **dedicated subdirectory** (its own folder/module, e.g. `openhuman/my_domain/mod.rs` plus related files, or a new subfolder under an existing domain). Do **not** add new standalone `*.rs` files directly at `src/openhuman/` root; place new code in a module directory and declare it from `mod.rs` (or merge into an existing domain folder).
- **Controller schema contract**: Shared controller metadata types live in **`src/core/mod.rs`** (`ControllerSchema`, `FieldSchema`, `TypeSchema`) and are consumed by adapters (RPC/CLI) in different ways.
- **Domain schema files**: For each domain, define controller schema metadata in a dedicated module inside the domain folder (example: **`src/openhuman/cron/schemas.rs`**) and export from the domain `mod.rs`.
- **Controller-only exposure rule**: Expose domain functionality to **CLI and JSON-RPC through the controller registry** (`schemas.rs` + registered handlers). Do **not** add domain-specific branches or one-off transport logic in `src/core/cli.rs` or `src/core/jsonrpc.rs` just to expose a feature.
- **Light `mod.rs` rule**: Keep domain `mod.rs` files light and export-focused. Put operational code in sibling files (example: `ops.rs`, `store.rs`, `schedule.rs`, `types.rs`), then re-export the public API from `mod.rs`.
- **`core_server/`** — Transport only: Axum/HTTP, JSON-RPC envelope, CLI parsing, **dispatch** (`core_server::dispatch`) — **no** heavy business logic here.
- **Layering**: Implementation in `openhuman::<domain>/`, controllers in `openhuman::<domain>/rpc.rs`, routes in `core_server/`.
@@ -384,6 +385,7 @@ In the parent **OpenHuman** desktop app, **Tauri / Rust is a delivery vehicle**:
- **Unix-style modules**: Prefer **individual modules** with a **single, sharp responsibility**—each should do one thing really well. Compose behavior through small, well-named units and clear boundaries instead of monolithic code.
- **Tests before the next layer**: Ship **enough unit tests and coverage** for the behavior you are adding or changing **before** building additional features on top of it. Treat untested code as incomplete; do not accumulate depth on a shaky base.
- **Documentation with code**: New or changed behavior must ship with matching documentation. At minimum, add concise rustdoc / code comments where the flow is not obvious, and update `AGENTS.md`, architecture docs, or feature docs when repository rules or user-visible behavior change.
---
Generated
+1 -1
View File
@@ -5120,7 +5120,7 @@ checksum = "c08d65885ee38876c4f86fa503fb49d7b507c2b62552df7c70b2fce627e06381"
[[package]]
name = "openhuman"
version = "0.49.17"
version = "0.51.19"
dependencies = [
"aes-gcm",
"anyhow",
@@ -6,8 +6,8 @@
*/
import type React from 'react';
import type { DaemonStatus } from '../../features/daemon/store';
import { formatRelativeTime, useDaemonHealth } from '../../hooks/useDaemonHealth';
import type { DaemonStatus } from '../../store/daemonSlice';
interface Props {
userId?: string;
@@ -15,8 +15,8 @@ import {
} from '@heroicons/react/24/outline';
import { useEffect, useState } from 'react';
import type { ComponentHealth, DaemonStatus } from '../../features/daemon/store';
import { formatRelativeTime, useDaemonHealth } from '../../hooks/useDaemonHealth';
import type { ComponentHealth, DaemonStatus } from '../../store/daemonSlice';
import { IS_DEV } from '../../utils/config';
import {
isTauri,
@@ -1,11 +1,9 @@
import { useCallback, useEffect } from 'react';
import { useCallback } from 'react';
import {
fetchAccessibilityStatus,
fetchAccessibilityVisionRecent,
runCaptureTest,
} from '../../store/accessibilitySlice';
import { useAppDispatch, useAppSelector } from '../../store/hooks';
type ScreenIntelligenceState,
useScreenIntelligenceState,
} from '../../features/screen-intelligence/useScreenIntelligenceState';
const formatBytes = (bytes: number | null | undefined): string => {
if (bytes == null) return '-';
@@ -14,24 +12,42 @@ const formatBytes = (bytes: number | null | undefined): string => {
return `${(bytes / (1024 * 1024)).toFixed(2)} MB`;
};
const ScreenIntelligenceDebugPanel = () => {
const dispatch = useAppDispatch();
const { status, captureTestResult, isCaptureTestRunning, recentVisionSummaries, lastError } =
useAppSelector(state => state.accessibility);
interface ScreenIntelligenceDebugPanelProps {
state?: Pick<
ScreenIntelligenceState,
| 'status'
| 'captureTestResult'
| 'isCaptureTestRunning'
| 'recentVisionSummaries'
| 'lastError'
| 'refreshStatus'
| 'refreshVision'
| 'runCaptureTest'
>;
}
useEffect(() => {
void dispatch(fetchAccessibilityStatus());
void dispatch(fetchAccessibilityVisionRecent(5));
}, [dispatch]);
const ScreenIntelligenceDebugPanelContent = ({
state: providedState,
}: Required<Pick<ScreenIntelligenceDebugPanelProps, 'state'>>) => {
const {
status,
captureTestResult,
isCaptureTestRunning,
recentVisionSummaries,
lastError,
refreshStatus,
refreshVision,
runCaptureTest,
} = providedState;
const handleCaptureTest = useCallback(() => {
void dispatch(runCaptureTest());
}, [dispatch]);
void runCaptureTest();
}, [runCaptureTest]);
const handleRefreshStatus = useCallback(() => {
void dispatch(fetchAccessibilityStatus());
void dispatch(fetchAccessibilityVisionRecent(5));
}, [dispatch]);
void refreshStatus();
void refreshVision(5);
}, [refreshStatus, refreshVision]);
const permissions = status?.permissions;
const session = status?.session;
@@ -199,6 +215,18 @@ const ScreenIntelligenceDebugPanel = () => {
);
};
const OwnedScreenIntelligenceDebugPanel = () => {
const state = useScreenIntelligenceState({ loadVision: true, visionLimit: 5, pollMs: 2000 });
return <ScreenIntelligenceDebugPanelContent state={state} />;
};
const ScreenIntelligenceDebugPanel = ({ state }: ScreenIntelligenceDebugPanelProps) => {
if (state) {
return <ScreenIntelligenceDebugPanelContent state={state} />;
}
return <OwnedScreenIntelligenceDebugPanel />;
};
const PermissionDot = ({ label, value }: { label: string; value?: string }) => {
const color =
value === 'granted' ? 'bg-green-500' : value === 'denied' ? 'bg-red-500' : 'bg-stone-600';
@@ -1,177 +1,112 @@
import { configureStore } from '@reduxjs/toolkit';
import { fireEvent, render, screen, waitFor } from '@testing-library/react';
import { Provider } from 'react-redux';
import { fireEvent, render, screen } from '@testing-library/react';
import { beforeEach, describe, expect, it, vi } from 'vitest';
import accessibilityReducer from '../../../store/accessibilitySlice';
import authReducer from '../../../store/authSlice';
import socketReducer from '../../../store/socketSlice';
import teamReducer from '../../../store/teamSlice';
import userReducer from '../../../store/userSlice';
import {
type AccessibilityStatus,
type AccessibilityVisionRecentResult,
type CaptureTestResult,
type CommandResponse,
openhumanAccessibilityInputAction,
openhumanAccessibilityRequestPermission,
openhumanAccessibilityRequestPermissions,
openhumanAccessibilityStartSession,
openhumanAccessibilityStatus,
openhumanAccessibilityStopSession,
openhumanAccessibilityVisionFlush,
openhumanAccessibilityVisionRecent,
openhumanScreenIntelligenceCaptureTest,
restartCoreProcess,
} from '../../../utils/tauriCommands';
type ScreenIntelligenceState,
useScreenIntelligenceState,
} from '../../../features/screen-intelligence/useScreenIntelligenceState';
import ScreenIntelligenceDebugPanel from '../ScreenIntelligenceDebugPanel';
vi.mock('../../../utils/tauriCommands', () => ({
isTauri: vi.fn(() => true),
openhumanAccessibilityInputAction: vi.fn(),
openhumanAccessibilityRequestPermission: vi.fn(),
openhumanAccessibilityRequestPermissions: vi.fn(),
openhumanAccessibilityStartSession: vi.fn(),
openhumanAccessibilityStatus: vi.fn(),
openhumanAccessibilityStopSession: vi.fn(),
openhumanAccessibilityVisionFlush: vi.fn(),
openhumanAccessibilityVisionRecent: vi.fn(),
openhumanScreenIntelligenceCaptureTest: vi.fn(),
restartCoreProcess: vi.fn(),
vi.mock('../../../features/screen-intelligence/useScreenIntelligenceState', () => ({
useScreenIntelligenceState: vi.fn(),
}));
const sampleStatus: AccessibilityStatus = {
platform_supported: true,
permissions: {
screen_recording: 'granted',
accessibility: 'granted',
input_monitoring: 'granted',
},
features: { screen_monitoring: true, device_control: true, predictive_input: true },
session: {
active: false,
started_at_ms: null,
expires_at_ms: null,
remaining_ms: null,
ttl_secs: 300,
panic_hotkey: 'Cmd+Shift+.',
stop_reason: null,
frames_in_memory: 0,
last_capture_at_ms: null,
last_context: null,
vision_enabled: true,
vision_state: 'idle',
vision_queue_depth: 0,
last_vision_at_ms: null,
last_vision_summary: null,
},
config: {
enabled: true,
capture_policy: 'hybrid',
policy_mode: 'all_except_blacklist',
baseline_fps: 1,
vision_enabled: true,
session_ttl_secs: 300,
panic_stop_hotkey: 'Cmd+Shift+.',
autocomplete_enabled: true,
use_vision_model: true,
keep_screenshots: false,
allowlist: [],
denylist: [],
},
denylist: [],
is_context_blocked: false,
};
const emptyVisionResponse: CommandResponse<AccessibilityVisionRecentResult> = {
result: { summaries: [] },
logs: [],
};
const createStore = () =>
configureStore({
reducer: {
auth: authReducer,
socket: socketReducer,
user: userReducer,
team: teamReducer,
accessibility: accessibilityReducer,
const baseState: ScreenIntelligenceState = {
status: {
platform_supported: true,
permissions: {
screen_recording: 'granted',
accessibility: 'granted',
input_monitoring: 'granted',
},
});
function renderPanel() {
const store = createStore();
render(
<Provider store={store}>
<ScreenIntelligenceDebugPanel />
</Provider>
);
return store;
}
features: { screen_monitoring: true },
session: {
active: false,
started_at_ms: null,
expires_at_ms: null,
remaining_ms: null,
ttl_secs: 300,
panic_hotkey: 'Cmd+Shift+.',
stop_reason: null,
frames_in_memory: 0,
last_capture_at_ms: null,
last_context: null,
vision_enabled: true,
vision_state: 'idle',
vision_queue_depth: 0,
last_vision_at_ms: null,
last_vision_summary: null,
},
config: {
enabled: true,
capture_policy: 'hybrid',
policy_mode: 'all_except_blacklist',
baseline_fps: 1,
vision_enabled: true,
session_ttl_secs: 300,
panic_stop_hotkey: 'Cmd+Shift+.',
autocomplete_enabled: true,
use_vision_model: true,
keep_screenshots: false,
allowlist: [],
denylist: [],
},
denylist: [],
is_context_blocked: false,
},
lastRestartSummary: null,
recentVisionSummaries: [],
captureTestResult: null,
isCaptureTestRunning: false,
isLoading: false,
isRequestingPermissions: false,
isRestartingCore: false,
isStartingSession: false,
isStoppingSession: false,
isLoadingVision: false,
isFlushingVision: false,
lastError: null,
refreshStatus: vi.fn().mockResolvedValue(null),
requestPermission: vi.fn().mockResolvedValue(null),
refreshPermissionsWithRestart: vi.fn().mockResolvedValue(null),
startSession: vi.fn().mockResolvedValue(null),
stopSession: vi.fn().mockResolvedValue(null),
refreshVision: vi.fn().mockResolvedValue([]),
flushVision: vi.fn().mockResolvedValue(undefined),
runCaptureTest: vi.fn().mockResolvedValue(undefined),
clearError: vi.fn(),
};
describe('ScreenIntelligenceDebugPanel', () => {
beforeEach(() => {
vi.mocked(openhumanAccessibilityStatus).mockResolvedValue({ result: sampleStatus, logs: [] });
vi.mocked(openhumanAccessibilityVisionRecent).mockResolvedValue(emptyVisionResponse);
vi.mocked(openhumanAccessibilityInputAction).mockResolvedValue({
result: {} as never,
logs: [],
});
vi.mocked(openhumanAccessibilityRequestPermission).mockResolvedValue({
result: sampleStatus.permissions,
logs: [],
} as never);
vi.mocked(openhumanAccessibilityRequestPermissions).mockResolvedValue({
result: sampleStatus.permissions,
logs: [],
} as never);
vi.mocked(openhumanAccessibilityStartSession).mockResolvedValue({
result: sampleStatus.session,
logs: [],
} as never);
vi.mocked(openhumanAccessibilityStopSession).mockResolvedValue({
result: sampleStatus.session,
logs: [],
} as never);
vi.mocked(openhumanAccessibilityVisionFlush).mockResolvedValue({
result: { accepted: true, summary: null },
logs: [],
} as never);
vi.mocked(restartCoreProcess).mockResolvedValue(undefined);
vi.clearAllMocks();
vi.mocked(useScreenIntelligenceState).mockReturnValue(baseState);
});
it('renders successful capture diagnostics and preview image', async () => {
const captureResult: CaptureTestResult = {
ok: true,
capture_mode: 'windowed',
context: {
app_name: 'Safari',
window_title: 'GitHub',
bounds_x: 10,
bounds_y: 20,
bounds_width: 1440,
bounds_height: 900,
const state: ScreenIntelligenceState = {
...baseState,
captureTestResult: {
ok: true,
capture_mode: 'windowed',
context: {
app_name: 'Safari',
window_title: 'GitHub',
bounds_x: 10,
bounds_y: 20,
bounds_width: 1440,
bounds_height: 900,
},
image_ref: 'data:image/png;base64,ZmFrZQ==',
bytes_estimate: 2048,
error: null,
timing_ms: 155,
},
image_ref: 'data:image/png;base64,ZmFrZQ==',
bytes_estimate: 2048,
error: null,
timing_ms: 155,
};
vi.mocked(openhumanScreenIntelligenceCaptureTest).mockResolvedValue({
result: captureResult,
logs: [],
});
renderPanel();
render(<ScreenIntelligenceDebugPanel state={state} />);
await waitFor(() => {
expect(openhumanAccessibilityStatus).toHaveBeenCalled();
expect(openhumanAccessibilityVisionRecent).toHaveBeenCalledWith(5);
});
fireEvent.click(screen.getByRole('button', { name: 'Test Capture' }));
expect(await screen.findByText('Success')).toBeInTheDocument();
expect(screen.getByText('Success')).toBeInTheDocument();
expect(screen.getByText('windowed')).toBeInTheDocument();
expect(screen.getByText('155ms')).toBeInTheDocument();
expect(screen.getByText('2.0 KB')).toBeInTheDocument();
@@ -183,8 +118,9 @@ describe('ScreenIntelligenceDebugPanel', () => {
});
it('renders capture failures without breaking the diagnostics panel', async () => {
vi.mocked(openhumanScreenIntelligenceCaptureTest).mockResolvedValue({
result: {
const state: ScreenIntelligenceState = {
...baseState,
captureTestResult: {
ok: false,
capture_mode: 'fullscreen',
context: null,
@@ -193,18 +129,19 @@ describe('ScreenIntelligenceDebugPanel', () => {
error: 'screen recording permission is not granted',
timing_ms: 42,
},
logs: [],
});
};
renderPanel();
render(<ScreenIntelligenceDebugPanel state={state} />);
fireEvent.click(screen.getByRole('button', { name: 'Test Capture' }));
fireEvent.click(screen.getByRole('button', { name: 'Refresh' }));
expect(await screen.findByText('Failed')).toBeInTheDocument();
expect(screen.getByText('Failed')).toBeInTheDocument();
expect(screen.getByText('fullscreen')).toBeInTheDocument();
expect(screen.getByText('42ms')).toBeInTheDocument();
expect(screen.getByText('screen recording permission is not granted')).toBeInTheDocument();
expect(screen.queryByAltText('Capture test result')).not.toBeInTheDocument();
expect(screen.getByText('Permissions')).toBeInTheDocument();
expect(baseState.refreshStatus).toHaveBeenCalledTimes(1);
expect(baseState.refreshVision).toHaveBeenCalledWith(5);
});
});
@@ -1,15 +1,6 @@
import { useEffect, useMemo, useState } from 'react';
import { useMemo, useState } from 'react';
import {
fetchAccessibilityStatus,
fetchAccessibilityVisionRecent,
flushAccessibilityVision,
refreshPermissionsWithRestart,
requestAccessibilityPermission,
startAccessibilitySession,
stopAccessibilitySession,
} from '../../../store/accessibilitySlice';
import { useAppDispatch, useAppSelector } from '../../../store/hooks';
import { useScreenIntelligenceState } from '../../../features/screen-intelligence/useScreenIntelligenceState';
import SettingsHeader from '../components/SettingsHeader';
import { useSettingsNavigation } from '../hooks/useSettingsNavigation';
@@ -46,7 +37,6 @@ const PermissionBadge = ({ label, value }: { label: string; value: string }) =>
const AccessibilityPanel = () => {
const { navigateBack, breadcrumbs } = useSettingsNavigation();
const dispatch = useAppDispatch();
const {
status,
isLoading,
@@ -58,31 +48,15 @@ const AccessibilityPanel = () => {
isFlushingVision,
recentVisionSummaries,
lastError,
} = useAppSelector(state => state.accessibility);
const [featureOverrides, setFeatureOverrides] = useState<{
screen_monitoring?: boolean;
device_control?: boolean;
predictive_input?: boolean;
}>({});
useEffect(() => {
void dispatch(fetchAccessibilityStatus());
}, [dispatch]);
useEffect(() => {
if (!status?.session.active) {
return;
}
const intervalId = window.setInterval(() => {
void dispatch(fetchAccessibilityStatus());
void dispatch(fetchAccessibilityVisionRecent(10));
}, 1000);
return () => window.clearInterval(intervalId);
}, [dispatch, status?.session.active]);
useEffect(() => {
void dispatch(fetchAccessibilityVisionRecent(10));
}, [dispatch]);
requestPermission,
refreshPermissionsWithRestart,
refreshStatus,
refreshVision,
startSession,
stopSession,
flushVision,
} = useScreenIntelligenceState({ loadVision: true, visionLimit: 10, pollMs: 2000 });
const [featureOverrides, setFeatureOverrides] = useState<{ screen_monitoring?: boolean }>({});
const anyPermissionDenied =
status?.permissions.accessibility === 'denied' ||
@@ -90,9 +64,6 @@ const AccessibilityPanel = () => {
const screenMonitoring =
featureOverrides.screen_monitoring ?? status?.features.screen_monitoring ?? true;
const deviceControl = featureOverrides.device_control ?? status?.features.device_control ?? true;
const predictiveInput =
featureOverrides.predictive_input ?? status?.features.predictive_input ?? true;
const remaining = useMemo(
() => formatRemaining(status?.session.remaining_ms ?? null),
@@ -152,14 +123,14 @@ const AccessibilityPanel = () => {
<button
type="button"
onClick={() => void dispatch(requestAccessibilityPermission('accessibility'))}
onClick={() => void requestPermission('accessibility')}
disabled={isRequestingPermissions || isRestartingCore}
className="mt-1 rounded-lg border border-primary-500/60 bg-primary-50 px-3 py-2 text-sm text-primary-600 disabled:opacity-50">
{isRequestingPermissions ? 'Requesting…' : 'Request Accessibility'}
</button>
<button
type="button"
onClick={() => void dispatch(requestAccessibilityPermission('input_monitoring'))}
onClick={() => void requestPermission('input_monitoring')}
disabled={isRequestingPermissions || isRestartingCore}
className="rounded-lg border border-primary-500/60 bg-primary-50 px-3 py-2 text-sm text-primary-600 disabled:opacity-50">
{isRequestingPermissions ? 'Requesting…' : 'Open Input Monitoring'}
@@ -168,7 +139,7 @@ const AccessibilityPanel = () => {
{anyPermissionDenied ? (
<button
type="button"
onClick={() => void dispatch(refreshPermissionsWithRestart())}
onClick={() => void refreshPermissionsWithRestart()}
disabled={isRestartingCore || isLoading}
className="rounded-lg border border-amber-500/60 bg-amber-50 px-3 py-2 text-sm text-amber-700 disabled:opacity-50">
{isRestartingCore ? 'Restarting core…' : 'Restart & Refresh Permissions'}
@@ -176,7 +147,7 @@ const AccessibilityPanel = () => {
) : (
<button
type="button"
onClick={() => void dispatch(fetchAccessibilityStatus())}
onClick={() => void refreshStatus()}
disabled={isLoading || isRestartingCore}
className="rounded-lg border border-stone-300 bg-stone-100 px-3 py-2 text-sm text-stone-700 disabled:opacity-50">
{isLoading ? 'Refreshing…' : 'Refresh Status'}
@@ -200,34 +171,6 @@ const AccessibilityPanel = () => {
}
/>
</label>
<label className="flex items-center justify-between rounded-xl border border-stone-200 bg-stone-50 px-3 py-2">
<span className="text-sm text-stone-700">Device Control</span>
<input
type="checkbox"
checked={deviceControl}
onChange={event =>
setFeatureOverrides(current => ({
...current,
device_control: event.target.checked,
}))
}
/>
</label>
<label className="flex items-center justify-between rounded-xl border border-stone-200 bg-stone-50 px-3 py-2">
<span className="text-sm text-stone-700">Predictive Input</span>
<input
type="checkbox"
checked={predictiveInput}
onChange={event =>
setFeatureOverrides(current => ({
...current,
predictive_input: event.target.checked,
}))
}
/>
</label>
</section>
<section className="space-y-3">
@@ -251,15 +194,11 @@ const AccessibilityPanel = () => {
<button
type="button"
onClick={() =>
void dispatch(
startAccessibilitySession({
consent: true,
ttl_secs: status?.config.session_ttl_secs ?? 300,
screen_monitoring: screenMonitoring,
device_control: deviceControl,
predictive_input: predictiveInput,
})
)
void startSession({
consent: true,
ttl_secs: status?.config.session_ttl_secs ?? 300,
screen_monitoring: screenMonitoring,
})
}
disabled={startDisabled}
className="rounded-lg border border-green-500/60 bg-green-50 px-3 py-2 text-sm text-green-700 disabled:opacity-50">
@@ -267,14 +206,14 @@ const AccessibilityPanel = () => {
</button>
<button
type="button"
onClick={() => void dispatch(stopAccessibilitySession('manual_stop'))}
onClick={() => void stopSession('manual_stop')}
disabled={stopDisabled}
className="rounded-lg border border-red-500/60 bg-red-50 px-3 py-2 text-sm text-red-600 disabled:opacity-50">
{isStoppingSession ? 'Stopping…' : 'Stop Session'}
</button>
<button
type="button"
onClick={() => void dispatch(flushAccessibilityVision())}
onClick={() => void flushVision()}
disabled={isFlushingVision || !status?.session.active}
className="rounded-lg border border-primary-500/60 bg-primary-50 px-3 py-2 text-sm text-primary-600 disabled:opacity-50">
{isFlushingVision ? 'Analyzing…' : 'Analyze Now'}
@@ -287,7 +226,7 @@ const AccessibilityPanel = () => {
<h3 className="text-sm font-semibold text-stone-900">Vision Summaries</h3>
<button
type="button"
onClick={() => void dispatch(fetchAccessibilityVisionRecent(10))}
onClick={() => void refreshVision(10)}
disabled={isLoadingVision}
className="rounded-lg border border-stone-300 bg-stone-100 px-3 py-1.5 text-xs text-stone-700 disabled:opacity-50">
{isLoadingVision ? 'Refreshing…' : 'Refresh'}
@@ -1,16 +1,11 @@
import { useEffect, useMemo, useState } from 'react';
import { type ComponentProps, useEffect, useMemo, useState } from 'react';
import ScreenIntelligenceDebugPanel from '../../../components/intelligence/ScreenIntelligenceDebugPanel';
import {
fetchAccessibilityStatus,
fetchAccessibilityVisionRecent,
} from '../../../store/accessibilitySlice';
import { useAppDispatch, useAppSelector } from '../../../store/hooks';
import { useScreenIntelligenceState } from '../../../features/screen-intelligence/useScreenIntelligenceState';
import { isTauri, openhumanUpdateScreenIntelligenceSettings } from '../../../utils/tauriCommands';
import SettingsHeader from '../components/SettingsHeader';
import { useSettingsNavigation } from '../hooks/useSettingsNavigation';
import PermissionsSection from './screen-intelligence/PermissionsSection';
import SessionAndVisionSection from './screen-intelligence/SessionAndVisionSection';
const formatRemaining = (remainingMs: number | null): string => {
if (remainingMs === null || remainingMs <= 0) {
@@ -25,7 +20,11 @@ const formatRemaining = (remainingMs: number | null): string => {
return `${mins}:${secs}`;
};
const DebugSection = () => {
const DebugSection = ({
state,
}: {
state: ComponentProps<typeof ScreenIntelligenceDebugPanel>['state'];
}) => {
const [isOpen, setIsOpen] = useState(false);
return (
@@ -37,16 +36,16 @@ const DebugSection = () => {
<span>Debug & Diagnostics</span>
<span className="text-xs text-stone-400">{isOpen ? 'Collapse' : 'Expand'}</span>
</button>
{isOpen && <ScreenIntelligenceDebugPanel />}
{isOpen && <ScreenIntelligenceDebugPanel state={state} />}
</section>
);
};
const ScreenIntelligencePanel = () => {
const { navigateBack, breadcrumbs } = useSettingsNavigation();
const dispatch = useAppDispatch();
const {
status,
lastRestartSummary,
isLoading,
isRequestingPermissions,
isRestartingCore,
@@ -56,12 +55,18 @@ const ScreenIntelligencePanel = () => {
isFlushingVision,
recentVisionSummaries,
lastError,
} = useAppSelector(state => state.accessibility);
const [featureOverrides, setFeatureOverrides] = useState<{
screen_monitoring?: boolean;
device_control?: boolean;
predictive_input?: boolean;
}>({});
refreshStatus,
refreshVision,
startSession,
stopSession,
flushVision,
requestPermission,
refreshPermissionsWithRestart,
runCaptureTest,
captureTestResult,
isCaptureTestRunning,
} = useScreenIntelligenceState({ loadVision: true, visionLimit: 10, pollMs: 2000 });
const [featureOverrides, setFeatureOverrides] = useState<{ screen_monitoring?: boolean }>({});
const [enabled, setEnabled] = useState<boolean>(false);
const [policyMode, setPolicyMode] = useState<'all_except_blacklist' | 'whitelist_only'>(
'all_except_blacklist'
@@ -74,25 +79,6 @@ const ScreenIntelligencePanel = () => {
const [isSavingConfig, setIsSavingConfig] = useState(false);
const [configError, setConfigError] = useState<string | null>(null);
useEffect(() => {
void dispatch(fetchAccessibilityStatus());
}, [dispatch]);
useEffect(() => {
if (!status?.session.active) {
return;
}
const intervalId = window.setInterval(() => {
void dispatch(fetchAccessibilityStatus());
void dispatch(fetchAccessibilityVisionRecent(10));
}, 1000);
return () => window.clearInterval(intervalId);
}, [dispatch, status?.session.active]);
useEffect(() => {
void dispatch(fetchAccessibilityVisionRecent(10));
}, [dispatch]);
useEffect(() => {
if (!status?.config) {
return;
@@ -110,9 +96,6 @@ const ScreenIntelligencePanel = () => {
const screenMonitoring =
featureOverrides.screen_monitoring ?? status?.features.screen_monitoring ?? true;
const deviceControl = featureOverrides.device_control ?? status?.features.device_control ?? true;
const predictiveInput =
featureOverrides.predictive_input ?? status?.features.predictive_input ?? true;
const remaining = useMemo(
() => formatRemaining(status?.session.remaining_ms ?? null),
@@ -154,7 +137,7 @@ const ScreenIntelligencePanel = () => {
.map(v => v.trim())
.filter(Boolean),
});
await dispatch(fetchAccessibilityStatus());
await refreshStatus();
} catch (error) {
setConfigError(error instanceof Error ? error.message : 'Failed to save screen intelligence');
} finally {
@@ -177,10 +160,14 @@ const ScreenIntelligencePanel = () => {
accessibility={status?.permissions.accessibility ?? 'unknown'}
inputMonitoring={status?.permissions.input_monitoring ?? 'unknown'}
anyPermissionDenied={anyPermissionDenied ?? false}
lastRestartSummary={lastRestartSummary}
permissionCheckProcessPath={status?.permission_check_process_path}
isRequestingPermissions={isRequestingPermissions}
isRestartingCore={isRestartingCore}
isLoading={isLoading}
requestPermission={requestPermission}
refreshPermissionsWithRestart={refreshPermissionsWithRestart}
refreshStatus={refreshStatus}
/>
<section className="space-y-3">
@@ -296,53 +283,100 @@ const ScreenIntelligencePanel = () => {
}
/>
</label>
<label className="flex items-center justify-between rounded-xl border border-stone-200 bg-stone-50 px-3 py-2">
<span className="text-sm text-stone-700">Device Control</span>
<input
type="checkbox"
checked={deviceControl}
onChange={event =>
setFeatureOverrides(current => ({
...current,
device_control: event.target.checked,
}))
}
/>
</label>
<label className="flex items-center justify-between rounded-xl border border-stone-200 bg-stone-50 px-3 py-2">
<span className="text-sm text-stone-700">Predictive Input</span>
<input
type="checkbox"
checked={predictiveInput}
onChange={event =>
setFeatureOverrides(current => ({
...current,
predictive_input: event.target.checked,
}))
}
/>
</label>
</section>
<SessionAndVisionSection
status={status}
isStartingSession={isStartingSession}
isStoppingSession={isStoppingSession}
isFlushingVision={isFlushingVision}
isLoadingVision={isLoadingVision}
startDisabled={startDisabled}
stopDisabled={stopDisabled}
remaining={remaining}
screenMonitoring={screenMonitoring}
deviceControl={deviceControl}
predictiveInput={predictiveInput}
recentVisionSummaries={recentVisionSummaries}
<section className="space-y-3">
<h3 className="text-sm font-semibold text-stone-900">Session</h3>
<div className="text-sm text-stone-600 space-y-1">
<div>Status: {status?.session.active ? 'Active' : 'Stopped'}</div>
<div>Remaining: {remaining}</div>
<div>Frames (ephemeral): {status?.session.frames_in_memory ?? 0}</div>
<div>Panic stop: {status?.session.panic_hotkey ?? 'Cmd+Shift+.'}</div>
<div>Vision: {status?.session.vision_state ?? 'idle'}</div>
<div>Vision queue: {status?.session.vision_queue_depth ?? 0}</div>
<div>
Last vision:{' '}
{status?.session.last_vision_at_ms
? new Date(status.session.last_vision_at_ms).toLocaleTimeString()
: 'n/a'}
</div>
</div>
<div className="flex gap-2">
<button
type="button"
onClick={() =>
void startSession({
consent: true,
ttl_secs: status?.config.session_ttl_secs ?? 300,
screen_monitoring: screenMonitoring,
})
}
disabled={startDisabled}
className="rounded-lg border border-green-400 bg-green-50 px-3 py-2 text-sm text-green-700 disabled:opacity-50">
{isStartingSession ? 'Starting…' : 'Start Session'}
</button>
<button
type="button"
onClick={() => void stopSession('manual_stop')}
disabled={stopDisabled}
className="rounded-lg border border-red-400 bg-red-50 px-3 py-2 text-sm text-red-700 disabled:opacity-50">
{isStoppingSession ? 'Stopping…' : 'Stop Session'}
</button>
<button
type="button"
onClick={() => void flushVision()}
disabled={isFlushingVision || !status?.session.active}
className="rounded-lg border border-primary-400 bg-primary-50 px-3 py-2 text-sm text-primary-700 disabled:opacity-50">
{isFlushingVision ? 'Analyzing…' : 'Analyze Now'}
</button>
</div>
</section>
<section className="space-y-3">
<div className="flex items-center justify-between">
<h3 className="text-sm font-semibold text-stone-900">Vision Summaries</h3>
<button
type="button"
onClick={() => void refreshVision(10)}
disabled={isLoadingVision}
className="rounded-lg border border-stone-200 bg-stone-50 px-3 py-1.5 text-xs text-stone-600 disabled:opacity-50">
{isLoadingVision ? 'Refreshing…' : 'Refresh'}
</button>
</div>
{recentVisionSummaries.length === 0 ? (
<div className="text-xs text-stone-500">No summaries yet.</div>
) : (
<div className="space-y-2">
{recentVisionSummaries.map(summary => (
<div
key={summary.id}
className="rounded-xl border border-stone-200 bg-white p-3 text-xs text-stone-200">
<div className="text-stone-500">
{new Date(summary.captured_at_ms).toLocaleTimeString()} ·{' '}
{summary.app_name ?? 'Unknown App'}
{summary.window_title ? ` · ${summary.window_title}` : ''}
</div>
<div className="mt-1 text-stone-800">{summary.actionable_notes}</div>
</div>
))}
</div>
)}
</section>
<DebugSection
state={{
status,
recentVisionSummaries,
lastError,
captureTestResult,
isCaptureTestRunning,
refreshStatus,
refreshVision,
runCaptureTest,
}}
/>
<DebugSection />
{status !== null && !status.platform_supported && (
<div className="rounded-xl border border-amber-300 bg-amber-50 p-3 text-sm text-amber-700">
Screen Intelligence V1 is currently supported on macOS only.
@@ -1,97 +1,91 @@
import { configureStore } from '@reduxjs/toolkit';
import { render, screen } from '@testing-library/react';
import { Provider } from 'react-redux';
import { MemoryRouter } from 'react-router-dom';
import { describe, expect, it } from 'vitest';
import { describe, expect, it, vi } from 'vitest';
import accessibilityReducer from '../../../../store/accessibilitySlice';
import authReducer from '../../../../store/authSlice';
import socketReducer from '../../../../store/socketSlice';
import teamReducer from '../../../../store/teamSlice';
import userReducer from '../../../../store/userSlice';
import type { AccessibilityStatus } from '../../../../utils/tauriCommands';
import {
type ScreenIntelligenceState,
useScreenIntelligenceState,
} from '../../../../features/screen-intelligence/useScreenIntelligenceState';
import AccessibilityPanel from '../AccessibilityPanel';
const status: AccessibilityStatus = {
platform_supported: true,
permissions: {
screen_recording: 'unknown',
accessibility: 'granted',
input_monitoring: 'unknown',
},
features: { screen_monitoring: true, device_control: true, predictive_input: true },
session: {
active: false,
started_at_ms: null,
expires_at_ms: null,
remaining_ms: null,
ttl_secs: 300,
panic_hotkey: 'Cmd+Shift+.',
stop_reason: null,
frames_in_memory: 0,
last_capture_at_ms: null,
last_context: null,
vision_enabled: true,
vision_state: 'idle',
vision_queue_depth: 0,
last_vision_at_ms: null,
last_vision_summary: null,
},
config: {
enabled: true,
capture_policy: 'hybrid',
policy_mode: 'all_except_blacklist',
baseline_fps: 1,
vision_enabled: true,
session_ttl_secs: 300,
panic_stop_hotkey: 'Cmd+Shift+.',
autocomplete_enabled: true,
use_vision_model: true,
keep_screenshots: false,
allowlist: [],
denylist: ['wallet'],
},
denylist: ['wallet'],
is_context_blocked: false,
};
vi.mock('../../../../features/screen-intelligence/useScreenIntelligenceState', () => ({
useScreenIntelligenceState: vi.fn(),
}));
const createStore = () =>
configureStore({
reducer: {
auth: authReducer,
socket: socketReducer,
user: userReducer,
team: teamReducer,
accessibility: accessibilityReducer,
const mockState: ScreenIntelligenceState = {
status: {
platform_supported: true,
permissions: {
screen_recording: 'unknown',
accessibility: 'granted',
input_monitoring: 'unknown',
},
preloadedState: {
accessibility: {
status,
recentVisionSummaries: [],
captureTestResult: null,
isCaptureTestRunning: false,
isLoading: false,
isRequestingPermissions: false,
isRestartingCore: false,
isStartingSession: false,
isStoppingSession: false,
isLoadingVision: false,
isFlushingVision: false,
lastError: null,
},
features: { screen_monitoring: true },
session: {
active: false,
started_at_ms: null,
expires_at_ms: null,
remaining_ms: null,
ttl_secs: 300,
panic_hotkey: 'Cmd+Shift+.',
stop_reason: null,
frames_in_memory: 0,
last_capture_at_ms: null,
last_context: null,
vision_enabled: true,
vision_state: 'idle',
vision_queue_depth: 0,
last_vision_at_ms: null,
last_vision_summary: null,
},
});
config: {
enabled: true,
capture_policy: 'hybrid',
policy_mode: 'all_except_blacklist',
baseline_fps: 1,
vision_enabled: true,
session_ttl_secs: 300,
panic_stop_hotkey: 'Cmd+Shift+.',
autocomplete_enabled: true,
use_vision_model: true,
keep_screenshots: false,
allowlist: [],
denylist: ['wallet'],
},
denylist: ['wallet'],
is_context_blocked: false,
},
lastRestartSummary: null,
recentVisionSummaries: [],
captureTestResult: null,
isCaptureTestRunning: false,
isLoading: false,
isRequestingPermissions: false,
isRestartingCore: false,
isStartingSession: false,
isStoppingSession: false,
isLoadingVision: false,
isFlushingVision: false,
lastError: null,
refreshStatus: vi.fn(),
requestPermission: vi.fn(),
refreshPermissionsWithRestart: vi.fn(),
startSession: vi.fn(),
stopSession: vi.fn(),
refreshVision: vi.fn(),
flushVision: vi.fn(),
runCaptureTest: vi.fn(),
clearError: vi.fn(),
};
describe('AccessibilityPanel', () => {
it('renders permission and session sections', () => {
const store = createStore();
vi.mocked(useScreenIntelligenceState).mockReturnValue(mockState);
render(
<Provider store={store}>
<MemoryRouter initialEntries={['/settings/accessibility']}>
<AccessibilityPanel />
</MemoryRouter>
</Provider>
<MemoryRouter initialEntries={['/settings/accessibility']}>
<AccessibilityPanel />
</MemoryRouter>
);
expect(screen.getByText('Accessibility Automation')).toBeInTheDocument();
@@ -1,119 +1,106 @@
import { configureStore } from '@reduxjs/toolkit';
import { fireEvent, render, screen, waitFor } from '@testing-library/react';
import { Provider } from 'react-redux';
import { MemoryRouter } from 'react-router-dom';
import { beforeEach, describe, expect, it, vi } from 'vitest';
import accessibilityReducer from '../../../../store/accessibilitySlice';
import authReducer from '../../../../store/authSlice';
import socketReducer from '../../../../store/socketSlice';
import teamReducer from '../../../../store/teamSlice';
import userReducer from '../../../../store/userSlice';
import {
type AccessibilityStatus,
type AccessibilityVisionRecentResult,
type CommandResponse,
type ScreenIntelligenceState,
useScreenIntelligenceState,
} from '../../../../features/screen-intelligence/useScreenIntelligenceState';
import {
type ConfigSnapshot,
isTauri,
openhumanAccessibilityInputAction,
openhumanAccessibilityRequestPermission,
openhumanAccessibilityRequestPermissions,
openhumanAccessibilityStartSession,
openhumanAccessibilityStatus,
openhumanAccessibilityStopSession,
openhumanAccessibilityVisionFlush,
openhumanAccessibilityVisionRecent,
openhumanScreenIntelligenceCaptureTest,
openhumanUpdateScreenIntelligenceSettings,
restartCoreProcess,
} from '../../../../utils/tauriCommands';
import ScreenIntelligencePanel from '../ScreenIntelligencePanel';
vi.mock('../../../../utils/tauriCommands', () => ({
isTauri: vi.fn(() => true),
openhumanAccessibilityInputAction: vi.fn(),
openhumanAccessibilityRequestPermission: vi.fn(),
openhumanAccessibilityRequestPermissions: vi.fn(),
openhumanAccessibilityStartSession: vi.fn(),
openhumanAccessibilityStatus: vi.fn(),
openhumanAccessibilityStopSession: vi.fn(),
openhumanAccessibilityVisionFlush: vi.fn(),
openhumanAccessibilityVisionRecent: vi.fn(),
openhumanScreenIntelligenceCaptureTest: vi.fn(),
openhumanUpdateScreenIntelligenceSettings: vi.fn(),
restartCoreProcess: vi.fn(),
vi.mock('../../../../features/screen-intelligence/useScreenIntelligenceState', () => ({
useScreenIntelligenceState: vi.fn(),
}));
const baseStatus: AccessibilityStatus = {
platform_supported: true,
permissions: {
screen_recording: 'granted',
accessibility: 'granted',
input_monitoring: 'unknown',
},
features: { screen_monitoring: true, device_control: true, predictive_input: true },
session: {
active: false,
started_at_ms: null,
expires_at_ms: null,
remaining_ms: null,
ttl_secs: 300,
panic_hotkey: 'Cmd+Shift+.',
stop_reason: null,
frames_in_memory: 0,
last_capture_at_ms: null,
last_context: null,
vision_enabled: true,
vision_state: 'idle',
vision_queue_depth: 0,
last_vision_at_ms: null,
last_vision_summary: null,
},
config: {
enabled: false,
capture_policy: 'hybrid',
policy_mode: 'all_except_blacklist',
baseline_fps: 1,
vision_enabled: true,
session_ttl_secs: 300,
panic_stop_hotkey: 'Cmd+Shift+.',
autocomplete_enabled: true,
use_vision_model: true,
keep_screenshots: false,
allowlist: ['Code'],
denylist: ['1Password'],
},
denylist: ['1Password'],
is_context_blocked: false,
permission_check_process_path: '/tmp/openhuman-core',
};
vi.mock('../../../../utils/tauriCommands', async importOriginal => {
const actual = await importOriginal<typeof import('../../../../utils/tauriCommands')>();
return {
...actual,
isTauri: vi.fn(() => true),
openhumanUpdateScreenIntelligenceSettings: vi.fn(),
};
});
const emptyVisionResponse: CommandResponse<AccessibilityVisionRecentResult> = {
result: { summaries: [] },
logs: [],
};
const createStore = () =>
configureStore({
reducer: {
auth: authReducer,
socket: socketReducer,
user: userReducer,
team: teamReducer,
accessibility: accessibilityReducer,
const baseState: ScreenIntelligenceState = {
status: {
platform_supported: true,
core_process: { pid: 4242, started_at_ms: 1712700000000 },
permissions: {
screen_recording: 'granted',
accessibility: 'granted',
input_monitoring: 'unknown',
},
});
features: { screen_monitoring: true },
session: {
active: false,
started_at_ms: null,
expires_at_ms: null,
remaining_ms: null,
ttl_secs: 300,
panic_hotkey: 'Cmd+Shift+.',
stop_reason: null,
frames_in_memory: 0,
last_capture_at_ms: null,
last_context: null,
vision_enabled: true,
vision_state: 'idle',
vision_queue_depth: 0,
last_vision_at_ms: null,
last_vision_summary: null,
},
config: {
enabled: false,
capture_policy: 'hybrid',
policy_mode: 'all_except_blacklist',
baseline_fps: 1,
vision_enabled: true,
session_ttl_secs: 300,
panic_stop_hotkey: 'Cmd+Shift+.',
autocomplete_enabled: true,
use_vision_model: true,
keep_screenshots: false,
allowlist: ['Code'],
denylist: ['1Password'],
},
denylist: ['1Password'],
is_context_blocked: false,
permission_check_process_path: '/tmp/openhuman-core',
},
lastRestartSummary: null,
recentVisionSummaries: [],
captureTestResult: null,
isCaptureTestRunning: false,
isLoading: false,
isRequestingPermissions: false,
isRestartingCore: false,
isStartingSession: false,
isStoppingSession: false,
isLoadingVision: false,
isFlushingVision: false,
lastError: null,
refreshStatus: vi.fn().mockResolvedValue(null),
requestPermission: vi.fn().mockResolvedValue(null),
refreshPermissionsWithRestart: vi.fn().mockResolvedValue(null),
startSession: vi.fn().mockResolvedValue(null),
stopSession: vi.fn().mockResolvedValue(null),
refreshVision: vi.fn().mockResolvedValue([]),
flushVision: vi.fn().mockResolvedValue(undefined),
runCaptureTest: vi.fn().mockResolvedValue(undefined),
clearError: vi.fn(),
};
function renderPanel() {
const store = createStore();
function renderPanel(state: ScreenIntelligenceState = baseState) {
vi.mocked(useScreenIntelligenceState).mockReturnValue(state);
render(
<Provider store={store}>
<MemoryRouter initialEntries={['/settings/screen-intelligence']}>
<ScreenIntelligencePanel />
</MemoryRouter>
</Provider>
<MemoryRouter initialEntries={['/settings/screen-intelligence']}>
<ScreenIntelligencePanel />
</MemoryRouter>
);
return store;
}
function createDeferred<T>() {
@@ -126,50 +113,12 @@ function createDeferred<T>() {
describe('ScreenIntelligencePanel', () => {
beforeEach(() => {
vi.clearAllMocks();
vi.mocked(isTauri).mockReturnValue(true);
vi.mocked(openhumanAccessibilityStatus).mockResolvedValue({ result: baseStatus, logs: [] });
vi.mocked(openhumanAccessibilityVisionRecent).mockResolvedValue(emptyVisionResponse);
vi.mocked(openhumanAccessibilityInputAction).mockResolvedValue({
result: {} as never,
logs: [],
});
vi.mocked(openhumanAccessibilityRequestPermission).mockResolvedValue({
result: baseStatus.permissions,
logs: [],
} as never);
vi.mocked(openhumanAccessibilityRequestPermissions).mockResolvedValue({
result: baseStatus.permissions,
logs: [],
} as never);
vi.mocked(openhumanAccessibilityStartSession).mockResolvedValue({
result: baseStatus.session,
logs: [],
} as never);
vi.mocked(openhumanAccessibilityStopSession).mockResolvedValue({
result: baseStatus.session,
logs: [],
} as never);
vi.mocked(openhumanAccessibilityVisionFlush).mockResolvedValue({
result: { accepted: true, summary: null },
logs: [],
} as never);
vi.mocked(openhumanScreenIntelligenceCaptureTest).mockResolvedValue({
result: {
ok: false,
capture_mode: 'fullscreen',
context: null,
image_ref: null,
bytes_estimate: null,
error: 'screen capture is unsupported on this platform',
timing_ms: 12,
},
logs: [],
});
vi.mocked(restartCoreProcess).mockResolvedValue(undefined);
});
it('saves screen intelligence settings and clears the saving state', async () => {
const deferred = createDeferred<CommandResponse<ConfigSnapshot>>();
it('saves screen intelligence settings and refreshes core-backed status', async () => {
const deferred = createDeferred<{ result: ConfigSnapshot; logs: [] }>();
vi.mocked(openhumanUpdateScreenIntelligenceSettings).mockReturnValueOnce(deferred.promise);
renderPanel();
@@ -206,13 +155,14 @@ describe('ScreenIntelligencePanel', () => {
screen.getByRole('button', { name: 'Save Screen Intelligence Settings' })
).toBeInTheDocument();
});
expect(openhumanAccessibilityStatus).toHaveBeenCalledTimes(2);
expect(baseState.refreshStatus).toHaveBeenCalledTimes(1);
});
it('shows permission restart guidance and unsupported-platform messaging', async () => {
vi.mocked(openhumanAccessibilityStatus).mockResolvedValueOnce({
result: {
...baseStatus,
renderPanel({
...baseState,
status: {
...baseState.status!,
platform_supported: false,
permissions: {
screen_recording: 'denied',
@@ -220,11 +170,8 @@ describe('ScreenIntelligencePanel', () => {
input_monitoring: 'unknown',
},
},
logs: [],
});
renderPanel();
expect(await screen.findByText('Permissions')).toBeInTheDocument();
expect(screen.getByText(/After granting in System Settings, click/i)).toBeInTheDocument();
expect(
@@ -234,4 +181,13 @@ describe('ScreenIntelligencePanel', () => {
screen.getByText('Screen Intelligence V1 is currently supported on macOS only.')
).toBeInTheDocument();
});
it('shows the last successful restart summary', async () => {
renderPanel({
...baseState,
lastRestartSummary: 'Core restarted: PID 4000 at 9:00:00 AM -> PID 4242 at 9:01:00 AM.',
});
expect(await screen.findByText(/Core restarted: PID 4000/i)).toBeInTheDocument();
});
});
@@ -1,9 +1,4 @@
import {
fetchAccessibilityStatus,
refreshPermissionsWithRestart,
requestAccessibilityPermission,
} from '../../../../store/accessibilitySlice';
import { useAppDispatch } from '../../../../store/hooks';
import type { AccessibilityPermissionKind } from '../../../../utils/tauriCommands';
interface PermissionsBadgeProps {
label: string;
@@ -33,10 +28,14 @@ interface PermissionsSectionProps {
accessibility: string;
inputMonitoring: string;
anyPermissionDenied: boolean;
lastRestartSummary: string | null;
permissionCheckProcessPath: string | null | undefined;
isRequestingPermissions: boolean;
isRestartingCore: boolean;
isLoading: boolean;
requestPermission: (permission: AccessibilityPermissionKind) => Promise<unknown>;
refreshPermissionsWithRestart: () => Promise<unknown>;
refreshStatus: () => Promise<unknown>;
}
const PermissionsSection = ({
@@ -44,13 +43,15 @@ const PermissionsSection = ({
accessibility,
inputMonitoring,
anyPermissionDenied,
lastRestartSummary,
permissionCheckProcessPath,
isRequestingPermissions,
isRestartingCore,
isLoading,
requestPermission,
refreshPermissionsWithRestart,
refreshStatus,
}: PermissionsSectionProps) => {
const dispatch = useAppDispatch();
return (
<section className="space-y-3">
<h3 className="text-sm font-semibold text-stone-900">Permissions</h3>
@@ -75,23 +76,29 @@ const PermissionsSection = ({
</div>
)}
{lastRestartSummary ? (
<div className="rounded-xl border border-green-300 bg-green-50 p-3 text-sm text-green-700">
{lastRestartSummary}
</div>
) : null}
<button
type="button"
onClick={() => void dispatch(requestAccessibilityPermission('screen_recording'))}
onClick={() => void requestPermission('screen_recording')}
disabled={isRequestingPermissions || isRestartingCore}
className="mt-1 rounded-lg border border-primary-400 bg-primary-50 px-3 py-2 text-sm text-primary-700 disabled:opacity-50">
{isRequestingPermissions ? 'Requesting…' : 'Request Screen Recording'}
</button>
<button
type="button"
onClick={() => void dispatch(requestAccessibilityPermission('accessibility'))}
onClick={() => void requestPermission('accessibility')}
disabled={isRequestingPermissions || isRestartingCore}
className="rounded-lg border border-primary-400 bg-primary-50 px-3 py-2 text-sm text-primary-700 disabled:opacity-50">
{isRequestingPermissions ? 'Requesting…' : 'Request Accessibility'}
</button>
<button
type="button"
onClick={() => void dispatch(requestAccessibilityPermission('input_monitoring'))}
onClick={() => void requestPermission('input_monitoring')}
disabled={isRequestingPermissions || isRestartingCore}
className="rounded-lg border border-primary-400 bg-primary-50 px-3 py-2 text-sm text-primary-700 disabled:opacity-50">
{isRequestingPermissions ? 'Requesting…' : 'Open Input Monitoring'}
@@ -99,7 +106,7 @@ const PermissionsSection = ({
{anyPermissionDenied ? (
<button
type="button"
onClick={() => void dispatch(refreshPermissionsWithRestart())}
onClick={() => void refreshPermissionsWithRestart()}
disabled={isRestartingCore || isLoading}
className="rounded-lg border border-amber-400 bg-amber-50 px-3 py-2 text-sm text-amber-700 disabled:opacity-50">
{isRestartingCore ? 'Restarting core…' : 'Restart & Refresh Permissions'}
@@ -107,7 +114,7 @@ const PermissionsSection = ({
) : (
<button
type="button"
onClick={() => void dispatch(fetchAccessibilityStatus())}
onClick={() => void refreshStatus()}
disabled={isLoading || isRestartingCore}
className="rounded-lg border border-stone-200 bg-stone-50 px-3 py-2 text-sm text-stone-700 disabled:opacity-50">
{isLoading ? 'Refreshing…' : 'Refresh Status'}
@@ -1,133 +0,0 @@
import {
fetchAccessibilityVisionRecent,
flushAccessibilityVision,
startAccessibilitySession,
stopAccessibilitySession,
} from '../../../../store/accessibilitySlice';
import { useAppDispatch } from '../../../../store/hooks';
import type {
AccessibilityStatus,
AccessibilityVisionSummary,
} from '../../../../utils/tauriCommands';
interface SessionAndVisionSectionProps {
status: AccessibilityStatus | null;
isStartingSession: boolean;
isStoppingSession: boolean;
isFlushingVision: boolean;
isLoadingVision: boolean;
startDisabled: boolean;
stopDisabled: boolean;
remaining: string;
screenMonitoring: boolean;
deviceControl: boolean;
predictiveInput: boolean;
recentVisionSummaries: AccessibilityVisionSummary[];
}
const SessionAndVisionSection = ({
status,
isStartingSession,
isStoppingSession,
isFlushingVision,
isLoadingVision,
startDisabled,
stopDisabled,
remaining,
screenMonitoring,
deviceControl,
predictiveInput,
recentVisionSummaries,
}: SessionAndVisionSectionProps) => {
const dispatch = useAppDispatch();
return (
<>
<section className="space-y-3">
<h3 className="text-sm font-semibold text-stone-900">Session</h3>
<div className="text-sm text-stone-600 space-y-1">
<div>Status: {status?.session.active ? 'Active' : 'Stopped'}</div>
<div>Remaining: {remaining}</div>
<div>Frames (ephemeral): {status?.session.frames_in_memory ?? 0}</div>
<div>Panic stop: {status?.session.panic_hotkey ?? 'Cmd+Shift+.'}</div>
<div>Vision: {status?.session.vision_state ?? 'idle'}</div>
<div>Vision queue: {status?.session.vision_queue_depth ?? 0}</div>
<div>
Last vision:{' '}
{status?.session.last_vision_at_ms
? new Date(status.session.last_vision_at_ms).toLocaleTimeString()
: 'n/a'}
</div>
</div>
<div className="flex gap-2">
<button
type="button"
onClick={() =>
void dispatch(
startAccessibilitySession({
consent: true,
ttl_secs: status?.config.session_ttl_secs ?? 300,
screen_monitoring: screenMonitoring,
device_control: deviceControl,
predictive_input: predictiveInput,
})
)
}
disabled={startDisabled}
className="rounded-lg border border-green-400 bg-green-50 px-3 py-2 text-sm text-green-700 disabled:opacity-50">
{isStartingSession ? 'Starting…' : 'Start Session'}
</button>
<button
type="button"
onClick={() => void dispatch(stopAccessibilitySession('manual_stop'))}
disabled={stopDisabled}
className="rounded-lg border border-red-400 bg-red-50 px-3 py-2 text-sm text-red-700 disabled:opacity-50">
{isStoppingSession ? 'Stopping…' : 'Stop Session'}
</button>
<button
type="button"
onClick={() => void dispatch(flushAccessibilityVision())}
disabled={isFlushingVision || !status?.session.active}
className="rounded-lg border border-primary-400 bg-primary-50 px-3 py-2 text-sm text-primary-700 disabled:opacity-50">
{isFlushingVision ? 'Analyzing…' : 'Analyze Now'}
</button>
</div>
</section>
<section className="space-y-3">
<div className="flex items-center justify-between">
<h3 className="text-sm font-semibold text-stone-900">Vision Summaries</h3>
<button
type="button"
onClick={() => void dispatch(fetchAccessibilityVisionRecent(10))}
disabled={isLoadingVision}
className="rounded-lg border border-stone-200 bg-stone-50 px-3 py-1.5 text-xs text-stone-600 disabled:opacity-50">
{isLoadingVision ? 'Refreshing…' : 'Refresh'}
</button>
</div>
{recentVisionSummaries.length === 0 ? (
<div className="text-xs text-stone-500">No summaries yet.</div>
) : (
<div className="space-y-2">
{recentVisionSummaries.map(summary => (
<div
key={summary.id}
className="rounded-xl border border-stone-200 bg-white p-3 text-xs text-stone-200">
<div className="text-stone-500">
{new Date(summary.captured_at_ms).toLocaleTimeString()} ·{' '}
{summary.app_name ?? 'Unknown App'}
{summary.window_title ? ` · ${summary.window_title}` : ''}
</div>
<div className="mt-1 text-stone-800">{summary.actionable_notes}</div>
</div>
))}
</div>
)}
</section>
</>
);
};
export default SessionAndVisionSection;
+1 -1
View File
@@ -1,7 +1,7 @@
import { useState } from 'react';
import type { TunnelRegistration } from '../../features/webhooks/types';
import { type Tunnel, tunnelsApi } from '../../services/api/tunnelsApi';
import type { TunnelRegistration } from '../../store/webhooksSlice';
import { BACKEND_URL } from '../../utils/config';
interface TunnelListProps {
@@ -1,4 +1,4 @@
import type { WebhookActivityEntry } from '../../store/webhooksSlice';
import type { WebhookActivityEntry } from '../../features/webhooks/types';
interface WebhookActivityProps {
activity: WebhookActivityEntry[];
+146
View File
@@ -0,0 +1,146 @@
import { useSyncExternalStore } from 'react';
export type DaemonStatus = 'starting' | 'stopping' | 'running' | 'error' | 'disconnected';
export type ComponentStatus = 'ok' | 'error' | 'starting';
export interface ComponentHealth {
status: ComponentStatus;
updated_at: string;
last_ok?: string;
last_error?: string;
restart_count: number;
}
export interface HealthSnapshot {
pid: number;
updated_at: string;
uptime_seconds: number;
components: Record<string, ComponentHealth>;
}
export interface DaemonUserState {
status: DaemonStatus;
healthSnapshot: HealthSnapshot | null;
components: {
gateway?: ComponentHealth;
channels?: ComponentHealth;
heartbeat?: ComponentHealth;
scheduler?: ComponentHealth;
};
lastHealthUpdate: string | null;
connectionAttempts: number;
autoStartEnabled: boolean;
isRecovering: boolean;
}
interface DaemonState {
byUser: Record<string, DaemonUserState>;
}
const initialUserState: DaemonUserState = {
status: 'disconnected',
healthSnapshot: null,
components: {},
lastHealthUpdate: null,
connectionAttempts: 0,
autoStartEnabled: false,
isRecovering: false,
};
let daemonState: DaemonState = { byUser: {} };
const listeners = new Set<() => void>();
const emitChange = (): void => {
for (const listener of listeners) {
listener();
}
};
const currentUserState = (userId: string): DaemonUserState =>
daemonState.byUser[userId] ?? initialUserState;
const updateUserState = (
userId: string,
updater: (current: DaemonUserState) => DaemonUserState
): void => {
daemonState = {
...daemonState,
byUser: { ...daemonState.byUser, [userId]: updater(currentUserState(userId)) },
};
emitChange();
};
export const subscribeDaemonStore = (listener: () => void): (() => void) => {
listeners.add(listener);
return () => {
listeners.delete(listener);
};
};
export const getDaemonUserState = (userId?: string): DaemonUserState =>
currentUserState(userId || '__pending__');
export const useDaemonUserState = (userId?: string): DaemonUserState =>
useSyncExternalStore(
subscribeDaemonStore,
() => getDaemonUserState(userId),
() => getDaemonUserState(userId)
);
export const updateHealthSnapshot = (userId: string, healthSnapshot: HealthSnapshot): void => {
updateUserState(userId, current => {
const componentStatuses = Object.values(healthSnapshot.components).map(
component => component.status
);
let status = current.status;
if (componentStatuses.length === 0) {
status = 'disconnected';
} else if (componentStatuses.every(componentStatus => componentStatus === 'ok')) {
status = 'running';
} else if (componentStatuses.some(componentStatus => componentStatus === 'error')) {
status = 'error';
} else if (componentStatuses.some(componentStatus => componentStatus === 'starting')) {
status = 'starting';
}
return {
...current,
status,
healthSnapshot,
components: healthSnapshot.components,
lastHealthUpdate: new Date().toISOString(),
isRecovering: status === 'running' ? false : current.isRecovering,
connectionAttempts: status === 'running' ? 0 : current.connectionAttempts,
};
});
};
export const setDaemonStatus = (userId: string, status: DaemonStatus): void => {
updateUserState(userId, current => ({
...current,
status,
healthSnapshot: status === 'disconnected' ? null : current.healthSnapshot,
components: status === 'disconnected' ? {} : current.components,
lastHealthUpdate: status === 'disconnected' ? null : current.lastHealthUpdate,
}));
};
export const incrementConnectionAttempts = (userId: string): void => {
updateUserState(userId, current => ({
...current,
connectionAttempts: current.connectionAttempts + 1,
}));
};
export const resetConnectionAttempts = (userId: string): void => {
updateUserState(userId, current => ({ ...current, connectionAttempts: 0 }));
};
export const setAutoStartEnabled = (userId: string, enabled: boolean): void => {
updateUserState(userId, current => ({ ...current, autoStartEnabled: enabled }));
};
export const setIsRecovering = (userId: string, isRecovering: boolean): void => {
updateUserState(userId, current => ({ ...current, isRecovering }));
};
+156
View File
@@ -0,0 +1,156 @@
import {
type AccessibilityPermissionKind,
type AccessibilityStartSessionParams,
type AccessibilityStatus,
openhumanAccessibilityRequestPermission,
openhumanAccessibilityStartSession,
openhumanAccessibilityStatus,
openhumanAccessibilityStopSession,
openhumanAccessibilityVisionFlush,
openhumanAccessibilityVisionRecent,
openhumanScreenIntelligenceCaptureTest,
openhumanServiceRestart,
} from '../../utils/tauriCommands';
const ACCESSIBILITY_ERROR_PREFIX = '[screen-intelligence]';
const extractError = (error: unknown, fallback: string): string => {
if (error instanceof Error && error.message.trim()) {
return error.message;
}
if (typeof error === 'string' && error.trim()) {
return error;
}
if (error && typeof error === 'object') {
const msg = (error as { message?: unknown }).message;
if (typeof msg === 'string' && msg.trim()) {
return msg;
}
}
return fallback;
};
const formatCoreIdentity = (status: AccessibilityStatus | null | undefined): string | null => {
const process = status?.core_process;
if (!process) {
return null;
}
const startedAt = Number.isFinite(process.started_at_ms)
? new Date(process.started_at_ms).toLocaleTimeString()
: null;
return startedAt ? `PID ${process.pid} at ${startedAt}` : `PID ${process.pid}`;
};
export interface RefreshPermissionsResult {
status: AccessibilityStatus;
restartSummary: string;
}
export async function fetchScreenIntelligenceStatus(): Promise<AccessibilityStatus> {
const response = await openhumanAccessibilityStatus();
return response.result;
}
export async function requestScreenIntelligencePermission(
permission: AccessibilityPermissionKind
): Promise<AccessibilityStatus> {
await openhumanAccessibilityRequestPermission(permission);
return await fetchScreenIntelligenceStatus();
}
export async function refreshScreenIntelligencePermissionsWithRestart(
previousStatus: AccessibilityStatus | null
): Promise<RefreshPermissionsResult> {
try {
const previousProcess = previousStatus?.core_process;
console.debug(
`${ACCESSIBILITY_ERROR_PREFIX} refreshPermissionsWithRestart: requesting core self-restart`
);
await openhumanServiceRestart('screen-intelligence-ui', 'refresh_permissions');
console.debug(
`${ACCESSIBILITY_ERROR_PREFIX} refreshPermissionsWithRestart: waiting for sidecar ready`
);
await new Promise<void>(resolve => setTimeout(resolve, 400));
console.debug(
`${ACCESSIBILITY_ERROR_PREFIX} refreshPermissionsWithRestart: fetching updated status`
);
for (let attempt = 1; attempt <= 5; attempt += 1) {
try {
const status = await fetchScreenIntelligenceStatus();
console.debug(
`${ACCESSIBILITY_ERROR_PREFIX} refreshPermissionsWithRestart: done screen_recording=%s accessibility=%s input_monitoring=%s`,
status.permissions.screen_recording,
status.permissions.accessibility,
status.permissions.input_monitoring
);
const currentProcess = status.core_process;
if (
previousProcess &&
currentProcess &&
previousProcess.pid === currentProcess.pid &&
previousProcess.started_at_ms === currentProcess.started_at_ms
) {
throw new Error(
`Core restart command completed, but the same core instance is still serving requests (${formatCoreIdentity(status)}).`
);
}
const previousLabel = formatCoreIdentity(previousStatus);
const currentLabel = formatCoreIdentity(status);
const restartSummary =
previousLabel && currentLabel
? `Core restarted: ${previousLabel} -> ${currentLabel}.`
: currentLabel
? `Core restarted. Now serving from ${currentLabel}.`
: 'Core restarted and permissions refreshed.';
return { status, restartSummary };
} catch (error) {
if (attempt === 5) {
throw error;
}
console.debug(
`${ACCESSIBILITY_ERROR_PREFIX} refreshPermissionsWithRestart: status fetch failed (attempt %s), retrying`,
attempt
);
await new Promise<void>(resolve => setTimeout(resolve, 350 * attempt));
}
}
throw new Error('Failed to fetch accessibility status after core restart');
} catch (error) {
const message = extractError(error, 'Failed to restart core and refresh permissions');
console.error(`${ACCESSIBILITY_ERROR_PREFIX} refreshPermissionsWithRestart: error`, message);
throw new Error(message);
}
}
export async function startScreenIntelligenceSession(
params: AccessibilityStartSessionParams
): Promise<AccessibilityStatus> {
await openhumanAccessibilityStartSession(params);
return await fetchScreenIntelligenceStatus();
}
export async function stopScreenIntelligenceSession(reason?: string): Promise<AccessibilityStatus> {
await openhumanAccessibilityStopSession(reason ? { reason } : undefined);
return await fetchScreenIntelligenceStatus();
}
export async function fetchScreenIntelligenceVisionRecent(limit?: number) {
const response = await openhumanAccessibilityVisionRecent(limit);
return response.result.summaries;
}
export async function flushScreenIntelligenceVision() {
const response = await openhumanAccessibilityVisionFlush();
return response.result.summary;
}
export async function runScreenIntelligenceCaptureTest() {
const response = await openhumanScreenIntelligenceCaptureTest();
return response.result;
}
export { extractError };
@@ -0,0 +1,255 @@
import { useCallback, useEffect, useState } from 'react';
import { getCoreStateSnapshot } from '../../lib/coreState/store';
import { useCoreState } from '../../providers/CoreStateProvider';
import type {
AccessibilityPermissionKind,
AccessibilityStartSessionParams,
AccessibilityStatus,
AccessibilityVisionSummary,
CaptureTestResult,
} from '../../utils/tauriCommands';
import {
extractError,
fetchScreenIntelligenceVisionRecent,
flushScreenIntelligenceVision,
refreshScreenIntelligencePermissionsWithRestart,
requestScreenIntelligencePermission,
runScreenIntelligenceCaptureTest,
startScreenIntelligenceSession,
stopScreenIntelligenceSession,
} from './api';
export interface ScreenIntelligenceState {
status: AccessibilityStatus | null;
lastRestartSummary: string | null;
recentVisionSummaries: AccessibilityVisionSummary[];
captureTestResult: CaptureTestResult | null;
isCaptureTestRunning: boolean;
isLoading: boolean;
isRequestingPermissions: boolean;
isRestartingCore: boolean;
isStartingSession: boolean;
isStoppingSession: boolean;
isLoadingVision: boolean;
isFlushingVision: boolean;
lastError: string | null;
refreshStatus: () => Promise<AccessibilityStatus | null>;
requestPermission: (
permission: AccessibilityPermissionKind
) => Promise<AccessibilityStatus | null>;
refreshPermissionsWithRestart: () => Promise<AccessibilityStatus | null>;
startSession: (params: AccessibilityStartSessionParams) => Promise<AccessibilityStatus | null>;
stopSession: (reason?: string) => Promise<AccessibilityStatus | null>;
refreshVision: (limit?: number) => Promise<AccessibilityVisionSummary[]>;
flushVision: () => Promise<void>;
runCaptureTest: () => Promise<void>;
clearError: () => void;
}
export interface UseScreenIntelligenceStateOptions {
pollMs?: number;
visionLimit?: number;
loadVision?: boolean;
}
export function useScreenIntelligenceState(
options: UseScreenIntelligenceStateOptions = {}
): ScreenIntelligenceState {
const { pollMs = 2000, visionLimit = 10, loadVision = false } = options;
const { refresh: refreshCoreState, snapshot } = useCoreState();
const status = snapshot.runtime.screenIntelligence;
const [lastRestartSummary, setLastRestartSummary] = useState<string | null>(null);
const [recentVisionSummaries, setRecentVisionSummaries] = useState<AccessibilityVisionSummary[]>(
[]
);
const [captureTestResult, setCaptureTestResult] = useState<CaptureTestResult | null>(null);
const [isCaptureTestRunning, setIsCaptureTestRunning] = useState(false);
const [isLoading, setIsLoading] = useState(false);
const [isRequestingPermissions, setIsRequestingPermissions] = useState(false);
const [isRestartingCore, setIsRestartingCore] = useState(false);
const [isStartingSession, setIsStartingSession] = useState(false);
const [isStoppingSession, setIsStoppingSession] = useState(false);
const [isLoadingVision, setIsLoadingVision] = useState(false);
const [isFlushingVision, setIsFlushingVision] = useState(false);
const [lastError, setLastError] = useState<string | null>(null);
const latestScreenIntelligenceStatus = useCallback(
(): AccessibilityStatus | null => getCoreStateSnapshot().snapshot.runtime.screenIntelligence,
[]
);
const refreshStatus = useCallback(async () => {
setIsLoading(true);
setLastError(null);
try {
await refreshCoreState();
return latestScreenIntelligenceStatus();
} catch (error) {
setLastError(extractError(error, 'Failed to fetch accessibility status'));
return null;
} finally {
setIsLoading(false);
}
}, [latestScreenIntelligenceStatus, refreshCoreState]);
const refreshVision = useCallback(
async (limit = visionLimit) => {
setIsLoadingVision(true);
try {
const summaries = await fetchScreenIntelligenceVisionRecent(limit);
setRecentVisionSummaries(summaries);
return summaries;
} catch (error) {
setLastError(extractError(error, 'Failed to fetch accessibility vision summaries'));
return [];
} finally {
setIsLoadingVision(false);
}
},
[visionLimit]
);
const requestPermission = useCallback(
async (permission: AccessibilityPermissionKind) => {
setIsRequestingPermissions(true);
setLastError(null);
setLastRestartSummary(null);
try {
await requestScreenIntelligencePermission(permission);
await refreshCoreState();
return latestScreenIntelligenceStatus();
} catch (error) {
setLastError(extractError(error, 'Failed to request accessibility permission'));
return null;
} finally {
setIsRequestingPermissions(false);
}
},
[latestScreenIntelligenceStatus, refreshCoreState]
);
const refreshPermissionsWithRestart = useCallback(async () => {
setIsRestartingCore(true);
setLastError(null);
setLastRestartSummary(null);
try {
const result = await refreshScreenIntelligencePermissionsWithRestart(status);
setLastRestartSummary(result.restartSummary);
await refreshCoreState();
return latestScreenIntelligenceStatus() ?? result.status;
} catch (error) {
setLastError(extractError(error, 'Failed to restart core and refresh permissions'));
return null;
} finally {
setIsRestartingCore(false);
}
}, [latestScreenIntelligenceStatus, refreshCoreState, status]);
const startSession = useCallback(
async (params: AccessibilityStartSessionParams) => {
setIsStartingSession(true);
setLastError(null);
try {
await startScreenIntelligenceSession(params);
await refreshCoreState();
return latestScreenIntelligenceStatus();
} catch (error) {
setLastError(extractError(error, 'Failed to start accessibility session'));
return null;
} finally {
setIsStartingSession(false);
}
},
[latestScreenIntelligenceStatus, refreshCoreState]
);
const stopSession = useCallback(
async (reason?: string) => {
setIsStoppingSession(true);
setLastError(null);
try {
await stopScreenIntelligenceSession(reason);
await refreshCoreState();
return latestScreenIntelligenceStatus();
} catch (error) {
setLastError(extractError(error, 'Failed to stop accessibility session'));
return null;
} finally {
setIsStoppingSession(false);
}
},
[latestScreenIntelligenceStatus, refreshCoreState]
);
const flushVision = useCallback(async () => {
setIsFlushingVision(true);
try {
const summary = await flushScreenIntelligenceVision();
if (summary) {
setRecentVisionSummaries(current => [summary, ...current].slice(0, 30));
}
} catch (error) {
setLastError(extractError(error, 'Failed to flush accessibility vision'));
} finally {
setIsFlushingVision(false);
}
}, []);
const runCaptureTest = useCallback(async () => {
setIsCaptureTestRunning(true);
setCaptureTestResult(null);
setLastError(null);
try {
const result = await runScreenIntelligenceCaptureTest();
setCaptureTestResult(result);
} catch (error) {
setLastError(extractError(error, 'Failed to run capture test'));
} finally {
setIsCaptureTestRunning(false);
}
}, []);
useEffect(() => {
if (loadVision) {
void refreshVision(visionLimit);
}
}, [loadVision, refreshVision, visionLimit]);
useEffect(() => {
if (!loadVision) {
return;
}
const intervalId = window.setInterval(() => {
void refreshVision(visionLimit);
}, pollMs);
return () => window.clearInterval(intervalId);
}, [loadVision, pollMs, refreshVision, visionLimit]);
return {
status,
lastRestartSummary,
recentVisionSummaries,
captureTestResult,
isCaptureTestRunning,
isLoading,
isRequestingPermissions,
isRestartingCore,
isStartingSession,
isStoppingSession,
isLoadingVision,
isFlushingVision,
lastError,
refreshStatus,
requestPermission,
refreshPermissionsWithRestart,
startSession,
stopSession,
refreshVision,
flushVision,
runCaptureTest,
clearError: () => setLastError(null),
};
}
+21
View File
@@ -0,0 +1,21 @@
import type { Tunnel } from '../../services/api/tunnelsApi';
export type { Tunnel };
export interface TunnelRegistration {
tunnel_uuid: string;
target_kind?: string;
skill_id: string;
tunnel_name: string | null;
backend_tunnel_id: string | null;
}
export interface WebhookActivityEntry {
correlation_id: string;
tunnel_name: string;
method: string;
path: string;
status_code: number | null;
skill_id: string | null;
timestamp: number;
}
+63 -54
View File
@@ -8,18 +8,12 @@ import { useCallback, useEffect } from 'react';
import {
resetConnectionAttempts,
selectDaemonComponents,
selectDaemonConnectionAttempts,
selectDaemonHealthSnapshot,
selectDaemonLastHealthUpdate,
selectDaemonStatus,
selectIsDaemonAutoStartEnabled,
selectIsDaemonRecovering,
setAutoStartEnabled,
setDaemonStatus,
setIsRecovering,
} from '../store/daemonSlice';
import { useAppDispatch, useAppSelector } from '../store/hooks';
useDaemonUserState,
} from '../features/daemon/store';
import { daemonHealthService } from '../services/daemonHealthService';
import {
type CommandResponse,
openhumanAgentServerStatus,
@@ -30,30 +24,21 @@ import {
} from '../utils/tauriCommands';
export const useDaemonHealth = (userId?: string) => {
const dispatch = useAppDispatch();
// Selectors
const status = useAppSelector(state => selectDaemonStatus(state, userId));
const components = useAppSelector(state => selectDaemonComponents(state, userId));
const healthSnapshot = useAppSelector(state => selectDaemonHealthSnapshot(state, userId));
const lastUpdate = useAppSelector(state => selectDaemonLastHealthUpdate(state, userId));
const isAutoStartEnabled = useAppSelector(state => selectIsDaemonAutoStartEnabled(state, userId));
const connectionAttempts = useAppSelector(state => selectDaemonConnectionAttempts(state, userId));
const isRecovering = useAppSelector(state => selectIsDaemonRecovering(state, userId));
const daemonState = useDaemonUserState(userId);
const uid = userId || '__pending__';
const probeAgentStatus = useCallback(async (): Promise<boolean> => {
try {
const result = await openhumanAgentServerStatus();
const running = !!result?.result?.running;
dispatch(setDaemonStatus({ userId: uid, status: running ? 'running' : 'disconnected' }));
setDaemonStatus(uid, running ? 'running' : 'disconnected');
return running;
} catch (error) {
console.error('[useDaemonHealth] Failed to probe agent status:', error);
dispatch(setDaemonStatus({ userId: uid, status: 'disconnected' }));
setDaemonStatus(uid, 'disconnected');
return false;
}
}, [dispatch, uid]);
}, [uid]);
const waitForAgentStatus = useCallback(
async (targetRunning: boolean, timeoutMs = 10000): Promise<boolean> => {
@@ -73,28 +58,28 @@ export const useDaemonHealth = (userId?: string) => {
// Action creators
const startDaemon = useCallback(async (): Promise<CommandResponse<ServiceStatus> | null> => {
try {
dispatch(setDaemonStatus({ userId: uid, status: 'starting' }));
setDaemonStatus(uid, 'starting');
const result = await openhumanServiceStart();
const running = await waitForAgentStatus(true);
if (running) {
if (result?.result) {
(result.result as { state?: string }).state = 'Running';
}
dispatch(resetConnectionAttempts({ userId: uid }));
resetConnectionAttempts(uid);
} else {
dispatch(setDaemonStatus({ userId: uid, status: 'error' }));
setDaemonStatus(uid, 'error');
}
return result;
} catch (error) {
console.error('[useDaemonHealth] Failed to start daemon:', error);
dispatch(setDaemonStatus({ userId: uid, status: 'error' }));
setDaemonStatus(uid, 'error');
return null;
}
}, [dispatch, uid, waitForAgentStatus]);
}, [uid, waitForAgentStatus]);
const stopDaemon = useCallback(async (): Promise<CommandResponse<ServiceStatus> | null> => {
try {
dispatch(setDaemonStatus({ userId: uid, status: 'starting' }));
setDaemonStatus(uid, 'stopping');
const result = await openhumanServiceStop();
await waitForAgentStatus(false, 7000);
return result;
@@ -102,12 +87,12 @@ export const useDaemonHealth = (userId?: string) => {
console.error('[useDaemonHealth] Failed to stop daemon:', error);
return null;
}
}, [dispatch, uid, waitForAgentStatus]);
}, [uid, waitForAgentStatus]);
const restartDaemon = useCallback(async (): Promise<boolean> => {
try {
dispatch(setIsRecovering({ userId: uid, isRecovering: true }));
dispatch(setDaemonStatus({ userId: uid, status: 'starting' }));
setIsRecovering(uid, true);
setDaemonStatus(uid, 'starting');
// Stop first
await openhumanServiceStop();
@@ -121,20 +106,20 @@ export const useDaemonHealth = (userId?: string) => {
const success = await waitForAgentStatus(true, 12000);
if (success) {
dispatch(resetConnectionAttempts({ userId: uid }));
resetConnectionAttempts(uid);
} else {
dispatch(setDaemonStatus({ userId: uid, status: 'error' }));
setDaemonStatus(uid, 'error');
}
dispatch(setIsRecovering({ userId: uid, isRecovering: false }));
setIsRecovering(uid, false);
return success;
} catch (error) {
console.error('[useDaemonHealth] Failed to restart daemon:', error);
dispatch(setIsRecovering({ userId: uid, isRecovering: false }));
dispatch(setDaemonStatus({ userId: uid, status: 'error' }));
setIsRecovering(uid, false);
setDaemonStatus(uid, 'error');
return false;
}
}, [dispatch, uid, waitForAgentStatus]);
}, [uid, waitForAgentStatus]);
const checkDaemonStatus =
useCallback(async (): Promise<CommandResponse<ServiceStatus> | null> => {
@@ -152,37 +137,61 @@ export const useDaemonHealth = (userId?: string) => {
const setAutoStart = useCallback(
(enabled: boolean) => {
dispatch(setAutoStartEnabled({ userId: userId || '__pending__', enabled }));
setAutoStartEnabled(userId || '__pending__', enabled);
},
[dispatch, userId]
[userId]
);
// Derived state
const isHealthy = status === 'running';
const hasErrors = status === 'error';
const isConnected = status !== 'disconnected';
const isStarting = status === 'starting';
const isHealthy = daemonState.status === 'running';
const hasErrors = daemonState.status === 'error';
const isConnected = daemonState.status !== 'disconnected';
const isStarting = daemonState.status === 'starting';
const componentCount = Object.keys(components).length;
const healthyComponentCount = Object.values(components).filter(c => c.status === 'ok').length;
const errorComponentCount = Object.values(components).filter(c => c.status === 'error').length;
const componentCount = Object.keys(daemonState.components).length;
const healthyComponentCount = Object.values(daemonState.components).filter(
c => c.status === 'ok'
).length;
const errorComponentCount = Object.values(daemonState.components).filter(
c => c.status === 'error'
).length;
// Get uptime in human readable format
const uptimeText = healthSnapshot ? formatUptime(healthSnapshot.uptime_seconds) : 'Unknown';
const uptimeText = daemonState.healthSnapshot
? formatUptime(daemonState.healthSnapshot.uptime_seconds)
: 'Unknown';
useEffect(() => {
void probeAgentStatus();
}, [probeAgentStatus]);
useEffect(() => {
let cleanup: (() => void) | null = null;
let cancelled = false;
void daemonHealthService.setupHealthListener().then(result => {
if (cancelled) {
result?.();
} else {
cleanup = result;
}
});
return () => {
cancelled = true;
cleanup?.();
};
}, []);
return {
// State
status,
components,
healthSnapshot,
lastUpdate,
isAutoStartEnabled,
connectionAttempts,
isRecovering,
status: daemonState.status,
components: daemonState.components,
healthSnapshot: daemonState.healthSnapshot,
lastUpdate: daemonState.lastHealthUpdate,
isAutoStartEnabled: daemonState.autoStartEnabled,
connectionAttempts: daemonState.connectionAttempts,
isRecovering: daemonState.isRecovering,
// Derived state
isHealthy,
+22 -42
View File
@@ -12,13 +12,9 @@ import { useCallback, useEffect, useRef } from 'react';
import {
incrementConnectionAttempts,
resetConnectionAttempts,
selectDaemonConnectionAttempts,
selectDaemonStatus,
selectIsDaemonAutoStartEnabled,
selectIsDaemonRecovering,
setIsRecovering,
} from '../store/daemonSlice';
import { useAppDispatch, useAppSelector } from '../store/hooks';
useDaemonUserState,
} from '../features/daemon/store';
import { isTauri } from '../utils/tauriCommands';
import { useDaemonHealth } from './useDaemonHealth';
@@ -29,14 +25,14 @@ const MAX_RETRY_DELAY_MS = 30000; // 30 seconds
const AUTO_START_DELAY_MS = 3000; // 3 seconds after app start
export const useDaemonLifecycle = (userId?: string) => {
const dispatch = useAppDispatch();
const daemonHealth = useDaemonHealth(userId);
const daemonState = useDaemonUserState(userId);
// Selectors
const status = useAppSelector(state => selectDaemonStatus(state, userId));
const isAutoStartEnabled = useAppSelector(state => selectIsDaemonAutoStartEnabled(state, userId));
const connectionAttempts = useAppSelector(state => selectDaemonConnectionAttempts(state, userId));
const isRecovering = useAppSelector(state => selectIsDaemonRecovering(state, userId));
const status = daemonState.status;
const isAutoStartEnabled = daemonState.autoStartEnabled;
const connectionAttempts = daemonState.connectionAttempts;
const isRecovering = daemonState.isRecovering;
const uid = userId || '__pending__';
// Refs for cleanup
const autoStartTimeoutRef = useRef<ReturnType<typeof setTimeout> | null>(null);
@@ -60,32 +56,24 @@ export const useDaemonLifecycle = (userId?: string) => {
console.log('[DaemonLifecycle] Attempting auto-start of daemon');
try {
dispatch(setIsRecovering({ userId: userId || '__pending__', isRecovering: true }));
setIsRecovering(uid, true);
const result = await daemonHealth.startDaemon();
if (result?.result && result.result.state === 'Running') {
console.log('[DaemonLifecycle] Auto-start successful');
dispatch(resetConnectionAttempts({ userId: userId || '__pending__' }));
resetConnectionAttempts(uid);
} else {
console.warn('[DaemonLifecycle] Auto-start failed:', result);
dispatch(incrementConnectionAttempts({ userId: userId || '__pending__' }));
incrementConnectionAttempts(uid);
}
} catch (error) {
console.error('[DaemonLifecycle] Auto-start error:', error);
dispatch(incrementConnectionAttempts({ userId: userId || '__pending__' }));
incrementConnectionAttempts(uid);
} finally {
dispatch(setIsRecovering({ userId: userId || '__pending__', isRecovering: false }));
setIsRecovering(uid, false);
}
}
}, [
isAutoStartEnabled,
status,
isRecovering,
connectionAttempts,
userId,
dispatch,
daemonHealth,
]);
}, [isAutoStartEnabled, status, isRecovering, connectionAttempts, uid, daemonHealth]);
// Retry connection with exponential backoff
const scheduleRetry = useCallback(() => {
@@ -118,14 +106,14 @@ export const useDaemonLifecycle = (userId?: string) => {
if (!isMountedRef.current) return;
try {
dispatch(setIsRecovering({ userId: userId || '__pending__', isRecovering: true }));
dispatch(incrementConnectionAttempts({ userId: userId || '__pending__' }));
setIsRecovering(uid, true);
incrementConnectionAttempts(uid);
const result = await daemonHealth.startDaemon();
if (result?.result && result.result.state === 'Running') {
console.log('[DaemonLifecycle] Retry successful');
dispatch(resetConnectionAttempts({ userId: userId || '__pending__' }));
resetConnectionAttempts(uid);
} else {
console.warn('[DaemonLifecycle] Retry failed:', result);
// Will trigger another retry via useEffect
@@ -134,18 +122,10 @@ export const useDaemonLifecycle = (userId?: string) => {
console.error('[DaemonLifecycle] Retry error:', error);
// Will trigger another retry via useEffect
} finally {
dispatch(setIsRecovering({ userId: userId || '__pending__', isRecovering: false }));
setIsRecovering(uid, false);
}
}, retryDelay);
}, [
connectionAttempts,
status,
isRecovering,
calculateRetryDelay,
userId,
dispatch,
daemonHealth,
]);
}, [connectionAttempts, status, isRecovering, calculateRetryDelay, uid, daemonHealth]);
// Handle visibility change (background/foreground)
const handleVisibilityChange = useCallback(() => {
@@ -231,7 +211,7 @@ export const useDaemonLifecycle = (userId?: string) => {
useEffect(() => {
if (status === 'running' && connectionAttempts > 0) {
console.log('[DaemonLifecycle] Daemon healthy - resetting connection attempts');
dispatch(resetConnectionAttempts({ userId: userId || '__pending__' }));
resetConnectionAttempts(uid);
// Clear retry timeout if running
if (retryTimeoutRef.current) {
@@ -239,7 +219,7 @@ export const useDaemonLifecycle = (userId?: string) => {
retryTimeoutRef.current = null;
}
}
}, [status, connectionAttempts, userId, dispatch]);
}, [status, connectionAttempts, uid]);
// Return lifecycle state and controls
return {
@@ -252,7 +232,7 @@ export const useDaemonLifecycle = (userId?: string) => {
// Actions
attemptAutoStart,
resetRetries: () => {
dispatch(resetConnectionAttempts({ userId: userId || '__pending__' }));
resetConnectionAttempts(uid);
if (retryTimeoutRef.current) {
clearTimeout(retryTimeoutRef.current);
retryTimeoutRef.current = null;
+20 -5
View File
@@ -1,10 +1,12 @@
import { useCallback, useEffect, useState } from 'react';
import { callCoreRpc } from '../services/coreRpcClient';
import type { AIStatus } from '../store/aiSlice';
import { useAppSelector } from '../store/hooks';
import { aiListMemoryFiles, type GraphRelation, memoryGraphQuery } from '../utils/tauriCommands';
export type AIStatus = 'idle' | 'initializing' | 'ready' | 'error';
const POLL_MS = 5000;
interface SessionEntry {
sessionId: string;
updatedAt: number;
@@ -46,7 +48,7 @@ function entityCountsFromRelations(relations: GraphRelation[]): Record<string, n
}
export function useIntelligenceStats(): IntelligenceStats {
const aiStatus = useAppSelector(state => state.ai.status);
const [aiStatus, setAiStatus] = useState<AIStatus>('idle');
const [sessions, setSessions] = useState<SessionStats | null>(null);
const [memoryFiles, setMemoryFiles] = useState<number | null>(null);
const [entities, setEntities] = useState<Record<string, number> | null>(null);
@@ -54,7 +56,9 @@ export function useIntelligenceStats(): IntelligenceStats {
const [isLoading, setIsLoading] = useState(true);
const fetchStats = useCallback(async () => {
setAiStatus('initializing');
setIsLoading(true);
let hasSuccess = false;
// Fetch local stats (Tauri invoke)
try {
@@ -68,6 +72,7 @@ export function useIntelligenceStats(): IntelligenceStats {
compactions: entries.reduce((sum, e) => sum + (e.compactionCount || 0), 0),
memoryFlushes: entries.filter(e => e.memoryFlushAt).length,
});
hasSuccess = true;
} catch {
setSessions(null);
}
@@ -75,6 +80,7 @@ export function useIntelligenceStats(): IntelligenceStats {
try {
const files = await aiListMemoryFiles('memory');
setMemoryFiles(files.length);
hasSuccess = true;
} catch {
setMemoryFiles(null);
}
@@ -90,17 +96,26 @@ export function useIntelligenceStats(): IntelligenceStats {
setEntities(null);
setEntityError(false);
}
hasSuccess = true;
} catch {
setEntities(null);
setEntityError(true);
}
setAiStatus(hasSuccess ? 'ready' : 'error');
setIsLoading(false);
}, []);
useEffect(() => {
fetchStats();
}, [fetchStats, aiStatus]);
void fetchStats();
const intervalId = window.setInterval(() => {
void fetchStats();
}, POLL_MS);
return () => {
window.clearInterval(intervalId);
};
}, [fetchStats]);
return { sessions, memoryFiles, entities, entityError, aiStatus, isLoading, refetch: fetchStats };
}
+8 -10
View File
@@ -1,7 +1,6 @@
import { useEffect, useMemo } from 'react';
import { useMemo } from 'react';
import { fetchAccessibilityVisionRecent } from '../store/accessibilitySlice';
import { useAppDispatch, useAppSelector } from '../store/hooks';
import { useScreenIntelligenceState } from '../features/screen-intelligence/useScreenIntelligenceState';
import type { ActionableItem, ActionableItemPriority } from '../types/intelligence';
function confidenceToPriority(confidence: number): ActionableItemPriority {
@@ -11,12 +10,11 @@ function confidenceToPriority(confidence: number): ActionableItemPriority {
}
export function useScreenIntelligenceItems() {
const dispatch = useAppDispatch();
const { recentVisionSummaries, isLoadingVision } = useAppSelector(state => state.accessibility);
useEffect(() => {
void dispatch(fetchAccessibilityVisionRecent(20));
}, [dispatch]);
const { recentVisionSummaries, isLoadingVision, refreshVision } = useScreenIntelligenceState({
loadVision: true,
visionLimit: 20,
pollMs: 2000,
});
const items: ActionableItem[] = useMemo(() => {
return recentVisionSummaries.map(summary => ({
@@ -33,5 +31,5 @@ export function useScreenIntelligenceItems() {
}));
}, [recentVisionSummaries]);
return { items, loading: isLoadingVision };
return { items, loading: isLoadingVision, refresh: () => refreshVision(20) };
}
+41 -70
View File
@@ -1,20 +1,10 @@
import debug from 'debug';
import { useCallback, useEffect, useRef, useState } from 'react';
import type { Tunnel, TunnelRegistration, WebhookActivityEntry } from '../features/webhooks/types';
import { useCoreState } from '../providers/CoreStateProvider';
import { tunnelsApi } from '../services/api/tunnelsApi';
import { getCoreHttpBaseUrl } from '../services/coreRpcClient';
import { useAppDispatch, useAppSelector } from '../store/hooks';
import {
addActivity,
addTunnel,
removeTunnel,
setError,
setLoading,
setRegistrations,
setTunnels,
type WebhookActivityEntry,
} from '../store/webhooksSlice';
import {
openhumanWebhooksListLogs,
openhumanWebhooksListRegistrations,
@@ -47,11 +37,12 @@ function logToActivity(entry: WebhookDebugLogEntry): WebhookActivityEntry {
*/
export function useWebhooks() {
const { snapshot } = useCoreState();
const dispatch = useAppDispatch();
const { tunnels, registrations, activity, loading, error } = useAppSelector(
state => state.webhooks
);
const token = snapshot.sessionToken;
const [tunnels, setTunnels] = useState<Tunnel[]>([]);
const [registrations, setRegistrations] = useState<TunnelRegistration[]>([]);
const [activity, setActivity] = useState<WebhookActivityEntry[]>([]);
const [loading, setLoading] = useState(false);
const [error, setError] = useState<string | null>(null);
const [coreConnected, setCoreConnected] = useState(false);
const eventSourceRef = useRef<EventSource | null>(null);
@@ -62,13 +53,11 @@ export function useWebhooks() {
openhumanWebhooksListRegistrations(),
openhumanWebhooksListLogs(100),
]);
dispatch(setRegistrations(regsResponse.result.result.registrations));
setRegistrations(regsResponse.result.result.registrations);
// Seed activity from debug logs
const logs = logsResponse.result.result.logs;
for (const entry of logs.reverse()) {
dispatch(addActivity(logToActivity(entry)));
}
setActivity(logs.reverse().map(logToActivity));
log(
'Loaded %d registrations, %d logs from core',
regsResponse.result.result.registrations.length,
@@ -80,21 +69,24 @@ export function useWebhooks() {
err instanceof Error ? err.message : err
);
}
}, [dispatch]);
}, []);
// ── Fetch tunnels from backend API ───────────────────────────────────────
const fetchTunnels = useCallback(async () => {
dispatch(setLoading(true));
setLoading(true);
setError(null);
try {
const data = await tunnelsApi.getTunnels();
dispatch(setTunnels(data));
setTunnels(data);
log('Fetched %d tunnels', data.length);
} catch (err) {
const msg = err instanceof Error ? err.message : 'Failed to fetch tunnels';
dispatch(setError(msg));
setError(msg);
log('Error fetching tunnels: %s', msg);
} finally {
setLoading(false);
}
}, [dispatch]);
}, []);
// ── Subscribe to SSE for real-time webhook events ────────────────────────
useEffect(() => {
@@ -147,36 +139,18 @@ export function useWebhooks() {
}, [token, fetchTunnels, loadCoreData]);
// ── CRUD actions ─────────────────────────────────────────────────────────
const createTunnel = useCallback(
async (name: string, description?: string) => {
try {
const tunnel = await tunnelsApi.createTunnel({ name, description });
dispatch(addTunnel(tunnel));
log('Created tunnel: %s (%s)', tunnel.name, tunnel.uuid);
return tunnel;
} catch (err) {
const msg = err instanceof Error ? err.message : 'Failed to create tunnel';
dispatch(setError(msg));
throw err;
}
},
[dispatch]
);
const createTunnel = useCallback(async (name: string, description?: string) => {
const tunnel = await tunnelsApi.createTunnel({ name, description });
setTunnels(current => [...current, tunnel]);
log('Created tunnel: %s (%s)', tunnel.name, tunnel.uuid);
return tunnel;
}, []);
const deleteTunnel = useCallback(
async (id: string) => {
try {
await tunnelsApi.deleteTunnel(id);
dispatch(removeTunnel(id));
log('Deleted tunnel: %s', id);
} catch (err) {
const msg = err instanceof Error ? err.message : 'Failed to delete tunnel';
dispatch(setError(msg));
throw err;
}
},
[dispatch]
);
const deleteTunnel = useCallback(async (id: string) => {
await tunnelsApi.deleteTunnel(id);
setTunnels(current => current.filter(tunnel => tunnel.id !== id));
log('Deleted tunnel: %s', id);
}, []);
const refreshTunnels = useCallback(async () => {
await fetchTunnels();
@@ -192,31 +166,28 @@ export function useWebhooks() {
tunnelName,
backendTunnelId
);
dispatch(setRegistrations(response.result.result.registrations));
setRegistrations(response.result.result.registrations);
log('Registered echo for tunnel %s', tunnelUuid);
} catch (err) {
const msg = err instanceof Error ? err.message : 'Failed to register echo';
dispatch(setError(msg));
setError(msg);
throw err;
}
},
[dispatch]
[]
);
const unregisterEcho = useCallback(
async (tunnelUuid: string) => {
try {
const response = await openhumanWebhooksUnregisterEcho(tunnelUuid);
dispatch(setRegistrations(response.result.result.registrations));
log('Unregistered echo for tunnel %s', tunnelUuid);
} catch (err) {
const msg = err instanceof Error ? err.message : 'Failed to unregister echo';
dispatch(setError(msg));
throw err;
}
},
[dispatch]
);
const unregisterEcho = useCallback(async (tunnelUuid: string) => {
try {
const response = await openhumanWebhooksUnregisterEcho(tunnelUuid);
setRegistrations(response.result.result.registrations);
log('Unregistered echo for tunnel %s', tunnelUuid);
} catch (err) {
const msg = err instanceof Error ? err.message : 'Failed to unregister echo';
setError(msg);
throw err;
}
}, []);
return {
tunnels,
+13
View File
@@ -1,5 +1,9 @@
import type { User } from '../../types/api';
import type { TeamInvite, TeamMember, TeamWithRole } from '../../types/team';
import type { AccessibilityStatus } from '../../utils/tauriCommands/accessibility';
import type { AutocompleteStatus } from '../../utils/tauriCommands/autocomplete';
import type { ServiceStatus } from '../../utils/tauriCommands/hardware';
import type { LocalAiStatus } from '../../utils/tauriCommands/localAi';
export interface CoreOnboardingTasks {
accessibilityPermissionGranted: boolean;
@@ -16,6 +20,13 @@ export interface CoreLocalState {
onboardingTasks: CoreOnboardingTasks | null;
}
export interface CoreRuntimeSnapshot {
screenIntelligence: AccessibilityStatus | null;
localAi: LocalAiStatus | null;
autocomplete: AutocompleteStatus | null;
service: ServiceStatus | null;
}
export interface CoreAppSnapshot {
auth: {
isAuthenticated: boolean;
@@ -28,6 +39,7 @@ export interface CoreAppSnapshot {
onboardingCompleted: boolean;
analyticsEnabled: boolean;
localState: CoreLocalState;
runtime: CoreRuntimeSnapshot;
}
export interface CoreState {
@@ -46,6 +58,7 @@ const emptySnapshot: CoreAppSnapshot = {
onboardingCompleted: false,
analyticsEnabled: false,
localState: { encryptionKey: null, primaryWalletAddress: null, onboardingTasks: null },
runtime: { screenIntelligence: null, localAi: null, autocomplete: null, service: null },
};
let currentState: CoreState = {
+66 -11
View File
@@ -1,10 +1,14 @@
import { useEffect, useState } from 'react';
import debugFactory from 'debug';
import { useEffect, useRef, useState } from 'react';
import { useUser } from '../hooks/useUser';
import { useAppDispatch, useAppSelector } from '../store/hooks';
import { clearRedeemStatus, fetchInviteCodes, redeemCode } from '../store/inviteSlice';
import { inviteApi } from '../services/api/inviteApi';
import type { InviteCode } from '../types/invite';
const log = debugFactory('invites');
type RedeemStatus = 'idle' | 'loading' | 'success' | 'error';
const CodeRow = ({ invite }: { invite: InviteCode }) => {
const [copied, setCopied] = useState(false);
const claimed = invite.currentUses >= invite.maxUses;
@@ -70,26 +74,75 @@ const CodeRow = ({ invite }: { invite: InviteCode }) => {
};
const Invites = () => {
const dispatch = useAppDispatch();
const { user, refetch: refetchUser } = useUser();
const { codes, isLoading, redeemStatus, redeemError } = useAppSelector(state => state.invite);
const [codes, setCodes] = useState<InviteCode[]>([]);
const [isLoading, setIsLoading] = useState(false);
const [redeemStatus, setRedeemStatus] = useState<RedeemStatus>('idle');
const [redeemError, setRedeemError] = useState<string | null>(null);
const [redeemInput, setRedeemInput] = useState('');
const redeemTimeoutRef = useRef<number | null>(null);
const loadRequestIdRef = useRef(0);
const hasBeenInvited = !!user?.referral?.invitedBy;
const [loadError, setLoadError] = useState<string | null>(null);
const loadInviteCodes = async () => {
const requestId = ++loadRequestIdRef.current;
setIsLoading(true);
setLoadError(null);
try {
const data = await inviteApi.getMyInviteCodes();
if (requestId !== loadRequestIdRef.current) return;
setCodes(data);
} catch (error) {
if (requestId !== loadRequestIdRef.current) return;
log('loadInviteCodes failed requestId=%d error=%O', requestId, error);
setLoadError(error instanceof Error ? error.message : 'Failed to load invite codes');
} finally {
if (requestId === loadRequestIdRef.current) {
setIsLoading(false);
}
}
};
useEffect(() => {
dispatch(fetchInviteCodes());
}, [dispatch]);
void loadInviteCodes();
return () => {
// Invalidate any in-flight loadInviteCodes requests
loadRequestIdRef.current += 1;
if (redeemTimeoutRef.current) {
clearTimeout(redeemTimeoutRef.current);
redeemTimeoutRef.current = null;
}
};
}, []);
const handleRedeem = async () => {
const trimmed = redeemInput.trim();
if (!trimmed) return;
const result = await dispatch(redeemCode(trimmed));
if (redeemCode.fulfilled.match(result)) {
setRedeemStatus('loading');
setRedeemError(null);
try {
await inviteApi.redeemInviteCode(trimmed);
await loadInviteCodes();
setRedeemInput('');
refetchUser();
setTimeout(() => dispatch(clearRedeemStatus()), 3000);
setRedeemStatus('success');
if (redeemTimeoutRef.current) {
clearTimeout(redeemTimeoutRef.current);
}
redeemTimeoutRef.current = window.setTimeout(() => {
redeemTimeoutRef.current = null;
setRedeemStatus('idle');
setRedeemError(null);
}, 3000);
// Refresh user in background — don't let failure override the successful redeem
refetchUser().catch(() => {});
} catch (error) {
setRedeemStatus('error');
setRedeemError(error instanceof Error ? error.message : 'Failed to redeem invite code');
}
};
@@ -140,6 +193,8 @@ const Invites = () => {
</p>
</div>
{loadError && <p className="text-coral-500 text-xs text-center py-2">{loadError}</p>}
{isLoading ? (
<div className="space-y-3">
{Array.from({ length: 5 }).map((_, i) => (
@@ -1,11 +1,6 @@
import { useEffect, useState } from 'react';
import {
fetchAccessibilityStatus,
refreshPermissionsWithRestart,
requestAccessibilityPermission,
} from '../../../store/accessibilitySlice';
import { useAppDispatch, useAppSelector } from '../../../store/hooks';
import { useScreenIntelligenceState } from '../../../features/screen-intelligence/useScreenIntelligenceState';
import OnboardingNextButton from '../components/OnboardingNextButton';
interface ScreenPermissionsStepProps {
@@ -14,15 +9,17 @@ interface ScreenPermissionsStepProps {
}
const ScreenPermissionsStep = ({ onNext, onBack: _onBack }: ScreenPermissionsStepProps) => {
const dispatch = useAppDispatch();
const { status, isLoading, isRequestingPermissions, isRestartingCore, lastError } =
useAppSelector(state => state.accessibility);
const {
status,
isLoading,
isRequestingPermissions,
isRestartingCore,
lastError,
requestPermission,
refreshPermissionsWithRestart,
} = useScreenIntelligenceState({ pollMs: 2000 });
const [shouldAutoRefreshOnReturn, setShouldAutoRefreshOnReturn] = useState(false);
useEffect(() => {
void dispatch(fetchAccessibilityStatus());
}, [dispatch]);
const accessibilityPermission = status?.permissions.accessibility ?? 'unknown';
const isGranted = accessibilityPermission === 'granted';
@@ -42,7 +39,7 @@ const ScreenPermissionsStep = ({ onNext, onBack: _onBack }: ScreenPermissionsSte
}
setShouldAutoRefreshOnReturn(false);
void dispatch(refreshPermissionsWithRestart());
void refreshPermissionsWithRestart();
};
window.addEventListener('focus', refreshAfterReturn);
@@ -52,11 +49,17 @@ const ScreenPermissionsStep = ({ onNext, onBack: _onBack }: ScreenPermissionsSte
window.removeEventListener('focus', refreshAfterReturn);
document.removeEventListener('visibilitychange', refreshAfterReturn);
};
}, [dispatch, isGranted, isLoading, isRestartingCore, shouldAutoRefreshOnReturn]);
}, [
isGranted,
isLoading,
isRestartingCore,
refreshPermissionsWithRestart,
shouldAutoRefreshOnReturn,
]);
const handleRequestPermissions = () => {
setShouldAutoRefreshOnReturn(true);
void dispatch(requestAccessibilityPermission('accessibility'));
void requestPermission('accessibility');
};
return (
@@ -102,7 +105,7 @@ const ScreenPermissionsStep = ({ onNext, onBack: _onBack }: ScreenPermissionsSte
<div className="flex gap-2">
<button
type="button"
onClick={() => void dispatch(refreshPermissionsWithRestart())}
onClick={() => void refreshPermissionsWithRestart()}
disabled={isRestartingCore || isLoading}
className="flex-1 py-2 text-sm font-medium rounded-xl border border-stone-200 hover:border-stone-400 text-stone-600 hover:text-stone-900 opacity-70 hover:opacity-100 transition-all disabled:opacity-40">
{isRestartingCore ? 'Restarting...' : 'Restart & Refresh'}
@@ -1,126 +1,107 @@
import { configureStore } from '@reduxjs/toolkit';
import { fireEvent, render, screen, waitFor } from '@testing-library/react';
import type { PropsWithChildren } from 'react';
import { Provider } from 'react-redux';
import { MemoryRouter } from 'react-router-dom';
import { beforeEach, describe, expect, it, vi } from 'vitest';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import accessibilityReducer from '../../../../store/accessibilitySlice';
import authReducer from '../../../../store/authSlice';
import socketReducer from '../../../../store/socketSlice';
import teamReducer from '../../../../store/teamSlice';
import userReducer from '../../../../store/userSlice';
import {
type AccessibilityStatus,
openhumanAccessibilityRequestPermission,
openhumanAccessibilityStatus,
restartCoreProcess,
} from '../../../../utils/tauriCommands';
type ScreenIntelligenceState,
useScreenIntelligenceState,
} from '../../../../features/screen-intelligence/useScreenIntelligenceState';
import ScreenPermissionsStep from '../ScreenPermissionsStep';
vi.mock('../../../../utils/tauriCommands', async importOriginal => {
const actual = await importOriginal<typeof import('../../../../utils/tauriCommands')>();
return {
...actual,
openhumanAccessibilityRequestPermission: vi.fn(),
openhumanAccessibilityStatus: vi.fn(),
restartCoreProcess: vi.fn(),
};
});
vi.mock('../../../../features/screen-intelligence/useScreenIntelligenceState', () => ({
useScreenIntelligenceState: vi.fn(),
}));
const deniedStatus: AccessibilityStatus = {
platform_supported: true,
permissions: {
screen_recording: 'unknown',
accessibility: 'denied',
input_monitoring: 'unknown',
},
features: { screen_monitoring: true, device_control: true, predictive_input: true },
session: {
active: false,
started_at_ms: null,
expires_at_ms: null,
remaining_ms: null,
ttl_secs: 300,
panic_hotkey: 'Cmd+Shift+.',
stop_reason: null,
frames_in_memory: 0,
last_capture_at_ms: null,
last_context: null,
vision_enabled: true,
vision_state: 'idle',
vision_queue_depth: 0,
last_vision_at_ms: null,
last_vision_summary: null,
},
config: {
enabled: true,
capture_policy: 'hybrid',
policy_mode: 'all_except_blacklist',
baseline_fps: 1,
vision_enabled: true,
session_ttl_secs: 300,
panic_stop_hotkey: 'Cmd+Shift+.',
autocomplete_enabled: true,
use_vision_model: true,
keep_screenshots: false,
allowlist: [],
denylist: [],
},
denylist: [],
is_context_blocked: false,
permission_check_process_path: '/tmp/openhuman-core-x86_64-apple-darwin',
};
const grantedStatus: AccessibilityStatus = {
...deniedStatus,
permissions: { ...deniedStatus.permissions, accessibility: 'granted' },
};
const createStore = () =>
configureStore({
reducer: {
auth: authReducer,
socket: socketReducer,
user: userReducer,
team: teamReducer,
accessibility: accessibilityReducer,
const deniedState: ScreenIntelligenceState = {
status: {
platform_supported: true,
core_process: { pid: 4242, started_at_ms: 1712700000000 },
permissions: {
screen_recording: 'unknown',
accessibility: 'denied',
input_monitoring: 'unknown',
},
});
features: { screen_monitoring: true },
session: {
active: false,
started_at_ms: null,
expires_at_ms: null,
remaining_ms: null,
ttl_secs: 300,
panic_hotkey: 'Cmd+Shift+.',
stop_reason: null,
frames_in_memory: 0,
last_capture_at_ms: null,
last_context: null,
vision_enabled: true,
vision_state: 'idle',
vision_queue_depth: 0,
last_vision_at_ms: null,
last_vision_summary: null,
},
config: {
enabled: true,
capture_policy: 'hybrid',
policy_mode: 'all_except_blacklist',
baseline_fps: 1,
vision_enabled: true,
session_ttl_secs: 300,
panic_stop_hotkey: 'Cmd+Shift+.',
autocomplete_enabled: true,
use_vision_model: true,
keep_screenshots: false,
allowlist: [],
denylist: [],
},
denylist: [],
is_context_blocked: false,
permission_check_process_path: '/tmp/openhuman-core-x86_64-apple-darwin',
},
lastRestartSummary: null,
recentVisionSummaries: [],
captureTestResult: null,
isCaptureTestRunning: false,
isLoading: false,
isRequestingPermissions: false,
isRestartingCore: false,
isStartingSession: false,
isStoppingSession: false,
isLoadingVision: false,
isFlushingVision: false,
lastError: null,
refreshStatus: vi.fn().mockResolvedValue(null),
requestPermission: vi.fn().mockResolvedValue(null),
refreshPermissionsWithRestart: vi.fn().mockResolvedValue(null),
startSession: vi.fn().mockResolvedValue(null),
stopSession: vi.fn().mockResolvedValue(null),
refreshVision: vi.fn().mockResolvedValue([]),
flushVision: vi.fn().mockResolvedValue(undefined),
runCaptureTest: vi.fn().mockResolvedValue(undefined),
clearError: vi.fn(),
};
function renderStep() {
const store = createStore();
const onNext = vi.fn();
const Wrapper = ({ children }: PropsWithChildren) => (
<Provider store={store}>
<MemoryRouter>{children}</MemoryRouter>
</Provider>
);
render(<ScreenPermissionsStep onNext={onNext} />, { wrapper: Wrapper });
return { store, onNext };
}
const originalVisibilityDescriptor = Object.getOwnPropertyDescriptor(document, 'visibilityState');
describe('ScreenPermissionsStep', () => {
beforeEach(() => {
vi.clearAllMocks();
vi.mocked(restartCoreProcess).mockResolvedValue(undefined);
vi.mocked(openhumanAccessibilityRequestPermission).mockResolvedValue({
result: deniedStatus.permissions,
logs: [],
});
vi.mocked(openhumanAccessibilityStatus).mockResolvedValue({ result: deniedStatus, logs: [] });
vi.mocked(useScreenIntelligenceState).mockReturnValue(deniedState);
});
afterEach(() => {
if (originalVisibilityDescriptor) {
Object.defineProperty(document, 'visibilityState', originalVisibilityDescriptor);
}
});
it('auto-refreshes permissions after returning from System Settings', async () => {
vi.mocked(openhumanAccessibilityStatus)
.mockResolvedValueOnce({ result: deniedStatus, logs: [] })
.mockResolvedValueOnce({ result: deniedStatus, logs: [] })
.mockResolvedValueOnce({ result: grantedStatus, logs: [] });
const onNext = vi.fn();
renderStep();
render(
<MemoryRouter>
<ScreenPermissionsStep onNext={onNext} />
</MemoryRouter>
);
await screen.findByText('Screen & Accessibility Permissions');
@@ -132,11 +113,7 @@ describe('ScreenPermissionsStep', () => {
fireEvent(window, new Event('focus'));
await waitFor(() => {
expect(restartCoreProcess).toHaveBeenCalledTimes(1);
});
await waitFor(() => {
expect(screen.getByText('granted')).toBeInTheDocument();
expect(deniedState.refreshPermissionsWithRestart).toHaveBeenCalledTimes(1);
});
});
});
+84 -36
View File
@@ -1,3 +1,4 @@
import debugFactory from 'debug';
import {
createContext,
type ReactNode,
@@ -32,9 +33,27 @@ import {
logout as tauriLogout,
} from '../utils/tauriCommands';
const POLL_MS = 3000;
const log = debugFactory('core-state');
const POLL_MS = 2000;
const MAX_BOOTSTRAP_RETRIES = 5;
/** Extract only non-sensitive fields from an RPC/fetch error. */
function sanitizeError(error: unknown): { message?: string; code?: string; status?: number } {
if (error instanceof Error) {
return { message: error.message };
}
if (error && typeof error === 'object') {
const e = error as Record<string, unknown>;
return {
message: typeof e.message === 'string' ? e.message : undefined,
code: typeof e.code === 'string' ? e.code : undefined,
status: typeof e.status === 'number' ? e.status : undefined,
};
}
return { message: String(error) };
}
interface CoreStateContextValue extends CoreState {
refresh: () => Promise<void>;
refreshTeams: () => Promise<void>;
@@ -69,6 +88,12 @@ function normalizeSnapshot(
primaryWalletAddress: result.localState.primaryWalletAddress ?? null,
onboardingTasks: result.localState.onboardingTasks ?? null,
},
runtime: {
screenIntelligence: result.runtime?.screenIntelligence ?? null,
localAi: result.runtime?.localAi ?? null,
autocomplete: result.runtime?.autocomplete ?? null,
service: result.runtime?.service ?? null,
},
};
}
@@ -78,6 +103,7 @@ export default function CoreStateProvider({ children }: { children: ReactNode })
const teamsRequestIdRef = useRef(0);
const memoryTokenRef = useRef<string | null>(state.snapshot.sessionToken);
const bootstrapFailCountRef = useRef(0);
const refreshInFlightRef = useRef<Promise<void> | null>(null);
const commitState = useCallback((updater: (previous: CoreState) => CoreState) => {
setState(previous => {
@@ -87,7 +113,7 @@ export default function CoreStateProvider({ children }: { children: ReactNode })
});
}, []);
const refresh = useCallback(async () => {
const refreshCore = useCallback(async () => {
const requestId = ++snapshotRequestIdRef.current;
const snapshot = normalizeSnapshot(await fetchCoreAppSnapshot());
commitState(previous => {
@@ -128,6 +154,18 @@ export default function CoreStateProvider({ children }: { children: ReactNode })
}
}, [commitState]);
/** Serialized refresh — all callers share the same in-flight promise. */
const refresh = useCallback(async () => {
if (refreshInFlightRef.current) {
return refreshInFlightRef.current;
}
const promise = refreshCore().finally(() => {
refreshInFlightRef.current = null;
});
refreshInFlightRef.current = promise;
return promise;
}, [refreshCore]);
const refreshTeams = useCallback(async () => {
const requestId = ++teamsRequestIdRef.current;
const identityAtStart = snapshotIdentity(getCoreStateSnapshot().snapshot);
@@ -169,57 +207,65 @@ export default function CoreStateProvider({ children }: { children: ReactNode })
useEffect(() => {
let cancelled = false;
const load = async () => {
const doRefresh = async () => {
try {
await refresh();
bootstrapFailCountRef.current = 0;
const next = getCoreStateSnapshot();
if (next.snapshot.auth.isAuthenticated) {
await refreshTeams().catch(() => {});
}
} catch (error) {
if (!cancelled) {
bootstrapFailCountRef.current += 1;
const safe = sanitizeError(error);
log(
'refresh failed attempt=%d/%d error=%O',
bootstrapFailCountRef.current,
MAX_BOOTSTRAP_RETRIES,
safe
);
console.warn(
`[core-state] initial refresh failed (attempt ${bootstrapFailCountRef.current}/${MAX_BOOTSTRAP_RETRIES}):`,
error
`[core-state] poll failed (attempt ${bootstrapFailCountRef.current}/${MAX_BOOTSTRAP_RETRIES}):`,
safe
);
if (bootstrapFailCountRef.current >= MAX_BOOTSTRAP_RETRIES) {
commitState(previous => ({ ...previous, isBootstrapping: false }));
commitState(previous => {
if (previous.isBootstrapping) {
return { ...previous, isBootstrapping: false };
}
return previous;
});
}
}
}
};
void load();
const interval = window.setInterval(() => {
void (async () => {
try {
await refresh();
bootstrapFailCountRef.current = 0;
} catch (error) {
if (!cancelled) {
bootstrapFailCountRef.current += 1;
console.warn(
`[core-state] poll failed (attempt ${bootstrapFailCountRef.current}/${MAX_BOOTSTRAP_RETRIES}):`,
error
);
if (bootstrapFailCountRef.current >= MAX_BOOTSTRAP_RETRIES) {
commitState(previous => {
if (previous.isBootstrapping) {
return { ...previous, isBootstrapping: false };
}
return previous;
});
}
}
const load = async () => {
await doRefresh();
if (!cancelled) {
const next = getCoreStateSnapshot();
if (next.snapshot.auth.isAuthenticated) {
await refreshTeams().catch(err => {
log('refreshTeams failed during bootstrap: %O', sanitizeError(err));
});
}
})();
}, POLL_MS);
}
};
void load();
let timeoutId: number | null = null;
const scheduleNext = () => {
timeoutId = window.setTimeout(async () => {
await doRefresh();
if (!cancelled) {
scheduleNext();
}
}, POLL_MS);
};
scheduleNext();
return () => {
cancelled = true;
window.clearInterval(interval);
if (timeoutId !== null) {
window.clearTimeout(timeoutId);
}
};
}, [commitState, refresh, refreshTeams]);
@@ -264,7 +310,9 @@ export default function CoreStateProvider({ children }: { children: ReactNode })
console.warn('[core-state] memory client sync failed after session store:', error);
}
await refresh();
await refreshTeams().catch(() => {});
await refreshTeams().catch(err => {
log('refreshTeams failed after session store: %O', sanitizeError(err));
});
},
[refresh, refreshTeams]
);
-79
View File
@@ -1,79 +0,0 @@
import { useEffect } from 'react';
import { clearToken, setAuthBootstrapComplete, setToken } from '../store/authSlice';
import { useAppDispatch, useAppSelector } from '../store/hooks';
import {
getAuthState,
getSessionToken,
isTauri,
setOnboardingCompleted,
} from '../utils/tauriCommands';
const AUTH_BOOTSTRAP_TIMEOUT_MS = 5000;
async function withTimeout<T>(promise: Promise<T>, timeoutMs: number): Promise<T> {
let timeoutId: ReturnType<typeof setTimeout> | null = null;
const timeoutPromise = new Promise<never>((_, reject) => {
timeoutId = setTimeout(() => reject(new Error(`timeout after ${timeoutMs}ms`)), timeoutMs);
});
try {
return await Promise.race([promise, timeoutPromise]);
} finally {
if (timeoutId) clearTimeout(timeoutId);
}
}
/**
* UserProvider bootstraps auth token from core session state.
*/
const UserProvider = ({ children }: { children: React.ReactNode }) => {
const dispatch = useAppDispatch();
const token = useAppSelector(state => state.auth.token);
const isAuthBootstrapComplete = useAppSelector(state => state.auth.isAuthBootstrapComplete);
useEffect(() => {
if (isAuthBootstrapComplete) return;
let mounted = true;
void (async () => {
if (!isTauri()) {
if (mounted) dispatch(setAuthBootstrapComplete(true));
return;
}
try {
const [authState, sessionToken] = await withTimeout(
Promise.all([getAuthState(), getSessionToken()]),
AUTH_BOOTSTRAP_TIMEOUT_MS
);
if (!mounted) return;
if (authState.is_authenticated && sessionToken) {
if (sessionToken !== token) {
dispatch(setToken(sessionToken));
}
} else if (!authState.is_authenticated && token) {
await dispatch(clearToken());
try {
await setOnboardingCompleted(false);
} catch (err) {
console.warn('[auth] Failed to clear onboarding_completed in config:', err);
}
}
} catch (err) {
console.warn('[auth] Failed to restore session token from core RPC:', err);
} finally {
if (mounted) dispatch(setAuthBootstrapComplete(true));
}
})();
return () => {
mounted = false;
};
}, [token, dispatch, isAuthBootstrapComplete]);
return <>{children}</>;
};
export default UserProvider;
@@ -9,12 +9,13 @@ function sampleAccessibilityStatus(
): AccessibilityStatus {
return {
platform_supported: true,
core_process: { pid: 4242, started_at_ms: 1712700000000 },
permissions: {
screen_recording: 'denied',
accessibility: 'granted',
input_monitoring: 'unknown',
},
features: { screen_monitoring: true, device_control: true, predictive_input: true },
features: { screen_monitoring: true },
session: {
active: false,
started_at_ms: null,
@@ -117,6 +118,7 @@ describe('coreRpcClient', () => {
expect(out.result.permissions.screen_recording).toBe('denied');
expect(out.result.permissions.accessibility).toBe('granted');
expect(out.result.permissions.input_monitoring).toBe('unknown');
expect(out.result.core_process?.pid).toBe(4242);
expect(out.result.permission_check_process_path).toBe(
'/Users/dev/openhuman/app/src-tauri/binaries/openhuman-core-aarch64-apple-darwin'
);
+10
View File
@@ -1,5 +1,9 @@
import type { User } from '../types/api';
import type { TeamInvite, TeamMember, TeamWithRole } from '../types/team';
import type { AccessibilityStatus } from '../utils/tauriCommands/accessibility';
import type { AutocompleteStatus } from '../utils/tauriCommands/autocomplete';
import type { ServiceStatus } from '../utils/tauriCommands/hardware';
import type { LocalAiStatus } from '../utils/tauriCommands/localAi';
import { callCoreRpc } from './coreRpcClient';
export interface OnboardingTasks {
@@ -33,6 +37,12 @@ interface AppStateSnapshotResult {
primaryWalletAddress?: string | null;
onboardingTasks?: OnboardingTasks | null;
};
runtime: {
screenIntelligence: AccessibilityStatus;
localAi: LocalAiStatus;
autocomplete: AutocompleteStatus;
service: ServiceStatus;
};
}
export const fetchCoreAppSnapshot = async (): Promise<AppStateSnapshotResult> => {
+8 -17
View File
@@ -4,15 +4,13 @@
* Manages health monitoring for the openhuman daemon by polling
* `openhuman.health_snapshot` over core RPC from the frontend.
*/
import { getCoreStateSnapshot } from '../lib/coreState/store';
import { store } from '../store';
import {
type ComponentHealth,
type HealthSnapshot,
setDaemonStatus,
setHealthTimeoutId,
updateHealthSnapshot,
} from '../store/daemonSlice';
} from '../features/daemon/store';
import { getCoreStateSnapshot } from '../lib/coreState/store';
import { callCoreRpc } from './coreRpcClient';
export class DaemonHealthService {
@@ -35,7 +33,7 @@ export class DaemonHealthService {
const payload = await callCoreRpc<unknown>({ method: 'openhuman.health_snapshot' });
const healthSnapshot = this.parseHealthSnapshot(payload);
if (healthSnapshot) {
this.updateReduxFromHealth(healthSnapshot);
this.updateDaemonStoreFromHealth(healthSnapshot);
this.startHealthTimeout();
}
} catch {
@@ -134,18 +132,15 @@ export class DaemonHealthService {
}
/**
* Update Redux state based on received health snapshot.
* Update daemon state based on received health snapshot.
*/
private updateReduxFromHealth(snapshot: HealthSnapshot): void {
private updateDaemonStoreFromHealth(snapshot: HealthSnapshot): void {
const userId = this.getUserId();
try {
// Update the health snapshot in Redux
store.dispatch(updateHealthSnapshot({ userId, healthSnapshot: snapshot }));
// console.log('[DaemonHealth] Updated health snapshot for user:', userId, snapshot);
updateHealthSnapshot(userId, snapshot);
} catch (error) {
console.error('[DaemonHealth] Error updating Redux from health:', error);
console.error('[DaemonHealth] Error updating daemon store from health:', error);
}
}
@@ -165,13 +160,9 @@ export class DaemonHealthService {
// Set new timeout
this.healthTimeoutId = setTimeout(() => {
console.warn('[DaemonHealth] Health timeout reached - setting status to disconnected');
store.dispatch(setDaemonStatus({ userId, status: 'disconnected' }));
store.dispatch(setHealthTimeoutId({ userId, timeoutId: null }));
setDaemonStatus(userId, 'disconnected');
this.healthTimeoutId = null;
}, this.HEALTH_TIMEOUT_MS);
// Store timeout ID in Redux for cleanup
store.dispatch(setHealthTimeoutId({ userId, timeoutId: this.healthTimeoutId.toString() }));
}
/**
@@ -1,166 +0,0 @@
import { describe, expect, it } from 'vitest';
import type { AccessibilityStatus, CaptureTestResult } from '../../utils/tauriCommands';
import reducer, {
clearAccessibilityError,
fetchAccessibilityStatus,
refreshPermissionsWithRestart,
runCaptureTest,
setAccessibilityStatus,
startAccessibilitySession,
stopAccessibilitySession,
} from '../accessibilitySlice';
const sampleStatus: AccessibilityStatus = {
platform_supported: true,
permissions: {
screen_recording: 'granted',
accessibility: 'granted',
input_monitoring: 'unknown',
},
features: { screen_monitoring: true, device_control: true, predictive_input: true },
session: {
active: false,
started_at_ms: null,
expires_at_ms: null,
remaining_ms: null,
ttl_secs: 300,
panic_hotkey: 'Cmd+Shift+.',
stop_reason: null,
frames_in_memory: 0,
last_capture_at_ms: null,
last_context: null,
vision_enabled: true,
vision_state: 'idle',
vision_queue_depth: 0,
last_vision_at_ms: null,
last_vision_summary: null,
},
config: {
enabled: true,
capture_policy: 'hybrid',
policy_mode: 'all_except_blacklist',
baseline_fps: 1,
vision_enabled: true,
session_ttl_secs: 300,
panic_stop_hotkey: 'Cmd+Shift+.',
autocomplete_enabled: true,
use_vision_model: true,
keep_screenshots: false,
allowlist: [],
denylist: ['wallet'],
},
denylist: ['wallet'],
is_context_blocked: false,
permission_check_process_path: '/test/app/src-tauri/binaries/openhuman-core-aarch64-apple-darwin',
};
describe('accessibilitySlice', () => {
it('has expected initial state', () => {
const state = reducer(undefined, { type: '@@INIT' });
expect(state.status).toBeNull();
expect(state.isLoading).toBe(false);
expect(state.lastError).toBeNull();
});
it('stores status payload', () => {
const state = reducer(undefined, setAccessibilityStatus(sampleStatus));
expect(state.status?.platform_supported).toBe(true);
expect(state.status?.config.capture_policy).toBe('hybrid');
});
it('tracks fetch lifecycle', () => {
const pending = reducer(undefined, { type: fetchAccessibilityStatus.pending.type });
expect(pending.isLoading).toBe(true);
const fulfilled = reducer(
pending,
fetchAccessibilityStatus.fulfilled(sampleStatus, 'req-1', undefined)
);
expect(fulfilled.isLoading).toBe(false);
expect(fulfilled.status?.permissions.accessibility).toBe('granted');
});
it('stores permission_check_process_path from fetched status', () => {
const fulfilled = reducer(
undefined,
fetchAccessibilityStatus.fulfilled(sampleStatus, 'req-path', undefined)
);
expect(fulfilled.status?.permission_check_process_path).toBe(
'/test/app/src-tauri/binaries/openhuman-core-aarch64-apple-darwin'
);
});
it('stores permission_check_process_path after refreshPermissionsWithRestart', () => {
const fulfilled = reducer(
undefined,
refreshPermissionsWithRestart.fulfilled(sampleStatus, 'req-restart', undefined)
);
expect(fulfilled.isRestartingCore).toBe(false);
expect(fulfilled.status?.permission_check_process_path).toBe(
'/test/app/src-tauri/binaries/openhuman-core-aarch64-apple-darwin'
);
});
it('tracks session start/stop async flags', () => {
const starting = reducer(undefined, { type: startAccessibilitySession.pending.type });
expect(starting.isStartingSession).toBe(true);
const started = reducer(
starting,
startAccessibilitySession.fulfilled(sampleStatus, 'req-2', { consent: true })
);
expect(started.isStartingSession).toBe(false);
const stopping = reducer(started, { type: stopAccessibilitySession.pending.type });
expect(stopping.isStoppingSession).toBe(true);
});
it('clears errors', () => {
const errored = reducer(undefined, {
type: fetchAccessibilityStatus.rejected.type,
payload: 'boom',
});
expect(errored.lastError).toBe('boom');
const cleared = reducer(errored, clearAccessibilityError());
expect(cleared.lastError).toBeNull();
});
it('tracks capture test lifecycle', () => {
const pending = reducer(undefined, { type: runCaptureTest.pending.type });
expect(pending.isCaptureTestRunning).toBe(true);
expect(pending.captureTestResult).toBeNull();
const testResult: CaptureTestResult = {
ok: true,
capture_mode: 'windowed',
context: {
app_name: 'Safari',
window_title: 'GitHub',
bounds_x: 0,
bounds_y: 0,
bounds_width: 1400,
bounds_height: 900,
},
image_ref: 'data:image/png;base64,abc',
bytes_estimate: 12345,
error: null,
timing_ms: 150,
};
const fulfilled = reducer(pending, runCaptureTest.fulfilled(testResult, 'req-3', undefined));
expect(fulfilled.isCaptureTestRunning).toBe(false);
expect(fulfilled.captureTestResult?.ok).toBe(true);
expect(fulfilled.captureTestResult?.capture_mode).toBe('windowed');
});
it('handles capture test failure', () => {
const rejected = reducer(undefined, {
type: runCaptureTest.rejected.type,
payload: 'capture failed',
});
expect(rejected.isCaptureTestRunning).toBe(false);
expect(rejected.lastError).toBe('capture failed');
});
});
-125
View File
@@ -1,125 +0,0 @@
import { describe, expect, it } from 'vitest';
import reducer, {
resetAIState,
setAIError,
setAIStatus,
setCurrentSessionId,
setLoadedSkillsCount,
setMemoryInitialized,
updateAIConfig,
} from '../aiSlice';
describe('aiSlice', () => {
const initialState = reducer(undefined, { type: '@@INIT' });
describe('initial state', () => {
it('should have idle status', () => {
expect(initialState.status).toBe('idle');
});
it('should have null error', () => {
expect(initialState.error).toBeNull();
});
it('should have null session ID', () => {
expect(initialState.currentSessionId).toBeNull();
});
it('should have 0 loaded skills', () => {
expect(initialState.loadedSkillsCount).toBe(0);
});
it('should not be memory initialized', () => {
expect(initialState.memoryInitialized).toBe(false);
});
it('should have default skills repo URL', () => {
expect(initialState.config.skillsRepoUrl).toBe('openhuman/openhuman-skills');
});
});
describe('setAIStatus', () => {
it('should update status', () => {
const state = reducer(initialState, setAIStatus('ready'));
expect(state.status).toBe('ready');
});
it('should clear error when status is not error', () => {
const errorState = reducer(initialState, setAIError('something broke'));
expect(errorState.error).toBe('something broke');
const readyState = reducer(errorState, setAIStatus('ready'));
expect(readyState.error).toBeNull();
});
it('should keep error when status is error', () => {
const state = reducer({ ...initialState, error: 'old error' }, setAIStatus('error'));
expect(state.status).toBe('error');
// error is not cleared since status is "error"
});
});
describe('setAIError', () => {
it('should set error message and status to error', () => {
const state = reducer(initialState, setAIError('Init failed'));
expect(state.status).toBe('error');
expect(state.error).toBe('Init failed');
});
});
describe('setCurrentSessionId', () => {
it('should set session ID', () => {
const state = reducer(initialState, setCurrentSessionId('session-abc'));
expect(state.currentSessionId).toBe('session-abc');
});
it('should clear session ID with null', () => {
const withSession = reducer(initialState, setCurrentSessionId('session-abc'));
const cleared = reducer(withSession, setCurrentSessionId(null));
expect(cleared.currentSessionId).toBeNull();
});
});
describe('setLoadedSkillsCount', () => {
it('should update skills count', () => {
const state = reducer(initialState, setLoadedSkillsCount(5));
expect(state.loadedSkillsCount).toBe(5);
});
});
describe('setMemoryInitialized', () => {
it('should update memory initialized flag', () => {
const state = reducer(initialState, setMemoryInitialized(true));
expect(state.memoryInitialized).toBe(true);
});
});
describe('updateAIConfig', () => {
it('should merge partial config', () => {
const state = reducer(
initialState,
updateAIConfig({ llmEndpoint: 'http://localhost:8080', llmModel: 'custom-v1' })
);
expect(state.config.llmEndpoint).toBe('http://localhost:8080');
expect(state.config.llmModel).toBe('custom-v1');
expect(state.config.skillsRepoUrl).toBe('openhuman/openhuman-skills');
});
it('should override existing config values', () => {
const state1 = reducer(initialState, updateAIConfig({ llmEndpoint: 'http://v1' }));
const state2 = reducer(state1, updateAIConfig({ llmEndpoint: 'http://v2' }));
expect(state2.config.llmEndpoint).toBe('http://v2');
});
});
describe('resetAIState', () => {
it('should reset to initial state', () => {
const modified = reducer(initialState, setAIStatus('ready'));
const modified2 = reducer(modified, setCurrentSessionId('session-x'));
const reset = reducer(modified2, resetAIState());
expect(reset.status).toBe('idle');
expect(reset.currentSessionId).toBeNull();
expect(reset.loadedSkillsCount).toBe(0);
});
});
});
@@ -1,76 +0,0 @@
import { describe, expect, it } from 'vitest';
import { selectHasIncompleteOnboarding, selectIsOnboarded } from '../authSelectors';
import type { RootState } from '../index';
function makeState(
overrides: Partial<{
token: string | null;
userId: string | undefined;
isOnboardedByUser: Record<string, boolean>;
}> = {}
): RootState {
const { token = null, userId, isOnboardedByUser = {} } = overrides;
return {
auth: {
token,
isAuthBootstrapComplete: true,
isOnboardedByUser,
onboardingTasksByUser: {},
hasIncompleteOnboardingByUser: {},
isAnalyticsEnabledByUser: {},
encryptionKeyByUser: {},
primaryWalletAddressByUser: {},
},
user: {
user: userId ? ({ _id: userId } as RootState['user']['user']) : null,
isLoading: false,
error: null,
},
socket: { byUser: {} },
team: {} as RootState['team'],
ai: {} as RootState['ai'],
daemon: {} as RootState['daemon'],
thread: {} as RootState['thread'],
intelligence: {} as RootState['intelligence'],
invite: {} as RootState['invite'],
accessibility: {} as RootState['accessibility'],
channelConnections: {} as RootState['channelConnections'],
} as unknown as RootState;
}
describe('selectIsOnboarded', () => {
it('returns false when no user is loaded', () => {
const state = makeState();
expect(selectIsOnboarded(state)).toBe(false);
});
it('returns false when user exists but is not onboarded', () => {
const state = makeState({ userId: 'u1' });
expect(selectIsOnboarded(state)).toBe(false);
});
it('returns true when user is onboarded', () => {
const state = makeState({ userId: 'u1', isOnboardedByUser: { u1: true } });
expect(selectIsOnboarded(state)).toBe(true);
});
it('returns false for a different user id', () => {
const state = makeState({ userId: 'u2', isOnboardedByUser: { u1: true } });
expect(selectIsOnboarded(state)).toBe(false);
});
});
describe('selectHasIncompleteOnboarding', () => {
it('returns false when no user is loaded', () => {
const state = makeState();
expect(selectHasIncompleteOnboarding(state)).toBe(false);
});
it('returns true when current user has incomplete tasks', () => {
const state = makeState({ userId: 'u1' });
state.auth.hasIncompleteOnboardingByUser = { u1: true };
expect(selectHasIncompleteOnboarding(state)).toBe(true);
});
});
-136
View File
@@ -1,136 +0,0 @@
import { configureStore } from '@reduxjs/toolkit';
import { describe, expect, it } from 'vitest';
import authReducer, {
clearToken,
setAnalyticsForUser,
setEncryptionKeyForUser,
setOnboardedForUser,
setOnboardingTasksForUser,
setPrimaryWalletAddressForUser,
setToken,
} from '../authSlice';
import teamReducer from '../teamSlice';
import userReducer from '../userSlice';
function createStore(preloaded?: Record<string, unknown>) {
return configureStore({
// Cast reducer map to any to avoid strict typing issues in this lightweight test harness.
reducer: { auth: authReducer, user: userReducer, team: teamReducer } as unknown as any,
preloadedState: preloaded as never,
});
}
describe('authSlice', () => {
it('has correct initial state', () => {
const store = createStore();
const state = store.getState().auth;
expect(state.token).toBeNull();
expect(state.isOnboardedByUser).toEqual({});
expect(state.onboardingTasksByUser).toEqual({});
expect(state.hasIncompleteOnboardingByUser).toEqual({});
expect(state.isAnalyticsEnabledByUser).toEqual({});
});
it('sets token via setToken', () => {
const store = createStore();
store.dispatch(setToken('jwt-abc-123'));
expect(store.getState().auth.token).toBe('jwt-abc-123');
});
it('sets onboarded flag per user', () => {
const store = createStore();
store.dispatch(setOnboardedForUser({ userId: 'u1', value: true }));
expect(store.getState().auth.isOnboardedByUser.u1).toBe(true);
store.dispatch(setOnboardedForUser({ userId: 'u2', value: false }));
expect(store.getState().auth.isOnboardedByUser.u2).toBe(false);
// u1 should be unaffected
expect(store.getState().auth.isOnboardedByUser.u1).toBe(true);
});
it('sets analytics consent per user', () => {
const store = createStore();
store.dispatch(setAnalyticsForUser({ userId: 'u1', enabled: true }));
expect(store.getState().auth.isAnalyticsEnabledByUser.u1).toBe(true);
store.dispatch(setAnalyticsForUser({ userId: 'u1', enabled: false }));
expect(store.getState().auth.isAnalyticsEnabledByUser.u1).toBe(false);
});
it('tracks onboarding tasks and incomplete flag per user', () => {
const store = createStore();
store.dispatch(
setOnboardingTasksForUser({
userId: 'u1',
tasks: {
accessibilityPermissionGranted: false,
localModelConsentGiven: true,
localModelDownloadStarted: false,
enabledTools: [],
connectedSources: [],
},
})
);
expect(store.getState().auth.hasIncompleteOnboardingByUser.u1).toBe(true);
expect(store.getState().auth.onboardingTasksByUser.u1.localModelConsentGiven).toBe(true);
store.dispatch(
setOnboardingTasksForUser({
userId: 'u1',
tasks: {
accessibilityPermissionGranted: true,
localModelConsentGiven: true,
localModelDownloadStarted: true,
enabledTools: ['shell', 'file_read'],
connectedSources: ['telegram'],
},
})
);
expect(store.getState().auth.hasIncompleteOnboardingByUser.u1).toBe(false);
});
it('clearToken resets all per-user state', async () => {
const store = createStore();
// Populate all per-user fields
store.dispatch(setToken('jwt-abc'));
store.dispatch(setOnboardedForUser({ userId: 'u1', value: true }));
store.dispatch(setAnalyticsForUser({ userId: 'u1', enabled: true }));
store.dispatch(setEncryptionKeyForUser({ userId: 'u1', key: 'aes-hex' }));
store.dispatch(setPrimaryWalletAddressForUser({ userId: 'u1', address: '0xabc' }));
store.dispatch(
setOnboardingTasksForUser({
userId: 'u1',
tasks: {
accessibilityPermissionGranted: true,
localModelConsentGiven: true,
localModelDownloadStarted: true,
enabledTools: ['shell'],
connectedSources: ['telegram'],
},
})
);
// Verify state is populated
const before = store.getState().auth;
expect(before.token).toBe('jwt-abc');
expect(before.isOnboardedByUser.u1).toBe(true);
expect(before.isAnalyticsEnabledByUser.u1).toBe(true);
expect(before.encryptionKeyByUser.u1).toBe('aes-hex');
// Dispatch clearToken thunk
await store.dispatch(clearToken());
// Verify everything is reset
const after = store.getState().auth;
expect(after.token).toBeNull();
expect(after.isOnboardedByUser).toEqual({});
expect(after.onboardingTasksByUser).toEqual({});
expect(after.hasIncompleteOnboardingByUser).toEqual({});
expect(after.isAnalyticsEnabledByUser).toEqual({});
expect(after.encryptionKeyByUser).toEqual({});
expect(after.primaryWalletAddressByUser).toEqual({});
});
});
@@ -21,6 +21,7 @@ function makeCoreState(token: string | null): CoreState {
onboardingCompleted: false,
analyticsEnabled: false,
localState: { encryptionKey: null, primaryWalletAddress: null, onboardingTasks: null },
runtime: { screenIntelligence: null, localAi: null, autocomplete: null, service: null },
},
teams: [],
teamMembersById: {},
-151
View File
@@ -1,151 +0,0 @@
import { describe, expect, it, vi } from 'vitest';
import type { TeamInvite, TeamMember, TeamWithRole } from '../../types/team';
import reducer, { clearTeamState, fetchInvites, fetchMembers, fetchTeams } from '../teamSlice';
// Mock the teamApi module
vi.mock('../../services/api/teamApi', () => ({
teamApi: { getTeams: vi.fn(), getMembers: vi.fn(), getInvites: vi.fn() },
}));
const mockTeam: TeamWithRole = {
team: {
_id: 'team-1',
name: 'Test Team',
slug: 'test-team',
createdBy: 'user-1',
isPersonal: false,
maxMembers: 10,
subscription: { plan: 'FREE', hasActiveSubscription: false },
usage: { dailyTokenLimit: 1000, remainingTokens: 500, activeSessionCount: 1 },
createdAt: '2026-01-01T00:00:00Z',
updatedAt: '2026-01-01T00:00:00Z',
},
role: 'ADMIN',
};
const mockMember: TeamMember = {
_id: 'member-1',
user: { _id: 'user-1', firstName: 'John', lastName: 'Doe', username: 'johndoe' },
role: 'ADMIN',
joinedAt: '2026-01-01T00:00:00Z',
};
const mockInvite: TeamInvite = {
_id: 'invite-1',
code: 'ABC123',
createdBy: 'user-1',
expiresAt: '2026-12-31T00:00:00Z',
maxUses: 10,
currentUses: 2,
usageHistory: [],
};
describe('teamSlice', () => {
const initialState = reducer(undefined, { type: '@@INIT' });
describe('initial state', () => {
it('should have empty teams array', () => {
expect(initialState.teams).toEqual([]);
});
it('should have empty members array', () => {
expect(initialState.members).toEqual([]);
});
it('should have empty invites array', () => {
expect(initialState.invites).toEqual([]);
});
it('should not be loading', () => {
expect(initialState.isLoading).toBe(false);
});
it('should have null error', () => {
expect(initialState.error).toBeNull();
});
});
describe('clearTeamState', () => {
it('should reset to initial state', () => {
const populated = {
...initialState,
teams: [mockTeam],
members: [mockMember],
invites: [mockInvite],
isLoading: true,
error: 'some error',
};
const cleared = reducer(populated, clearTeamState());
expect(cleared).toEqual(initialState);
});
});
describe('fetchTeams', () => {
it('should set isLoading on pending', () => {
const state = reducer(initialState, fetchTeams.pending('', undefined));
expect(state.isLoading).toBe(true);
expect(state.error).toBeNull();
});
it('should populate teams on fulfilled', () => {
const teams = [mockTeam];
const state = reducer(initialState, fetchTeams.fulfilled(teams, '', undefined));
expect(state.isLoading).toBe(false);
expect(state.teams).toEqual(teams);
});
it('should set error on rejected', () => {
const state = reducer(
initialState,
fetchTeams.rejected(null, '', undefined, 'Network error')
);
expect(state.isLoading).toBe(false);
expect(state.error).toBe('Network error');
});
it('should clear previous error on pending', () => {
const withError = { ...initialState, error: 'old error' };
const state = reducer(withError, fetchTeams.pending('', undefined));
expect(state.error).toBeNull();
});
});
describe('fetchMembers', () => {
it('should populate members on fulfilled', () => {
const members = [mockMember];
const state = reducer(initialState, fetchMembers.fulfilled(members, '', 'team-1'));
expect(state.members).toEqual(members);
});
it('should set error on rejected', () => {
const state = reducer(initialState, fetchMembers.rejected(null, '', 'team-1', 'Forbidden'));
expect(state.error).toBe('Forbidden');
});
it('should clear error on pending', () => {
const withError = { ...initialState, error: 'old' };
const state = reducer(withError, fetchMembers.pending('', 'team-1'));
expect(state.error).toBeNull();
});
});
describe('fetchInvites', () => {
it('should populate invites on fulfilled', () => {
const invites = [mockInvite];
const state = reducer(initialState, fetchInvites.fulfilled(invites, '', 'team-1'));
expect(state.invites).toEqual(invites);
});
it('should set error on rejected', () => {
const state = reducer(initialState, fetchInvites.rejected(null, '', 'team-1', 'Not found'));
expect(state.error).toBe('Not found');
});
it('should clear error on pending', () => {
const withError = { ...initialState, error: 'old' };
const state = reducer(withError, fetchInvites.pending('', 'team-1'));
expect(state.error).toBeNull();
});
});
});
-66
View File
@@ -1,66 +0,0 @@
import { configureStore } from '@reduxjs/toolkit';
import { describe, expect, it } from 'vitest';
import type { User } from '../../types/api';
import userReducer, { clearUser, setUser } from '../userSlice';
function createStore() {
return configureStore({ reducer: { user: userReducer } });
}
const mockUser: User = {
_id: 'user-123',
telegramId: 12345678,
hasAccess: true,
magicWord: 'alpha',
firstName: 'Test',
lastName: 'User',
username: 'testuser',
role: 'user',
activeTeamId: 'team-1',
referral: {},
subscription: { hasActiveSubscription: false, plan: 'FREE' },
settings: {
dailySummariesEnabled: false,
dailySummaryChatIds: [],
autoCompleteEnabled: false,
autoCompleteVisibility: 'always',
autoCompleteWhitelistChatIds: [],
autoCompleteBlacklistChatIds: [],
},
usage: { cycleBudgetUsd: 10, spentThisCycleUsd: 0, spentTodayUsd: 0, cycleStartDate: new Date() },
autoDeleteTelegramMessagesAfterDays: 30,
autoDeleteThreadsAfterDays: 30,
};
describe('userSlice', () => {
it('starts with null user', () => {
const store = createStore();
expect(store.getState().user.user).toBeNull();
expect(store.getState().user.isLoading).toBe(false);
expect(store.getState().user.error).toBeNull();
});
it('sets user via setUser', () => {
const store = createStore();
store.dispatch(setUser(mockUser));
expect(store.getState().user.user).toEqual(mockUser);
expect(store.getState().user.error).toBeNull();
});
it('clears user via clearUser', () => {
const store = createStore();
store.dispatch(setUser(mockUser));
store.dispatch(clearUser());
expect(store.getState().user.user).toBeNull();
expect(store.getState().user.isLoading).toBe(false);
expect(store.getState().user.error).toBeNull();
});
it('sets user to null via setUser(null)', () => {
const store = createStore();
store.dispatch(setUser(mockUser));
store.dispatch(setUser(null));
expect(store.getState().user.user).toBeNull();
});
});
-389
View File
@@ -1,389 +0,0 @@
import { createAsyncThunk, createSlice, type PayloadAction } from '@reduxjs/toolkit';
import {
type AccessibilityInputActionParams,
type AccessibilityPermissionKind,
type AccessibilitySessionStatus,
type AccessibilityStatus,
type AccessibilityVisionSummary,
type CaptureTestResult,
openhumanAccessibilityInputAction,
openhumanAccessibilityRequestPermission,
openhumanAccessibilityRequestPermissions,
openhumanAccessibilityStartSession,
openhumanAccessibilityStatus,
openhumanAccessibilityStopSession,
openhumanAccessibilityVisionFlush,
openhumanAccessibilityVisionRecent,
openhumanScreenIntelligenceCaptureTest,
restartCoreProcess,
} from '../utils/tauriCommands';
interface AccessibilityState {
status: AccessibilityStatus | null;
recentVisionSummaries: AccessibilityVisionSummary[];
captureTestResult: CaptureTestResult | null;
isCaptureTestRunning: boolean;
isLoading: boolean;
isRequestingPermissions: boolean;
isRestartingCore: boolean;
isStartingSession: boolean;
isStoppingSession: boolean;
isLoadingVision: boolean;
isFlushingVision: boolean;
lastError: string | null;
}
const initialState: AccessibilityState = {
status: null,
recentVisionSummaries: [],
captureTestResult: null,
isCaptureTestRunning: false,
isLoading: false,
isRequestingPermissions: false,
isRestartingCore: false,
isStartingSession: false,
isStoppingSession: false,
isLoadingVision: false,
isFlushingVision: false,
lastError: null,
};
const extractError = (error: unknown, fallback: string): string => {
if (error instanceof Error && error.message.trim()) {
return error.message;
}
if (typeof error === 'string' && error.trim()) {
return error;
}
if (error && typeof error === 'object') {
const msg = (error as { message?: unknown }).message;
if (typeof msg === 'string' && msg.trim()) {
return msg;
}
}
return fallback;
};
export const fetchAccessibilityStatus = createAsyncThunk(
'accessibility/fetchStatus',
async (_, { rejectWithValue }) => {
try {
const response = await openhumanAccessibilityStatus();
return response.result;
} catch (error) {
return rejectWithValue(extractError(error, 'Failed to fetch accessibility status'));
}
}
);
export const requestAccessibilityPermissions = createAsyncThunk(
'accessibility/requestPermissions',
async (_, { rejectWithValue }) => {
try {
await openhumanAccessibilityRequestPermissions();
const response = await openhumanAccessibilityStatus();
return response.result;
} catch (error) {
return rejectWithValue(extractError(error, 'Failed to request accessibility permissions'));
}
}
);
export const requestAccessibilityPermission = createAsyncThunk(
'accessibility/requestPermission',
async (permission: AccessibilityPermissionKind, { rejectWithValue }) => {
try {
await openhumanAccessibilityRequestPermission(permission);
const response = await openhumanAccessibilityStatus();
return response.result;
} catch (error) {
return rejectWithValue(extractError(error, 'Failed to request accessibility permission'));
}
}
);
/**
* Restart the core sidecar and re-fetch accessibility status.
*
* macOS caches permission state per-process. The running sidecar never sees a
* newly granted permission until it exits and a fresh process starts. This thunk:
* 1. Restarts the core sidecar via the `restart_core_process` Tauri command.
* 2. Waits briefly, then re-fetches status with retries while the new sidecar binds.
* 3. Updates Redux so the UI reflects the updated grants.
*
* @see https://github.com/tinyhumansai/openhuman/issues/133 — stale DENIED after granting in System Settings
*/
export const refreshPermissionsWithRestart = createAsyncThunk(
'accessibility/refreshPermissionsWithRestart',
async (_, { rejectWithValue }) => {
try {
console.debug('[accessibility] refreshPermissionsWithRestart: starting core restart');
await restartCoreProcess();
console.debug('[accessibility] refreshPermissionsWithRestart: waiting for sidecar ready');
await new Promise<void>(resolve => setTimeout(resolve, 400));
console.debug('[accessibility] refreshPermissionsWithRestart: fetching updated status');
for (let attempt = 1; attempt <= 5; attempt++) {
try {
const response = await openhumanAccessibilityStatus();
console.debug(
'[accessibility] refreshPermissionsWithRestart: done — screen_recording=%s accessibility=%s input_monitoring=%s',
response.result.permissions.screen_recording,
response.result.permissions.accessibility,
response.result.permissions.input_monitoring
);
return response.result;
} catch (e) {
if (attempt === 5) {
throw e;
}
console.debug(
'[accessibility] refreshPermissionsWithRestart: status fetch failed (attempt %s), retrying…',
attempt
);
await new Promise<void>(resolve => setTimeout(resolve, 350 * attempt));
}
}
throw new Error('Failed to fetch accessibility status after core restart');
} catch (error) {
const msg = extractError(error, 'Failed to restart core and refresh permissions');
console.error('[accessibility] refreshPermissionsWithRestart: error —', msg, error);
return rejectWithValue(msg);
}
}
);
export const startAccessibilitySession = createAsyncThunk(
'accessibility/startSession',
async (
params: {
consent: boolean;
ttl_secs?: number;
screen_monitoring?: boolean;
device_control?: boolean;
predictive_input?: boolean;
},
{ rejectWithValue }
) => {
try {
await openhumanAccessibilityStartSession(params);
const response = await openhumanAccessibilityStatus();
return response.result;
} catch (error) {
return rejectWithValue(extractError(error, 'Failed to start accessibility session'));
}
}
);
export const stopAccessibilitySession = createAsyncThunk(
'accessibility/stopSession',
async (reason: string | undefined, { rejectWithValue }) => {
try {
await openhumanAccessibilityStopSession(reason ? { reason } : undefined);
const response = await openhumanAccessibilityStatus();
return response.result;
} catch (error) {
return rejectWithValue(extractError(error, 'Failed to stop accessibility session'));
}
}
);
export const executeAccessibilityInputAction = createAsyncThunk(
'accessibility/inputAction',
async (params: AccessibilityInputActionParams, { rejectWithValue }) => {
try {
const response = await openhumanAccessibilityInputAction(params);
return response.result;
} catch (error) {
return rejectWithValue(extractError(error, 'Failed to execute input action'));
}
}
);
export const fetchAccessibilityVisionRecent = createAsyncThunk(
'accessibility/fetchVisionRecent',
async (limit: number | undefined, { rejectWithValue }) => {
try {
const response = await openhumanAccessibilityVisionRecent(limit);
return response.result.summaries;
} catch (error) {
return rejectWithValue(extractError(error, 'Failed to fetch accessibility vision summaries'));
}
}
);
export const flushAccessibilityVision = createAsyncThunk(
'accessibility/flushVision',
async (_, { rejectWithValue }) => {
try {
const response = await openhumanAccessibilityVisionFlush();
return response.result.summary;
} catch (error) {
return rejectWithValue(extractError(error, 'Failed to flush accessibility vision'));
}
}
);
export const runCaptureTest = createAsyncThunk(
'accessibility/captureTest',
async (_, { rejectWithValue }) => {
try {
const response = await openhumanScreenIntelligenceCaptureTest();
return response.result;
} catch (error) {
return rejectWithValue(extractError(error, 'Failed to run capture test'));
}
}
);
const accessibilitySlice = createSlice({
name: 'accessibility',
initialState,
reducers: {
clearAccessibilityError(state) {
state.lastError = null;
},
setAccessibilityStatus(state, action: PayloadAction<AccessibilityStatus | null>) {
state.status = action.payload;
},
setAccessibilitySessionFeatures(state, action: PayloadAction<AccessibilitySessionStatus>) {
if (state.status) {
state.status.session = action.payload;
}
},
setAccessibilityVisionSummaries(state, action: PayloadAction<AccessibilityVisionSummary[]>) {
state.recentVisionSummaries = action.payload;
},
},
extraReducers: builder => {
builder
.addCase(fetchAccessibilityStatus.pending, state => {
state.isLoading = true;
state.lastError = null;
})
.addCase(fetchAccessibilityStatus.fulfilled, (state, action) => {
state.isLoading = false;
state.status = action.payload;
})
.addCase(fetchAccessibilityStatus.rejected, (state, action) => {
state.isLoading = false;
state.lastError = (action.payload as string) ?? 'Failed to fetch accessibility status';
})
.addCase(requestAccessibilityPermissions.pending, state => {
state.isRequestingPermissions = true;
state.lastError = null;
})
.addCase(requestAccessibilityPermissions.fulfilled, (state, action) => {
state.isRequestingPermissions = false;
state.status = action.payload;
})
.addCase(requestAccessibilityPermissions.rejected, (state, action) => {
state.isRequestingPermissions = false;
state.lastError =
(action.payload as string) ?? 'Failed to request accessibility permissions';
})
.addCase(requestAccessibilityPermission.pending, state => {
state.isRequestingPermissions = true;
state.lastError = null;
})
.addCase(requestAccessibilityPermission.fulfilled, (state, action) => {
state.isRequestingPermissions = false;
state.status = action.payload;
})
.addCase(requestAccessibilityPermission.rejected, (state, action) => {
state.isRequestingPermissions = false;
state.lastError =
(action.payload as string) ?? 'Failed to request accessibility permission';
})
.addCase(refreshPermissionsWithRestart.pending, state => {
state.isRestartingCore = true;
state.lastError = null;
})
.addCase(refreshPermissionsWithRestart.fulfilled, (state, action) => {
state.isRestartingCore = false;
state.status = action.payload;
})
.addCase(refreshPermissionsWithRestart.rejected, (state, action) => {
state.isRestartingCore = false;
state.lastError =
(action.payload as string) ?? 'Failed to restart core and refresh permissions';
})
.addCase(startAccessibilitySession.pending, state => {
state.isStartingSession = true;
state.lastError = null;
})
.addCase(startAccessibilitySession.fulfilled, (state, action) => {
state.isStartingSession = false;
state.status = action.payload;
})
.addCase(startAccessibilitySession.rejected, (state, action) => {
state.isStartingSession = false;
state.lastError = (action.payload as string) ?? 'Failed to start accessibility session';
})
.addCase(stopAccessibilitySession.pending, state => {
state.isStoppingSession = true;
state.lastError = null;
})
.addCase(stopAccessibilitySession.fulfilled, (state, action) => {
state.isStoppingSession = false;
state.status = action.payload;
})
.addCase(stopAccessibilitySession.rejected, (state, action) => {
state.isStoppingSession = false;
state.lastError = (action.payload as string) ?? 'Failed to stop accessibility session';
})
.addCase(executeAccessibilityInputAction.rejected, (state, action) => {
state.lastError = (action.payload as string) ?? 'Failed to execute accessibility action';
})
.addCase(fetchAccessibilityVisionRecent.pending, state => {
state.isLoadingVision = true;
})
.addCase(fetchAccessibilityVisionRecent.fulfilled, (state, action) => {
state.isLoadingVision = false;
state.recentVisionSummaries = action.payload;
})
.addCase(fetchAccessibilityVisionRecent.rejected, (state, action) => {
state.isLoadingVision = false;
state.lastError =
(action.payload as string) ?? 'Failed to fetch accessibility vision summaries';
})
.addCase(flushAccessibilityVision.pending, state => {
state.isFlushingVision = true;
})
.addCase(flushAccessibilityVision.fulfilled, (state, action) => {
state.isFlushingVision = false;
if (action.payload) {
state.recentVisionSummaries = [action.payload, ...state.recentVisionSummaries].slice(
0,
30
);
}
})
.addCase(flushAccessibilityVision.rejected, (state, action) => {
state.isFlushingVision = false;
state.lastError = (action.payload as string) ?? 'Failed to flush accessibility vision';
})
.addCase(runCaptureTest.pending, state => {
state.isCaptureTestRunning = true;
state.captureTestResult = null;
state.lastError = null;
})
.addCase(runCaptureTest.fulfilled, (state, action) => {
state.isCaptureTestRunning = false;
state.captureTestResult = action.payload;
})
.addCase(runCaptureTest.rejected, (state, action) => {
state.isCaptureTestRunning = false;
state.lastError = (action.payload as string) ?? 'Failed to run capture test';
});
},
});
export const {
clearAccessibilityError,
setAccessibilitySessionFeatures,
setAccessibilityStatus,
setAccessibilityVisionSummaries,
} = accessibilitySlice.actions;
export default accessibilitySlice.reducer;
-90
View File
@@ -1,90 +0,0 @@
import { createSlice, type PayloadAction } from '@reduxjs/toolkit';
/** AI system connection/initialization status */
export type AIStatus = 'idle' | 'initializing' | 'ready' | 'error';
/** Persisted AI configuration */
export interface AIConfig {
/** Custom LLM endpoint URL */
llmEndpoint?: string;
/** Custom LLM model name */
llmModel?: string;
/** Embedding provider: 'openai' | 'custom' | 'none' */
embeddingProvider?: string;
/** OpenAI API key (for embeddings fallback) */
openaiApiKey?: string;
/** Web search API endpoint */
webSearchEndpoint?: string;
/** Web search API key */
webSearchApiKey?: string;
/** Skills repo URL */
skillsRepoUrl?: string;
}
interface AIState {
/** Current AI system status */
status: AIStatus;
/** Error message if status is 'error' */
error: string | null;
/** Current active session ID */
currentSessionId: string | null;
/** Number of loaded skills */
loadedSkillsCount: number;
/** Memory system initialized */
memoryInitialized: boolean;
/** Persisted AI configuration */
config: AIConfig;
}
const initialState: AIState = {
status: 'idle',
error: null,
currentSessionId: null,
loadedSkillsCount: 0,
memoryInitialized: false,
config: { skillsRepoUrl: 'openhuman/openhuman-skills' },
};
const aiSlice = createSlice({
name: 'ai',
initialState,
reducers: {
setAIStatus(state, action: PayloadAction<AIStatus>) {
state.status = action.payload;
if (action.payload !== 'error') {
state.error = null;
}
},
setAIError(state, action: PayloadAction<string>) {
state.status = 'error';
state.error = action.payload;
},
setCurrentSessionId(state, action: PayloadAction<string | null>) {
state.currentSessionId = action.payload;
},
setLoadedSkillsCount(state, action: PayloadAction<number>) {
state.loadedSkillsCount = action.payload;
},
setMemoryInitialized(state, action: PayloadAction<boolean>) {
state.memoryInitialized = action.payload;
},
updateAIConfig(state, action: PayloadAction<Partial<AIConfig>>) {
state.config = { ...state.config, ...action.payload };
},
resetAIState() {
return initialState;
},
},
});
export const {
setAIStatus,
setAIError,
setCurrentSessionId,
setLoadedSkillsCount,
setMemoryInitialized,
updateAIConfig,
resetAIState,
} = aiSlice.actions;
export default aiSlice.reducer;
-19
View File
@@ -1,19 +0,0 @@
import type { RootState } from './index';
export const selectIsOnboarded = (state: RootState): boolean => {
const userId = state.user.user?._id;
if (!userId) return false;
return state.auth.isOnboardedByUser[userId] ?? false;
};
export const selectHasEncryptionKey = (state: RootState): boolean => {
const userId = state.user.user?._id;
if (!userId) return false;
return !!state.auth.encryptionKeyByUser[userId];
};
export const selectHasIncompleteOnboarding = (state: RootState): boolean => {
const userId = state.user.user?._id;
if (!userId) return false;
return state.auth.hasIncompleteOnboardingByUser[userId] ?? false;
};
-112
View File
@@ -1,112 +0,0 @@
import { createAsyncThunk, createSlice, PayloadAction } from '@reduxjs/toolkit';
import { clearTeamState } from './teamSlice';
import { clearUser } from './userSlice';
export interface AuthState {
token: string | null;
/** True once startup auth/session bootstrap has checked core binary state */
isAuthBootstrapComplete: boolean;
/** Onboarding completion per user id */
isOnboardedByUser: Record<string, boolean>;
/** Additional onboarding task progress per user id */
onboardingTasksByUser: Record<string, UserOnboardingTasks>;
/** True when user completed onboarding route but skipped some optional setup tasks */
hasIncompleteOnboardingByUser: Record<string, boolean>;
/** Analytics consent per user id (opt-in during onboarding) */
isAnalyticsEnabledByUser: Record<string, boolean>;
/** AES encryption key (hex) derived from mnemonic, per user id */
encryptionKeyByUser: Record<string, string>;
/** Primary EVM wallet address (0x...) derived from mnemonic, per user id */
primaryWalletAddressByUser: Record<string, string>;
}
export interface UserOnboardingTasks {
accessibilityPermissionGranted: boolean;
localModelConsentGiven: boolean;
localModelDownloadStarted: boolean;
enabledTools: string[];
connectedSources: string[];
updatedAtMs: number;
}
const initialState: AuthState = {
token: null,
isAuthBootstrapComplete: false,
isOnboardedByUser: {},
onboardingTasksByUser: {},
hasIncompleteOnboardingByUser: {},
isAnalyticsEnabledByUser: {},
encryptionKeyByUser: {},
primaryWalletAddressByUser: {},
};
const authSlice = createSlice({
name: 'auth',
initialState,
reducers: {
setToken: (state, action: PayloadAction<string>) => {
state.token = action.payload;
},
setAuthBootstrapComplete: (state, action: PayloadAction<boolean>) => {
state.isAuthBootstrapComplete = action.payload;
},
_clearToken: state => {
state.token = null;
state.isOnboardedByUser = {};
state.onboardingTasksByUser = {};
state.hasIncompleteOnboardingByUser = {};
state.isAnalyticsEnabledByUser = {};
state.encryptionKeyByUser = {};
state.primaryWalletAddressByUser = {};
},
setOnboardedForUser: (state, action: PayloadAction<{ userId: string; value: boolean }>) => {
const { userId, value } = action.payload;
state.isOnboardedByUser[userId] = value;
},
setAnalyticsForUser: (state, action: PayloadAction<{ userId: string; enabled: boolean }>) => {
const { userId, enabled } = action.payload;
state.isAnalyticsEnabledByUser[userId] = enabled;
},
setOnboardingTasksForUser: (
state,
action: PayloadAction<{ userId: string; tasks: Omit<UserOnboardingTasks, 'updatedAtMs'> }>
) => {
const { userId, tasks } = action.payload;
state.onboardingTasksByUser[userId] = { ...tasks, updatedAtMs: Date.now() };
const hasIncomplete =
!tasks.accessibilityPermissionGranted || tasks.connectedSources.length === 0;
state.hasIncompleteOnboardingByUser[userId] = hasIncomplete;
},
setEncryptionKeyForUser: (state, action: PayloadAction<{ userId: string; key: string }>) => {
const { userId, key } = action.payload;
state.encryptionKeyByUser[userId] = key;
},
setPrimaryWalletAddressForUser: (
state,
action: PayloadAction<{ userId: string; address: string }>
) => {
const { userId, address } = action.payload;
state.primaryWalletAddressByUser[userId] = address;
},
},
});
// Thunk that clears both token and user data
export const clearToken = createAsyncThunk('auth/clearToken', async (_, { dispatch }) => {
dispatch(authSlice.actions._clearToken());
dispatch(clearUser());
dispatch(clearTeamState());
});
export const {
setToken,
setAuthBootstrapComplete,
setOnboardedForUser,
setAnalyticsForUser,
setOnboardingTasksForUser,
setEncryptionKeyForUser,
setPrimaryWalletAddressForUser,
} = authSlice.actions;
export default authSlice.reducer;
-199
View File
@@ -1,199 +0,0 @@
import { createSlice, type PayloadAction } from '@reduxjs/toolkit';
export type DaemonStatus = 'starting' | 'running' | 'error' | 'disconnected';
export type ComponentStatus = 'ok' | 'error' | 'starting';
export interface ComponentHealth {
status: ComponentStatus;
updated_at: string;
last_ok?: string;
last_error?: string;
restart_count: number;
}
export interface HealthSnapshot {
pid: number;
updated_at: string;
uptime_seconds: number;
components: Record<string, ComponentHealth>;
}
export interface DaemonUserState {
status: DaemonStatus;
healthSnapshot: HealthSnapshot | null;
components: {
gateway?: ComponentHealth;
channels?: ComponentHealth;
heartbeat?: ComponentHealth;
scheduler?: ComponentHealth;
};
lastHealthUpdate: string | null;
connectionAttempts: number;
autoStartEnabled: boolean;
isRecovering: boolean;
healthTimeoutId: string | null;
}
const initialUserState: DaemonUserState = {
status: 'disconnected',
healthSnapshot: null,
components: {},
lastHealthUpdate: null,
connectionAttempts: 0,
autoStartEnabled: false,
isRecovering: false,
healthTimeoutId: null,
};
interface DaemonState {
/** Daemon state per user id. Use __pending__ when user not loaded yet. */
byUser: Record<string, DaemonUserState>;
}
const initialState: DaemonState = { byUser: {} };
const ensureUserState = (state: DaemonState, userId: string): DaemonUserState => {
if (!state.byUser[userId]) {
state.byUser[userId] = { ...initialUserState };
}
return state.byUser[userId];
};
const daemonSlice = createSlice({
name: 'daemon',
initialState,
reducers: {
updateHealthSnapshot: (
state,
action: PayloadAction<{ userId: string; healthSnapshot: HealthSnapshot }>
) => {
const { userId, healthSnapshot } = action.payload;
const user = ensureUserState(state, userId);
user.healthSnapshot = healthSnapshot;
user.lastHealthUpdate = new Date().toISOString();
// Update component health
user.components = healthSnapshot.components;
// Determine overall daemon status based on component health
const componentStatuses = Object.values(healthSnapshot.components).map(c => c.status);
if (componentStatuses.length === 0) {
user.status = 'disconnected';
} else if (componentStatuses.every(status => status === 'ok')) {
user.status = 'running';
user.isRecovering = false;
user.connectionAttempts = 0;
} else if (componentStatuses.some(status => status === 'error')) {
user.status = 'error';
} else if (componentStatuses.some(status => status === 'starting')) {
user.status = 'starting';
}
},
setDaemonStatus: (state, action: PayloadAction<{ userId: string; status: DaemonStatus }>) => {
const { userId, status } = action.payload;
const user = ensureUserState(state, userId);
user.status = status;
if (status === 'disconnected') {
user.healthSnapshot = null;
user.components = {};
user.lastHealthUpdate = null;
}
},
incrementConnectionAttempts: (state, action: PayloadAction<{ userId: string }>) => {
const { userId } = action.payload;
const user = ensureUserState(state, userId);
user.connectionAttempts += 1;
},
resetConnectionAttempts: (state, action: PayloadAction<{ userId: string }>) => {
const { userId } = action.payload;
const user = ensureUserState(state, userId);
user.connectionAttempts = 0;
},
setAutoStartEnabled: (state, action: PayloadAction<{ userId: string; enabled: boolean }>) => {
const { userId, enabled } = action.payload;
const user = ensureUserState(state, userId);
user.autoStartEnabled = enabled;
},
setIsRecovering: (state, action: PayloadAction<{ userId: string; isRecovering: boolean }>) => {
const { userId, isRecovering } = action.payload;
const user = ensureUserState(state, userId);
user.isRecovering = isRecovering;
},
setHealthTimeoutId: (
state,
action: PayloadAction<{ userId: string; timeoutId: string | null }>
) => {
const { userId, timeoutId } = action.payload;
const user = ensureUserState(state, userId);
user.healthTimeoutId = timeoutId;
},
resetForUser: (state, action: PayloadAction<{ userId: string }>) => {
const { userId } = action.payload;
state.byUser[userId] = { ...initialUserState };
},
},
});
export const {
updateHealthSnapshot,
setDaemonStatus,
incrementConnectionAttempts,
resetConnectionAttempts,
setAutoStartEnabled,
setIsRecovering,
setHealthTimeoutId,
resetForUser,
} = daemonSlice.actions;
// Selectors
export const selectDaemonStateForUser = (state: { daemon: DaemonState }, userId?: string) => {
const uid = userId || '__pending__';
return state.daemon.byUser[uid] || initialUserState;
};
export const selectDaemonStatus = (state: { daemon: DaemonState }, userId?: string) => {
const daemonState = selectDaemonStateForUser(state, userId);
return daemonState.status;
};
export const selectDaemonComponents = (state: { daemon: DaemonState }, userId?: string) => {
const daemonState = selectDaemonStateForUser(state, userId);
return daemonState.components;
};
export const selectDaemonHealthSnapshot = (state: { daemon: DaemonState }, userId?: string) => {
const daemonState = selectDaemonStateForUser(state, userId);
return daemonState.healthSnapshot;
};
export const selectDaemonLastHealthUpdate = (state: { daemon: DaemonState }, userId?: string) => {
const daemonState = selectDaemonStateForUser(state, userId);
return daemonState.lastHealthUpdate;
};
export const selectIsDaemonAutoStartEnabled = (state: { daemon: DaemonState }, userId?: string) => {
const daemonState = selectDaemonStateForUser(state, userId);
return daemonState.autoStartEnabled;
};
export const selectDaemonConnectionAttempts = (state: { daemon: DaemonState }, userId?: string) => {
const daemonState = selectDaemonStateForUser(state, userId);
return daemonState.connectionAttempts;
};
export const selectIsDaemonRecovering = (state: { daemon: DaemonState }, userId?: string) => {
const daemonState = selectDaemonStateForUser(state, userId);
return daemonState.isRecovering;
};
export default daemonSlice.reducer;
+1 -36
View File
@@ -12,22 +12,10 @@ import {
} from 'redux-persist';
import storage from 'redux-persist/lib/storage';
import type { User } from '../types/api';
import type { TeamInvite, TeamMember, TeamWithRole } from '../types/team';
import { IS_DEV } from '../utils/config';
import accessibilityReducer from './accessibilitySlice';
import aiReducer from './aiSlice';
import type { AuthState } from './authSlice';
import channelConnectionsReducer from './channelConnectionsSlice';
import daemonReducer from './daemonSlice';
import type { IntelligenceState } from './intelligenceSlice';
import inviteReducer from './inviteSlice';
import socketReducer from './socketSlice';
import threadReducer from './threadSlice';
import webhooksReducer from './webhooksSlice';
// Persist config for AI state (config only)
const aiPersistConfig = { key: 'ai', storage, whitelist: ['config'] };
// Persist config for thread data and UI prefs (includes threads and messages)
// Note: activeThreadId is intentionally excluded as it's transient state
@@ -37,7 +25,6 @@ const threadPersistConfig = {
whitelist: ['panelWidth', 'lastViewedAt', 'threads', 'messagesByThreadId', 'selectedThreadId'],
};
const persistedAiReducer = persistReducer(aiPersistConfig, aiReducer);
const persistedThreadReducer = persistReducer(threadPersistConfig, threadReducer);
const channelConnectionsPersistConfig = {
key: 'channelConnections',
@@ -52,13 +39,8 @@ const persistedChannelConnectionsReducer = persistReducer(
export const store = configureStore({
reducer: {
socket: socketReducer,
daemon: daemonReducer,
ai: persistedAiReducer,
thread: persistedThreadReducer,
invite: inviteReducer,
accessibility: accessibilityReducer,
channelConnections: persistedChannelConnectionsReducer,
webhooks: webhooksReducer,
},
middleware: getDefaultMiddleware => {
const middleware = getDefaultMiddleware({
@@ -75,22 +57,5 @@ export const store = configureStore({
export const persistor = persistStore(store);
type RuntimeRootState = ReturnType<typeof store.getState>;
type LegacyRootState = {
auth: AuthState;
user: { user: User | null; isLoading: boolean; error: string | null };
team: {
teams: TeamWithRole[];
members: TeamMember[];
invites: TeamInvite[];
isLoading: boolean;
isLoadingMembers: boolean;
isLoadingInvites: boolean;
error: string | null;
};
intelligence: IntelligenceState;
};
export type RootState = RuntimeRootState & LegacyRootState;
export type RootState = ReturnType<typeof store.getState>;
export type AppDispatch = typeof store.dispatch;
-478
View File
@@ -1,478 +0,0 @@
import { createAsyncThunk, createSlice, type PayloadAction } from '@reduxjs/toolkit';
import { type ConnectedTool, intelligenceApi } from '../services/intelligenceApi';
import type {
ActionableItem,
ActionableItemSource,
ActionableItemStatus,
ChatMessage,
} from '../types/intelligence';
import {
transformBackendItemsToFrontend,
transformBackendMessagesToFrontend,
} from '../utils/intelligenceTransforms';
/**
* Chat session state for managing individual conversations
*/
export interface ChatSessionState {
threadId: string;
itemId: string;
messages: ChatMessage[];
isTyping: boolean;
lastMessageId?: string;
isConnected: boolean;
}
/**
* Execution state for tracking task progress
*/
export interface ExecutionState {
executionId: string;
sessionId: string;
itemId: string;
status: 'idle' | 'starting' | 'running' | 'completed' | 'failed';
progress: Array<{
id: string;
label: string;
status: 'pending' | 'in_progress' | 'completed' | 'failed';
timestamp?: Date;
}>;
result?: unknown;
error?: string;
}
/**
* Intelligence Redux state
*/
export interface IntelligenceState {
// Actionable Items
items: ActionableItem[];
loading: boolean;
error: string | null;
lastUpdate: Date | null;
// Chat Sessions
activeSessions: Record<string, ChatSessionState>;
currentChatSession: string | null;
// Execution Management
activeExecutions: Record<string, ExecutionState>;
// UI State
filters: {
source: ActionableItemSource | 'all';
priority: 'critical' | 'important' | 'normal' | 'all';
search: string;
};
// System state
initialized: boolean;
connectionStatus: 'disconnected' | 'connecting' | 'connected' | 'error';
}
const initialState: IntelligenceState = {
// Actionable Items
items: [],
loading: false,
error: null,
lastUpdate: null,
// Chat Sessions
activeSessions: {},
currentChatSession: null,
// Execution Management
activeExecutions: {},
// UI State
filters: { source: 'all', priority: 'all', search: '' },
// System state
initialized: false,
connectionStatus: 'disconnected',
};
// Async thunks for API operations
/**
* Fetch actionable items from backend
*/
export const fetchActionableItems = createAsyncThunk(
'intelligence/fetchActionableItems',
async (_, { rejectWithValue }) => {
try {
const backendItems = await intelligenceApi.getActionableItems();
return transformBackendItemsToFrontend(backendItems);
} catch (error) {
return rejectWithValue(
error && typeof error === 'object' && 'error' in error
? error.error
: 'Failed to fetch actionable items'
);
}
}
);
/**
* Update item status
*/
export const updateItemStatus = createAsyncThunk(
'intelligence/updateItemStatus',
async (
{ itemId, status }: { itemId: string; status: ActionableItemStatus },
{ rejectWithValue }
) => {
try {
await intelligenceApi.updateItemStatus(itemId, status);
return { itemId, status, updatedAt: new Date() };
} catch (error) {
return rejectWithValue(
error && typeof error === 'object' && 'error' in error
? error.error
: 'Failed to update item status'
);
}
}
);
/**
* Snooze item until specific time
*/
export const snoozeItem = createAsyncThunk(
'intelligence/snoozeItem',
async ({ itemId, snoozeUntil }: { itemId: string; snoozeUntil: Date }, { rejectWithValue }) => {
try {
await intelligenceApi.snoozeItem(itemId, snoozeUntil);
return { itemId, snoozeUntil, updatedAt: new Date() };
} catch (error) {
return rejectWithValue(
error && typeof error === 'object' && 'error' in error
? error.error
: 'Failed to snooze item'
);
}
}
);
/**
* Get or create chat session
*/
export const createChatSession = createAsyncThunk(
'intelligence/createChatSession',
async ({ itemId }: { itemId: string }, { rejectWithValue }) => {
try {
const threadResponse = await intelligenceApi.getOrCreateThread(itemId);
const messages = transformBackendMessagesToFrontend(threadResponse.messages);
return { threadId: threadResponse.threadId, itemId, messages };
} catch (error) {
return rejectWithValue(
error && typeof error === 'object' && 'error' in error
? error.error
: 'Failed to create chat session'
);
}
}
);
/**
* Execute task with connected tools
*/
export const executeTask = createAsyncThunk(
'intelligence/executeTask',
async (
{ itemId, connectedTools }: { itemId: string; connectedTools: ConnectedTool[] },
{ rejectWithValue }
) => {
try {
const response = await intelligenceApi.executeTask(itemId, connectedTools);
return {
executionId: response.executionId,
sessionId: response.sessionId,
itemId,
status: response.status,
};
} catch (error) {
return rejectWithValue(
error && typeof error === 'object' && 'error' in error
? error.error
: 'Failed to execute task'
);
}
}
);
/**
* Intelligence slice
*/
export const intelligenceSlice = createSlice({
name: 'intelligence',
initialState,
reducers: {
// System actions
setInitialized: (state, action: PayloadAction<boolean>) => {
state.initialized = action.payload;
},
setConnectionStatus: (
state,
action: PayloadAction<'disconnected' | 'connecting' | 'connected' | 'error'>
) => {
state.connectionStatus = action.payload;
},
// Items actions
setItems: (state, action: PayloadAction<ActionableItem[]>) => {
state.items = action.payload;
state.lastUpdate = new Date();
},
addItem: (state, action: PayloadAction<ActionableItem>) => {
state.items.unshift(action.payload);
state.lastUpdate = new Date();
},
removeItem: (state, action: PayloadAction<string>) => {
state.items = state.items.filter(item => item.id !== action.payload);
state.lastUpdate = new Date();
},
// Filters actions
setSourceFilter: (state, action: PayloadAction<ActionableItemSource | 'all'>) => {
state.filters.source = action.payload;
},
setPriorityFilter: (
state,
action: PayloadAction<'critical' | 'important' | 'normal' | 'all'>
) => {
state.filters.priority = action.payload;
},
setSearchFilter: (state, action: PayloadAction<string>) => {
state.filters.search = action.payload;
},
// Chat session actions
setChatSession: (
state,
action: PayloadAction<{ threadId: string; itemId: string; messages?: ChatMessage[] }>
) => {
const { threadId, itemId, messages = [] } = action.payload;
state.activeSessions[threadId] = {
threadId,
itemId,
messages,
isTyping: false,
isConnected: true,
};
state.currentChatSession = threadId;
},
addMessage: (state, action: PayloadAction<{ threadId: string; message: ChatMessage }>) => {
const { threadId, message } = action.payload;
const session = state.activeSessions[threadId];
if (session) {
session.messages.push(message);
session.lastMessageId = message.id;
}
},
setTyping: (state, action: PayloadAction<{ threadId: string; isTyping: boolean }>) => {
const { threadId, isTyping } = action.payload;
const session = state.activeSessions[threadId];
if (session) {
session.isTyping = isTyping;
}
},
closeChatSession: (state, action: PayloadAction<string>) => {
const threadId = action.payload;
delete state.activeSessions[threadId];
if (state.currentChatSession === threadId) {
state.currentChatSession = null;
}
},
setCurrentChatSession: (state, action: PayloadAction<string | null>) => {
state.currentChatSession = action.payload;
},
// Execution actions
setExecution: (
state,
action: PayloadAction<{ executionId: string; execution: ExecutionState }>
) => {
const { executionId, execution } = action.payload;
state.activeExecutions[executionId] = execution;
},
updateExecutionProgress: (
state,
action: PayloadAction<{ executionId: string; progress: ExecutionState['progress'] }>
) => {
const { executionId, progress } = action.payload;
const execution = state.activeExecutions[executionId];
if (execution) {
execution.progress = progress;
execution.status = 'running';
}
},
setExecutionResult: (
state,
action: PayloadAction<{
executionId: string;
result: unknown;
status: 'completed' | 'failed';
error?: string;
}>
) => {
const { executionId, result, status, error } = action.payload;
const execution = state.activeExecutions[executionId];
if (execution) {
execution.result = result;
execution.status = status;
execution.error = error;
}
},
clearExecution: (state, action: PayloadAction<string>) => {
const executionId = action.payload;
delete state.activeExecutions[executionId];
},
// Error handling
clearError: state => {
state.error = null;
},
},
extraReducers: builder => {
// Fetch actionable items
builder
.addCase(fetchActionableItems.pending, state => {
state.loading = true;
state.error = null;
})
.addCase(fetchActionableItems.fulfilled, (state, action) => {
state.loading = false;
state.items = action.payload;
state.lastUpdate = new Date();
state.error = null;
})
.addCase(fetchActionableItems.rejected, (state, action) => {
state.loading = false;
state.error = action.payload as string;
});
// Update item status
builder
.addCase(updateItemStatus.fulfilled, (state, action) => {
const { itemId, status, updatedAt } = action.payload;
const item = state.items.find(item => item.id === itemId);
if (item) {
item.status = status;
item.updatedAt = updatedAt;
// Set completion/dismissal timestamps
if (status === 'completed') {
item.completedAt = updatedAt;
} else if (status === 'dismissed') {
item.dismissedAt = updatedAt;
}
}
state.lastUpdate = new Date();
})
.addCase(updateItemStatus.rejected, (state, action) => {
state.error = action.payload as string;
});
// Snooze item
builder
.addCase(snoozeItem.fulfilled, (state, action) => {
const { itemId, snoozeUntil, updatedAt } = action.payload;
const item = state.items.find(item => item.id === itemId);
if (item) {
item.status = 'snoozed';
item.snoozeUntil = snoozeUntil;
item.updatedAt = updatedAt;
item.reminderCount = (item.reminderCount || 0) + 1;
}
state.lastUpdate = new Date();
})
.addCase(snoozeItem.rejected, (state, action) => {
state.error = action.payload as string;
});
// Create chat session
builder
.addCase(createChatSession.fulfilled, (state, action) => {
const { threadId, itemId, messages } = action.payload;
state.activeSessions[threadId] = {
threadId,
itemId,
messages,
isTyping: false,
isConnected: true,
};
state.currentChatSession = threadId;
// Update item with thread information
const item = state.items.find(item => item.id === itemId);
if (item) {
item.threadId = threadId;
}
})
.addCase(createChatSession.rejected, (state, action) => {
state.error = action.payload as string;
});
// Execute task
builder
.addCase(executeTask.fulfilled, (state, action) => {
const { executionId, sessionId, itemId, status } = action.payload;
state.activeExecutions[executionId] = {
executionId,
sessionId,
itemId,
status: status === 'started' ? 'running' : 'idle',
progress: [],
};
// Update item execution status
const item = state.items.find(item => item.id === itemId);
if (item) {
item.executionStatus = 'running';
item.currentSessionId = sessionId;
}
})
.addCase(executeTask.rejected, (state, action) => {
state.error = action.payload as string;
});
},
});
export const {
setInitialized,
setConnectionStatus,
setItems,
addItem,
removeItem,
setSourceFilter,
setPriorityFilter,
setSearchFilter,
setChatSession,
addMessage,
setTyping,
closeChatSession,
setCurrentChatSession,
setExecution,
updateExecutionProgress,
setExecutionResult,
clearExecution,
clearError,
} = intelligenceSlice.actions;
export default intelligenceSlice.reducer;
-95
View File
@@ -1,95 +0,0 @@
import { createAsyncThunk, createSlice } from '@reduxjs/toolkit';
import { inviteApi } from '../services/api/inviteApi';
import type { InviteCode } from '../types/invite';
interface InviteState {
codes: InviteCode[];
isLoading: boolean;
error: string | null;
redeemStatus: 'idle' | 'loading' | 'success' | 'error';
redeemError: string | null;
}
const initialState: InviteState = {
codes: [],
isLoading: false,
error: null,
redeemStatus: 'idle',
redeemError: null,
};
export const fetchInviteCodes = createAsyncThunk(
'invite/fetchInviteCodes',
async (_, { rejectWithValue }) => {
try {
return await inviteApi.getMyInviteCodes();
} catch (error) {
const msg =
error && typeof error === 'object' && 'error' in error
? String(error.error)
: 'Failed to fetch invite codes';
return rejectWithValue(msg);
}
}
);
export const redeemCode = createAsyncThunk(
'invite/redeemCode',
async (code: string, { dispatch, rejectWithValue }) => {
try {
const result = await inviteApi.redeemInviteCode(code);
// Re-fetch codes after successful redeem
dispatch(fetchInviteCodes());
return result;
} catch (error) {
const msg =
error && typeof error === 'object' && 'error' in error
? String(error.error)
: 'Failed to redeem invite code';
return rejectWithValue(msg);
}
}
);
const inviteSlice = createSlice({
name: 'invite',
initialState,
reducers: {
clearRedeemStatus: state => {
state.redeemStatus = 'idle';
state.redeemError = null;
},
},
extraReducers: builder => {
builder
// fetchInviteCodes
.addCase(fetchInviteCodes.pending, state => {
state.isLoading = true;
state.error = null;
})
.addCase(fetchInviteCodes.fulfilled, (state, action) => {
state.isLoading = false;
state.codes = action.payload;
})
.addCase(fetchInviteCodes.rejected, (state, action) => {
state.isLoading = false;
state.error = action.payload as string;
})
// redeemCode
.addCase(redeemCode.pending, state => {
state.redeemStatus = 'loading';
state.redeemError = null;
})
.addCase(redeemCode.fulfilled, state => {
state.redeemStatus = 'success';
})
.addCase(redeemCode.rejected, (state, action) => {
state.redeemStatus = 'error';
state.redeemError = action.payload as string;
});
},
});
export const { clearRedeemStatus } = inviteSlice.actions;
export default inviteSlice.reducer;
-117
View File
@@ -1,117 +0,0 @@
import { createAsyncThunk, createSlice } from '@reduxjs/toolkit';
import { teamApi } from '../services/api/teamApi';
import type { TeamInvite, TeamMember, TeamWithRole } from '../types/team';
interface TeamState {
teams: TeamWithRole[];
members: TeamMember[];
invites: TeamInvite[];
isLoading: boolean;
isLoadingMembers: boolean;
isLoadingInvites: boolean;
error: string | null;
}
const initialState: TeamState = {
teams: [],
members: [],
invites: [],
isLoading: false,
isLoadingMembers: false,
isLoadingInvites: false,
error: null,
};
export const fetchTeams = createAsyncThunk('team/fetchTeams', async (_, { rejectWithValue }) => {
try {
return await teamApi.getTeams();
} catch (error) {
const msg =
error && typeof error === 'object' && 'error' in error
? String(error.error)
: 'Failed to fetch teams';
return rejectWithValue(msg);
}
});
export const fetchMembers = createAsyncThunk(
'team/fetchMembers',
async (teamId: string, { rejectWithValue }) => {
try {
return await teamApi.getMembers(teamId);
} catch (error) {
const msg =
error && typeof error === 'object' && 'error' in error
? String(error.error)
: 'Failed to fetch members';
return rejectWithValue(msg);
}
}
);
export const fetchInvites = createAsyncThunk(
'team/fetchInvites',
async (teamId: string, { rejectWithValue }) => {
try {
return await teamApi.getInvites(teamId);
} catch (error) {
const msg =
error && typeof error === 'object' && 'error' in error
? String(error.error)
: 'Failed to fetch invites';
return rejectWithValue(msg);
}
}
);
const teamSlice = createSlice({
name: 'team',
initialState,
reducers: { clearTeamState: () => initialState },
extraReducers: builder => {
builder
// fetchTeams
.addCase(fetchTeams.pending, state => {
state.isLoading = true;
state.error = null;
})
.addCase(fetchTeams.fulfilled, (state, action) => {
state.isLoading = false;
state.teams = action.payload;
})
.addCase(fetchTeams.rejected, (state, action) => {
state.isLoading = false;
state.error = action.payload as string;
})
// fetchMembers
.addCase(fetchMembers.pending, state => {
state.isLoadingMembers = true;
state.error = null;
})
.addCase(fetchMembers.fulfilled, (state, action) => {
state.isLoadingMembers = false;
state.members = action.payload;
})
.addCase(fetchMembers.rejected, (state, action) => {
state.isLoadingMembers = false;
state.error = action.payload as string;
})
// fetchInvites
.addCase(fetchInvites.pending, state => {
state.isLoadingInvites = true;
state.error = null;
})
.addCase(fetchInvites.fulfilled, (state, action) => {
state.isLoadingInvites = false;
state.invites = action.payload;
})
.addCase(fetchInvites.rejected, (state, action) => {
state.isLoadingInvites = false;
state.error = action.payload as string;
});
},
});
export const { clearTeamState } = teamSlice.actions;
export default teamSlice.reducer;
-66
View File
@@ -1,66 +0,0 @@
import { createAsyncThunk, createSlice, PayloadAction } from '@reduxjs/toolkit';
import { userApi } from '../services/api/userApi';
import type { User } from '../types/api';
interface UserState {
user: User | null;
isLoading: boolean;
error: string | null;
}
const initialState: UserState = { user: null, isLoading: false, error: null };
/**
* Async thunk to fetch current user data
*/
export const fetchCurrentUser = createAsyncThunk(
'user/fetchCurrentUser',
async (_, { rejectWithValue }) => {
try {
const user = await userApi.getMe();
return user;
} catch (error) {
const errorMessage =
error && typeof error === 'object' && 'error' in error
? String(error.error)
: 'Failed to fetch user data';
return rejectWithValue(errorMessage);
}
}
);
const userSlice = createSlice({
name: 'user',
initialState,
reducers: {
setUser: (state, action: PayloadAction<User | null>) => {
state.user = action.payload;
state.error = null;
},
clearUser: state => {
state.user = null;
state.error = null;
state.isLoading = false;
},
},
extraReducers: builder => {
builder
.addCase(fetchCurrentUser.pending, state => {
state.isLoading = true;
state.error = null;
})
.addCase(fetchCurrentUser.fulfilled, (state, action) => {
state.isLoading = false;
state.user = action.payload;
state.error = null;
})
.addCase(fetchCurrentUser.rejected, (state, action) => {
state.isLoading = false;
state.error = action.payload as string;
});
},
});
export const { setUser, clearUser } = userSlice.actions;
export default userSlice.reducer;
-93
View File
@@ -1,93 +0,0 @@
import { createSlice, type PayloadAction } from '@reduxjs/toolkit';
import type { Tunnel } from '../services/api/tunnelsApi';
// ── Types ─────────────────────────────────────────────────────────────────────
/** Local tunnel-to-skill registration (from the Rust core WebhookRouter). */
export interface TunnelRegistration {
tunnel_uuid: string;
target_kind?: string;
skill_id: string;
tunnel_name: string | null;
backend_tunnel_id: string | null;
}
/** Entry in the webhook activity log. */
export interface WebhookActivityEntry {
correlation_id: string;
tunnel_name: string;
method: string;
path: string;
status_code: number | null;
skill_id: string | null;
timestamp: number;
}
interface WebhooksState {
/** Tunnels from the backend API. */
tunnels: Tunnel[];
/** Local tunnel-to-skill registrations from the Rust core. */
registrations: TunnelRegistration[];
/** Recent webhook activity (ring buffer, newest first). */
activity: WebhookActivityEntry[];
loading: boolean;
error: string | null;
}
// ── Slice ─────────────────────────────────────────────────────────────────────
const MAX_ACTIVITY_ENTRIES = 100;
const initialState: WebhooksState = {
tunnels: [],
registrations: [],
activity: [],
loading: false,
error: null,
};
const webhooksSlice = createSlice({
name: 'webhooks',
initialState,
reducers: {
setTunnels: (state, action: PayloadAction<Tunnel[]>) => {
state.tunnels = action.payload;
state.loading = false;
state.error = null;
},
addTunnel: (state, action: PayloadAction<Tunnel>) => {
state.tunnels.push(action.payload);
},
removeTunnel: (state, action: PayloadAction<string>) => {
state.tunnels = state.tunnels.filter(t => t.id !== action.payload);
},
setRegistrations: (state, action: PayloadAction<TunnelRegistration[]>) => {
state.registrations = action.payload;
},
addActivity: (state, action: PayloadAction<WebhookActivityEntry>) => {
state.activity.unshift(action.payload);
if (state.activity.length > MAX_ACTIVITY_ENTRIES) {
state.activity.splice(MAX_ACTIVITY_ENTRIES);
}
},
setLoading: (state, action: PayloadAction<boolean>) => {
state.loading = action.payload;
},
setError: (state, action: PayloadAction<string | null>) => {
state.error = action.payload;
state.loading = false;
},
},
});
export const {
setTunnels,
addTunnel,
removeTunnel,
setRegistrations,
addActivity,
setLoading,
setError,
} = webhooksSlice.actions;
export default webhooksSlice.reducer;
-6
View File
@@ -8,22 +8,16 @@ import type { PropsWithChildren, ReactElement } from 'react';
import { Provider } from 'react-redux';
import { MemoryRouter } from 'react-router-dom';
import authReducer from '../store/authSlice';
import channelConnectionsReducer from '../store/channelConnectionsSlice';
import socketReducer from '../store/socketSlice';
import teamReducer from '../store/teamSlice';
import userReducer from '../store/userSlice';
/**
* Creates a fresh Redux store for testing.
* Uses raw (non-persisted) reducers to avoid persist complexity in tests.
*/
const testRootReducer = combineReducers({
auth: authReducer,
channelConnections: channelConnectionsReducer,
socket: socketReducer,
user: userReducer,
team: teamReducer,
});
export function createTestStore(preloadedState?: Record<string, unknown>) {
+7 -4
View File
@@ -15,8 +15,6 @@ export interface AccessibilityPermissionStatus {
export interface AccessibilityFeatures {
screen_monitoring: boolean;
device_control: boolean;
predictive_input: boolean;
}
export interface AccessibilitySessionStatus {
@@ -52,6 +50,11 @@ export interface AccessibilityConfig {
denylist: string[];
}
export interface AccessibilityCoreProcessStatus {
pid: number;
started_at_ms: number;
}
export interface AccessibilityStatus {
platform_supported: boolean;
permissions: AccessibilityPermissionStatus;
@@ -62,14 +65,14 @@ export interface AccessibilityStatus {
is_context_blocked: boolean;
/** Absolute path of the core binary; macOS TCC applies to this executable. */
permission_check_process_path?: string | null;
/** Identity of the core process currently serving RPC requests. */
core_process?: AccessibilityCoreProcessStatus | null;
}
export interface AccessibilityStartSessionParams {
consent: boolean;
ttl_secs?: number;
screen_monitoring?: boolean;
device_control?: boolean;
predictive_input?: boolean;
}
export interface AccessibilityStopSessionParams {
+19
View File
@@ -41,6 +41,12 @@ export interface DaemonHostConfig {
show_tray: boolean;
}
export interface RestartStatus {
accepted: boolean;
source: string;
reason: string;
}
export async function openhumanHardwareDiscover(): Promise<CommandResponse<DiscoveredDevice[]>> {
if (!isTauri()) {
throw new Error('Not running in Tauri');
@@ -128,6 +134,19 @@ export async function openhumanServiceUninstall(): Promise<CommandResponse<Servi
}
}
export async function openhumanServiceRestart(
source?: string,
reason?: string
): Promise<CommandResponse<RestartStatus>> {
if (!isTauri()) {
throw new Error('Not running in Tauri');
}
return await callCoreRpc<CommandResponse<RestartStatus>>({
method: 'openhuman.service_restart',
params: { source, reason },
});
}
export async function openhumanAgentServerStatus(): Promise<CommandResponse<AgentServerStatus>> {
if (!isTauri()) {
throw new Error('Not running in Tauri');
+39 -12
View File
@@ -750,6 +750,41 @@ async fn run_server_inner(
Ok(())
}
/// Registers all long-lived domain event-bus subscribers exactly once.
///
/// Guarded by `std::sync::Once` so repeated calls (e.g. jsonrpc + repl paths
/// both invoking `bootstrap_skill_runtime`) are safe and idempotent.
fn register_domain_subscribers() {
use std::sync::{Arc, Once};
static REGISTERED: Once = Once::new();
REGISTERED.call_once(|| {
// Leak the SubscriptionHandle so the background tasks live for the
// entire process — SubscriptionHandle::drop aborts the task.
if let Some(handle) = crate::openhuman::event_bus::subscribe_global(Arc::new(
crate::openhuman::webhooks::bus::WebhookRequestSubscriber::new(),
)) {
std::mem::forget(handle);
} else {
log::warn!("[event_bus] failed to register webhook subscriber — bus not initialized");
}
if let Some(handle) = crate::openhuman::event_bus::subscribe_global(Arc::new(
crate::openhuman::channels::bus::ChannelInboundSubscriber::new(),
)) {
std::mem::forget(handle);
} else {
log::warn!("[event_bus] failed to register channel subscriber — bus not initialized");
}
// Restart requests go through a subscriber so every trigger path shares
// the same respawn logic.
crate::openhuman::service::bus::register_restart_subscriber();
log::info!("[event_bus] webhook, channel, and restart subscribers registered");
});
}
/// Initializes the QuickJS skill runtime, socket manager, and registers them
/// globally so RPC handlers (`openhuman.skills_*`, `openhuman.socket_*`) can
/// reach them.
@@ -790,19 +825,11 @@ pub async fn bootstrap_skill_runtime() {
// --- Event bus bootstrap ---
// Ensure the global event bus is initialized (no-op if already done by start_channels).
let bus =
crate::openhuman::event_bus::init_global(crate::openhuman::event_bus::DEFAULT_CAPACITY);
crate::openhuman::event_bus::init_global(crate::openhuman::event_bus::DEFAULT_CAPACITY);
// Register domain subscribers for cross-module event handling.
// Leak the handles so the background tasks live for the entire process —
// SubscriptionHandle::drop aborts the task, and bootstrap_skill_runtime()
// returns immediately after setup.
std::mem::forget(bus.subscribe(Arc::new(
crate::openhuman::webhooks::bus::WebhookRequestSubscriber::new(),
)));
std::mem::forget(bus.subscribe(Arc::new(
crate::openhuman::channels::bus::ChannelInboundSubscriber::new(),
)));
log::info!("[event_bus] webhook and channel subscribers registered");
// Uses a Once guard so repeated calls to bootstrap_skill_runtime() (e.g.
// from both jsonrpc and repl paths) cannot double-subscribe.
register_domain_subscribers();
// --- Socket manager bootstrap ---
let socket_mgr = Arc::new(SocketManager::new());
+2 -4
View File
@@ -393,8 +393,6 @@ fn run_start_session(args: &[String]) -> Result<()> {
consent: true,
ttl_secs: Some(opts.ttl_secs),
screen_monitoring: Some(true),
device_control: Some(false),
predictive_input: Some(false),
};
match engine.start_session(params).await {
@@ -515,13 +513,13 @@ fn run_doctor(args: &[String]) -> Result<()> {
let platform_ok = summary["platform_supported"].as_bool().unwrap_or(false);
let screen_ok = summary["screen_capture_ready"].as_bool().unwrap_or(false);
let control_ok = summary["device_control_ready"].as_bool().unwrap_or(false);
let control_ok = summary["accessibility_ready"].as_bool().unwrap_or(false);
let input_ok = summary["input_monitoring_ready"].as_bool().unwrap_or(false);
let overall_ok = summary["overall_ready"].as_bool().unwrap_or(false);
eprintln!(" {} Platform supported", check(platform_ok));
eprintln!(" {} Screen recording", check(screen_ok));
eprintln!(" {} Accessibility (device control)", check(control_ok));
eprintln!(" {} Accessibility automation", check(control_ok));
eprintln!(" {} Input monitoring", check(input_ok));
eprintln!();
+1
View File
@@ -26,5 +26,6 @@ pub use openhuman::memory::{MemoryClient, MemoryState};
///
/// Returns an error if command execution fails.
pub fn run_core_from_args(args: &[String]) -> anyhow::Result<()> {
openhuman::service::apply_startup_restart_delay_from_env();
core::cli::run_from_cli_args(args)
}
+73 -7
View File
@@ -13,9 +13,13 @@ use tempfile::NamedTempFile;
use crate::api::config::effective_api_url;
use crate::api::jwt::{bearer_authorization_value, get_session_token};
use crate::openhuman::autocomplete::AutocompleteStatus;
use crate::openhuman::config::rpc as config_rpc;
use crate::openhuman::config::Config;
use crate::openhuman::credentials::session_support::build_session_state;
use crate::openhuman::local_ai::LocalAiStatus;
use crate::openhuman::screen_intelligence::AccessibilityStatus;
use crate::openhuman::service::{ServiceState, ServiceStatus};
use crate::rpc::RpcOutcome;
const LOG_PREFIX: &str = "[app_state]";
@@ -61,6 +65,16 @@ pub struct AppStateSnapshot {
pub onboarding_completed: bool,
pub analytics_enabled: bool,
pub local_state: StoredAppState,
pub runtime: RuntimeSnapshot,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct RuntimeSnapshot {
pub screen_intelligence: AccessibilityStatus,
pub local_ai: LocalAiStatus,
pub autocomplete: AutocompleteStatus,
pub service: ServiceStatus,
}
#[derive(Debug, Clone, Deserialize, Default)]
@@ -256,23 +270,74 @@ async fn fetch_current_user(config: &Config, token: &str) -> Result<Option<Value
Ok(Some(user))
}
async fn build_runtime_snapshot(config: &Config) -> RuntimeSnapshot {
let screen_intelligence = {
let _ = crate::openhuman::screen_intelligence::global_engine()
.apply_config(config.screen_intelligence.clone())
.await;
crate::openhuman::screen_intelligence::global_engine()
.status()
.await
};
let local_ai = match crate::openhuman::local_ai::rpc::local_ai_status(config).await {
Ok(outcome) => outcome.value,
Err(error) => {
warn!("{LOG_PREFIX} local_ai status failed during snapshot: {error}");
crate::openhuman::local_ai::LocalAiStatus::disabled(config)
}
};
let autocomplete = crate::openhuman::autocomplete::global_engine()
.status()
.await;
let service = match crate::openhuman::service::status(config) {
Ok(status) => status,
Err(error) => {
let message = error.to_string();
warn!("{LOG_PREFIX} service status failed during snapshot: {message}");
ServiceStatus {
state: ServiceState::Unknown(message.clone()),
unit_path: None,
label: "OpenHuman".to_string(),
details: Some(message),
}
}
};
RuntimeSnapshot {
screen_intelligence,
local_ai,
autocomplete,
service,
}
}
pub async fn snapshot() -> Result<RpcOutcome<AppStateSnapshot>, String> {
let config = config_rpc::load_config_with_timeout().await?;
let auth = build_session_state(&config)?;
let session_token = get_session_token(&config)?;
let current_user = if let Some(token) = session_token.clone().filter(|t| !t.trim().is_empty()) {
fetch_current_user(&config, &token).await?
} else {
None
};
let current_user = auth.user.clone().or(
if let Some(token) = session_token.clone().filter(|t| !t.trim().is_empty()) {
fetch_current_user(&config, &token).await?
} else {
None
},
);
let local_state = load_stored_app_state(&config)?;
let runtime = build_runtime_snapshot(&config).await;
debug!(
"{LOG_PREFIX} snapshot auth={} onboarding={} analytics={} wallet_present={}",
"{LOG_PREFIX} snapshot auth={} onboarding={} analytics={} wallet_present={} si_active={} local_ai_state={} autocomplete_phase={} service_state={:?}",
auth.is_authenticated,
config.onboarding_completed,
config.observability.analytics_enabled,
local_state.primary_wallet_address.is_some()
local_state.primary_wallet_address.is_some(),
runtime.screen_intelligence.session.active,
runtime.local_ai.state,
runtime.autocomplete.phase,
runtime.service.state
);
Ok(RpcOutcome::new(
@@ -283,6 +348,7 @@ pub async fn snapshot() -> Result<RpcOutcome<AppStateSnapshot>, String> {
onboarding_completed: config.onboarding_completed,
analytics_enabled: config.observability.analytics_enabled,
local_state,
runtime,
},
vec!["core app state snapshot fetched".to_string()],
))
+1 -1
View File
@@ -47,7 +47,7 @@ pub fn app_state_schemas(function: &str) -> ControllerSchema {
outputs: vec![FieldSchema {
name: "result",
ty: TypeSchema::Json,
comment: "Auth, current user, core config flags, and locally persisted app state.",
comment: "Auth, current user, local app state, and compact runtime status for the React shell.",
required: true,
}],
},
+10
View File
@@ -178,6 +178,8 @@ pub enum DomainEvent {
SystemStartup { component: String },
/// A system component is shutting down.
SystemShutdown { component: String },
/// A restart of the current core process was requested.
SystemRestartRequested { source: String, reason: String },
/// A component's health status changed.
HealthChanged { component: String, healthy: bool },
}
@@ -223,6 +225,7 @@ impl DomainEvent {
Self::SystemStartup { .. }
| Self::SystemShutdown { .. }
| Self::SystemRestartRequested { .. }
| Self::HealthChanged { .. } => "system",
}
}
@@ -503,6 +506,13 @@ mod tests {
},
"system",
),
(
DomainEvent::SystemRestartRequested {
source: "rpc".into(),
reason: "test".into(),
},
"system",
),
(
DomainEvent::HealthChanged {
component: "c".into(),
+13 -12
View File
@@ -8,14 +8,14 @@
use crate::openhuman::config::ScreenIntelligenceConfig;
use std::collections::VecDeque;
use std::path::PathBuf;
use std::sync::Arc;
use std::sync::{Arc, OnceLock};
use super::capture::now_ms;
use super::helpers::push_ephemeral_frame;
use super::state::{AccessibilityEngine, EngineState, SessionRuntime};
use super::state::{AccessibilityEngine, SessionRuntime};
use super::types::{
AccessibilityStatus, AppContextInfo, CaptureFrame, CaptureImageRefResult, CaptureNowResult,
CaptureTestResult, SessionStatus, StartSessionParams,
CaptureTestResult, CoreProcessStatus, SessionStatus, StartSessionParams,
};
use crate::openhuman::accessibility::{
capture_screen_image_ref_for_context, detect_permissions, foreground_context,
@@ -26,9 +26,6 @@ use crate::openhuman::accessibility::{
open_macos_privacy_pane, request_accessibility_access, request_screen_recording_access,
};
// Re-export for backward compat.
pub use super::state::{global_engine, AccessibilityEngine as _AccessibilityEngineAlias};
impl AccessibilityEngine {
// ── Config ───────────────────────────────────────────────────────
@@ -39,7 +36,6 @@ impl AccessibilityEngine {
{
let mut state = self.inner.lock().await;
state.config = config.clone();
state.features.predictive_input = state.config.autocomplete_enabled;
}
if config.enabled {
@@ -73,7 +69,6 @@ impl AccessibilityEngine {
let now = now_ms();
state.features.screen_monitoring = true;
state.features.predictive_input = state.config.autocomplete_enabled;
state.session = Some(new_session_runtime(&state.config, now, i64::MAX, 0));
state.last_event = Some("screen_intelligence_enabled".to_string());
state.last_error = None;
@@ -126,10 +121,6 @@ impl AccessibilityEngine {
let now = now_ms();
let expires_at_ms = now + (ttl_secs as i64 * 1000);
state.features.screen_monitoring = screen_monitoring_requested;
state.features.device_control = params.device_control.unwrap_or(true);
state.features.predictive_input = params
.predictive_input
.unwrap_or(state.config.autocomplete_enabled);
state.session = Some(new_session_runtime(
&state.config,
@@ -343,6 +334,10 @@ impl AccessibilityEngine {
permission_check_process_path: std::env::current_exe()
.ok()
.map(|p| p.display().to_string()),
core_process: Some(CoreProcessStatus {
pid: std::process::id(),
started_at_ms: core_process_started_at_ms(),
}),
}
}
@@ -553,6 +548,11 @@ impl AccessibilityEngine {
}
}
fn core_process_started_at_ms() -> i64 {
static CORE_PROCESS_STARTED_AT_MS: OnceLock<i64> = OnceLock::new();
*CORE_PROCESS_STARTED_AT_MS.get_or_init(now_ms)
}
// ── Helpers ─────────────────────────────────────────────────────────────
fn new_session_runtime(
@@ -589,6 +589,7 @@ fn new_session_runtime(
#[cfg(test)]
mod tests {
use super::*;
use crate::openhuman::screen_intelligence::state::EngineState;
use tokio::sync::Mutex;
use tokio::time::Duration;
+3 -11
View File
@@ -1,4 +1,4 @@
//! Input actions and autocomplete — keyboard/mouse automation + predictive text.
//! Input actions and autocomplete helpers for the screen intelligence session.
use super::helpers::{generate_suggestions, truncate_tail, validate_input_action};
use super::limits::{MAX_CONTEXT_CHARS, MAX_SUGGESTION_CHARS};
@@ -34,14 +34,6 @@ impl AccessibilityEngine {
});
}
if !state.features.device_control {
return Ok(InputActionResult {
accepted: false,
blocked: true,
reason: Some("device control is disabled".to_string()),
});
}
let context = foreground_context();
if let Some(ctx) = &context {
if !self.should_capture_context(ctx, &state.config) {
@@ -82,7 +74,7 @@ impl AccessibilityEngine {
) -> Result<AutocompleteSuggestResult, String> {
let state = self.inner.lock().await;
if !state.features.predictive_input {
if !state.config.autocomplete_enabled {
return Ok(AutocompleteSuggestResult {
suggestions: Vec::new(),
});
@@ -113,7 +105,7 @@ impl AccessibilityEngine {
}
let mut state = self.inner.lock().await;
if !state.features.predictive_input {
if !state.config.autocomplete_enabled {
return Ok(AutocompleteCommitResult { committed: false });
}
if !state.autocomplete_context.is_empty() {
+1 -1
View File
@@ -73,7 +73,7 @@ pub async fn accessibility_doctor_cli_json() -> Result<serde_json::Value, String
"platform_supported": status.platform_supported,
"session_active": status.session.active,
"screen_capture_ready": screen_ready,
"device_control_ready": control_ready,
"accessibility_ready": control_ready,
"input_monitoring_ready": monitoring_ready
},
"permissions": permissions,
@@ -128,8 +128,6 @@ impl SiServer {
consent: true,
ttl_secs: Some(self.config.ttl_secs),
screen_monitoring: Some(true),
device_control: Some(false),
predictive_input: Some(false),
};
match self.engine.start_session(params).await {
@@ -54,8 +54,6 @@ impl EngineState {
},
features: AccessibilityFeatures {
screen_monitoring: true,
device_control: true,
predictive_input: config.autocomplete_enabled,
},
config,
session: None,
@@ -493,8 +493,6 @@ async fn session_lifecycle_transitions_and_ttl_expiry() {
consent: true,
ttl_secs: Some(1),
screen_monitoring: Some(true),
device_control: Some(true),
predictive_input: Some(true),
})
.await;
@@ -535,8 +533,6 @@ async fn panic_stop_behavior_stops_session() {
consent: true,
ttl_secs: Some(60),
screen_monitoring: Some(true),
device_control: Some(true),
predictive_input: Some(true),
})
.await;
@@ -580,8 +576,6 @@ async fn capture_scheduler_adds_baseline_frames() {
consent: true,
ttl_secs: Some(2),
screen_monitoring: Some(true),
device_control: Some(true),
predictive_input: Some(true),
})
.await;
+9 -4
View File
@@ -8,8 +8,6 @@ pub use crate::openhuman::accessibility::{PermissionKind, PermissionState, Permi
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct AccessibilityFeatures {
pub screen_monitoring: bool,
pub device_control: bool,
pub predictive_input: bool,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
@@ -42,6 +40,12 @@ pub struct AccessibilityHealth {
pub last_event: Option<String>,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct CoreProcessStatus {
pub pid: u32,
pub started_at_ms: i64,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct AccessibilityStatus {
pub platform_supported: bool,
@@ -56,6 +60,9 @@ pub struct AccessibilityStatus {
/// show this so users enable the same binary in System Settings (see GH #133).
#[serde(skip_serializing_if = "Option::is_none")]
pub permission_check_process_path: Option<String>,
/// Identity of the current core process so the UI can verify that a restart actually happened.
#[serde(skip_serializing_if = "Option::is_none")]
pub core_process: Option<CoreProcessStatus>,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
@@ -63,8 +70,6 @@ pub struct StartSessionParams {
pub consent: bool,
pub ttl_secs: Option<u64>,
pub screen_monitoring: Option<bool>,
pub device_control: Option<bool>,
pub predictive_input: Option<bool>,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
+102
View File
@@ -0,0 +1,102 @@
use std::sync::{
atomic::{AtomicBool, Ordering},
Arc, OnceLock,
};
use async_trait::async_trait;
use crate::openhuman::event_bus::{DomainEvent, EventHandler, SubscriptionHandle};
/// Holds the single process-lifetime subscription handle so it is never
/// double-registered and never dropped (which would abort the task).
static RESTART_HANDLE: OnceLock<SubscriptionHandle> = OnceLock::new();
/// Register the [`RestartSubscriber`] on the global event bus.
///
/// Idempotent: subsequent calls return immediately if the subscriber is already
/// registered. Owned by the service domain — called from the shared subscriber
/// bootstrap so jsonrpc.rs stays transport-focused.
pub fn register_restart_subscriber() {
if RESTART_HANDLE.get().is_some() {
return;
}
match crate::openhuman::event_bus::subscribe_global(Arc::new(RestartSubscriber)) {
Some(handle) => {
// Store the handle; OnceLock ensures at most one wins if there is a
// race between two threads calling this function concurrently.
let _ = RESTART_HANDLE.set(handle);
}
None => {
log::warn!("[event_bus] failed to register restart subscriber — bus not initialized");
}
}
}
/// One-shot gate so only the first restart event actually spawns a replacement
/// process. Subsequent events are ignored (the process is about to exit).
static RESTART_IN_PROGRESS: AtomicBool = AtomicBool::new(false);
/// Long-lived event-bus subscriber that turns restart requests into a real
/// process respawn.
///
/// This subscriber is registered during core bootstrap so any restart
/// request published from RPC, CLI, or another internal component goes through
/// the same execution path.
pub struct RestartSubscriber;
#[async_trait]
impl EventHandler for RestartSubscriber {
fn name(&self) -> &str {
"service::restart"
}
fn domains(&self) -> Option<&[&str]> {
Some(&["system"])
}
async fn handle(&self, event: &DomainEvent) {
let DomainEvent::SystemRestartRequested { source, reason } = event else {
return;
};
// Atomically claim the restart slot — only the first caller proceeds.
if RESTART_IN_PROGRESS
.compare_exchange(false, true, Ordering::SeqCst, Ordering::SeqCst)
.is_err()
{
log::debug!(
"[service:restart] ignoring duplicate restart request source={} (restart already in progress)",
source,
);
return;
}
log::warn!(
"[service:restart] executing restart request source={}",
source
);
match crate::openhuman::service::restart::trigger_self_restart_now(source, reason) {
Ok(child_pid) => {
log::warn!(
"[service:restart] replacement pid={} spawned; exiting current process",
child_pid
);
// Brief 150ms grace period before exit: allows in-flight log
// flushes and the replacement process to bind its listener before
// this process terminates. Empirically tuned — increase if logs
// are truncated on shutdown.
tokio::spawn(async move {
tokio::time::sleep(std::time::Duration::from_millis(150)).await;
std::process::exit(0);
});
}
Err(err) => {
log::error!("[service:restart] failed to restart current process: {err}");
// Reset the gate so a subsequent attempt can try again.
RESTART_IN_PROGRESS.store(false, Ordering::SeqCst);
}
}
}
}
+4
View File
@@ -1,9 +1,11 @@
//! Service management helpers for OpenHuman daemon.
pub mod bus;
mod core;
pub mod daemon;
pub mod daemon_host;
pub mod ops;
mod restart;
mod schemas;
mod common;
@@ -18,6 +20,8 @@ mod windows;
pub use core::*;
pub use ops as rpc;
pub use ops::*;
pub use restart::apply_startup_restart_delay_from_env;
pub use restart::RestartStatus;
pub use schemas::{
all_controller_schemas as all_service_controller_schemas,
all_registered_controllers as all_service_registered_controllers,
+7
View File
@@ -25,6 +25,13 @@ pub async fn service_status(config: &Config) -> Result<RpcOutcome<ServiceStatus>
Ok(RpcOutcome::single_log(status, "service status fetched"))
}
pub async fn service_restart(
source: Option<String>,
reason: Option<String>,
) -> Result<RpcOutcome<service::RestartStatus>, String> {
service::restart::service_restart(source, reason).await
}
pub async fn service_uninstall(config: &Config) -> Result<RpcOutcome<ServiceStatus>, String> {
let status = service::uninstall(config).map_err(|e| e.to_string())?;
Ok(RpcOutcome::single_log(
+210
View File
@@ -0,0 +1,210 @@
//! Core self-restart orchestration for the service domain.
//!
//! This module intentionally splits restart into two phases:
//! 1. RPC/CLI acknowledges the request and publishes an event.
//! 2. A long-lived event-bus subscriber performs the actual respawn and exit.
//!
//! Keeping the side effect behind the event bus lets JSON-RPC, CLI, and any
//! future in-process trigger share one restart path with the same logging.
use std::process::{Command, Stdio};
use std::sync::atomic::{AtomicBool, Ordering};
use std::time::Duration;
use serde::Serialize;
use crate::openhuman::event_bus::{self, DomainEvent};
use crate::rpc::RpcOutcome;
const RESTART_DELAY_ENV: &str = "OPENHUMAN_RESTART_DELAY_MS";
const DEFAULT_RESTART_DELAY_MS: u64 = 350;
static RESTART_IN_PROGRESS: AtomicBool = AtomicBool::new(false);
/// JSON-serializable acknowledgement returned to CLI / JSON-RPC callers before
/// the current process exits.
#[derive(Debug, Clone, Serialize)]
pub struct RestartStatus {
pub accepted: bool,
pub source: String,
pub reason: String,
}
/// Applies a short delay to a freshly respawned process.
///
/// The replacement child starts before the old process exits so the restart can
/// be initiated from inside the running server. A small delay reduces bind-race
/// failures on the HTTP port while the old process is still releasing sockets.
pub fn apply_startup_restart_delay_from_env() {
let Some(raw) = std::env::var(RESTART_DELAY_ENV)
.ok()
.filter(|value| !value.trim().is_empty())
else {
return;
};
match raw.parse::<u64>() {
Ok(delay_ms) => {
eprintln!(
"[service:restart] delaying restarted process startup by {}ms",
delay_ms
);
std::thread::sleep(Duration::from_millis(delay_ms));
}
Err(err) => {
eprintln!(
"[service:restart] ignoring invalid {}='{}': {}",
RESTART_DELAY_ENV, raw, err
);
}
}
}
/// Accepts a restart request and publishes it to the global event bus.
///
/// This function does not kill or respawn the process directly; that work is
/// performed by [`crate::openhuman::service::bus::RestartSubscriber`] so every
/// in-process trigger uses the same restart execution path.
pub async fn service_restart(
source: Option<String>,
reason: Option<String>,
) -> Result<RpcOutcome<RestartStatus>, String> {
let source = source
.map(|value| value.trim().to_string())
.filter(|value| !value.is_empty())
.unwrap_or_else(|| "jsonrpc".to_string());
let reason = reason
.map(|value| value.trim().to_string())
.filter(|value| !value.is_empty())
.unwrap_or_else(|| "service.restart".to_string());
event_bus::init_global(event_bus::DEFAULT_CAPACITY);
log::info!(
"[service:restart] accepted restart request source={} reason={}",
source,
reason
);
event_bus::publish_global(DomainEvent::SystemRestartRequested {
source: source.clone(),
reason: reason.clone(),
});
Ok(RpcOutcome::single_log(
RestartStatus {
accepted: true,
source,
reason,
},
"service restart requested",
))
}
/// Starts the replacement process for the current core instance.
///
/// The caller is responsible for exiting the current process after this returns
/// successfully.
pub fn trigger_self_restart_now(source: &str, reason: &str) -> Result<u32, String> {
if RESTART_IN_PROGRESS.swap(true, Ordering::SeqCst) {
return Err("restart already in progress".to_string());
}
match spawn_restart_child() {
Ok(child_pid) => {
log::info!(
"[service:restart] spawned replacement process pid={} source={} reason={}",
child_pid,
source,
reason
);
Ok(child_pid)
}
Err(err) => {
RESTART_IN_PROGRESS.store(false, Ordering::SeqCst);
Err(err)
}
}
}
/// Respawns the current executable with the original argument list.
///
/// This preserves the launch mode the user already chose, for example
/// `openhuman run --jsonrpc-only` or another long-lived server mode.
fn spawn_restart_child() -> Result<u32, String> {
let current_exe = std::env::current_exe().map_err(|e| format!("current_exe failed: {e}"))?;
let args: Vec<String> = std::env::args().skip(1).collect();
if args.is_empty() {
return Err("cannot self-restart without original launch arguments".to_string());
}
log::debug!(
"[service:restart] respawning exe={} args={:?}",
current_exe.display(),
args
);
let child = Command::new(&current_exe)
.args(&args)
.env(RESTART_DELAY_ENV, DEFAULT_RESTART_DELAY_MS.to_string())
.stdin(Stdio::null())
.stdout(Stdio::inherit())
.stderr(Stdio::inherit())
.spawn()
.map_err(|e| format!("failed to spawn replacement process: {e}"))?;
Ok(child.id())
}
#[cfg(test)]
mod tests {
use super::*;
use async_trait::async_trait;
use std::sync::Arc;
use tokio::sync::mpsc;
use tokio::time::{timeout, Duration};
struct RestartProbe {
tx: mpsc::UnboundedSender<(String, String)>,
}
#[async_trait]
impl crate::openhuman::event_bus::EventHandler for RestartProbe {
fn name(&self) -> &str {
"service::restart_probe"
}
fn domains(&self) -> Option<&[&str]> {
Some(&["system"])
}
async fn handle(&self, event: &crate::openhuman::event_bus::DomainEvent) {
if let crate::openhuman::event_bus::DomainEvent::SystemRestartRequested {
source,
reason,
} = event
{
let _ = self.tx.send((source.clone(), reason.clone()));
}
}
}
#[tokio::test]
async fn service_restart_publishes_restart_event() {
let bus = event_bus::init_global(event_bus::DEFAULT_CAPACITY);
let (tx, mut rx) = mpsc::unbounded_channel();
let handle = bus.subscribe(Arc::new(RestartProbe { tx }));
let outcome = service_restart(Some("test".into()), Some("integration".into()))
.await
.expect("restart request should succeed");
assert!(outcome.value.accepted);
let event = timeout(Duration::from_secs(1), rx.recv())
.await
.expect("restart event should arrive")
.expect("probe channel should stay open");
assert_eq!(event.0, "test");
assert_eq!(event.1, "integration");
handle.cancel();
}
}
+56 -4
View File
@@ -9,6 +9,7 @@ use crate::rpc::RpcOutcome;
pub fn all_controller_schemas() -> Vec<ControllerSchema> {
vec![
schemas("install"),
schemas("restart"),
schemas("start"),
schemas("stop"),
schemas("status"),
@@ -24,6 +25,10 @@ pub fn all_registered_controllers() -> Vec<RegisteredController> {
schema: schemas("install"),
handler: handle_install,
},
RegisteredController {
schema: schemas("restart"),
handler: handle_restart,
},
RegisteredController {
schema: schemas("start"),
handler: handle_start,
@@ -53,21 +58,47 @@ pub fn all_registered_controllers() -> Vec<RegisteredController> {
pub fn schemas(function: &str) -> ControllerSchema {
match function {
"install" | "start" | "stop" | "status" | "uninstall" => ControllerSchema {
"install" | "restart" | "start" | "stop" | "status" | "uninstall" => ControllerSchema {
namespace: "service",
function: match function {
"install" => "install",
"restart" => "restart",
"start" => "start",
"stop" => "stop",
"status" => "status",
_ => "uninstall",
},
description: "Manage desktop service lifecycle.",
inputs: vec![],
inputs: if function == "restart" {
vec![
FieldSchema {
name: "source",
ty: TypeSchema::String,
comment: "Optional caller label for restart diagnostics.",
required: false,
},
FieldSchema {
name: "reason",
ty: TypeSchema::String,
comment: "Optional free-form reason for the restart request.",
required: false,
},
]
} else {
vec![]
},
outputs: vec![FieldSchema {
name: "status",
ty: TypeSchema::Ref("ServiceStatus"),
comment: "Service status payload.",
ty: if function == "restart" {
TypeSchema::Json
} else {
TypeSchema::Ref("ServiceStatus")
},
comment: if function == "restart" {
"Restart request acknowledgement payload."
} else {
"Service status payload."
},
required: true,
}],
},
@@ -129,6 +160,27 @@ fn handle_start(_params: Map<String, Value>) -> ControllerFuture {
})
}
#[derive(Debug, Deserialize)]
struct ServiceRestartParams {
source: Option<String>,
reason: Option<String>,
}
/// Service restart is intentionally config-free.
///
/// Unlike install/start/stop/status, the restart action targets the currently
/// running core process itself, so it only needs restart metadata and not the
/// persisted service config.
fn handle_restart(params: Map<String, Value>) -> ControllerFuture {
Box::pin(async move {
let payload: ServiceRestartParams =
serde_json::from_value(Value::Object(params)).map_err(|e| e.to_string())?;
to_json(
crate::openhuman::service::rpc::service_restart(payload.source, payload.reason).await?,
)
})
}
fn handle_stop(_params: Map<String, Value>) -> ControllerFuture {
Box::pin(async move {
let config = config_rpc::load_config_with_timeout().await?;
+61
View File
@@ -891,6 +891,67 @@ async fn json_rpc_screen_intelligence_status_returns_stable_shape() {
rpc_join.abort();
}
#[tokio::test]
async fn json_rpc_app_state_snapshot_returns_runtime_shape() {
let _env_lock = json_rpc_e2e_env_lock();
let tmp = tempdir().expect("tempdir");
let home = tmp.path();
let openhuman_home = home.join(".openhuman");
let _home_guard = EnvVarGuard::set_to_path("HOME", home);
let _workspace_guard = EnvVarGuard::unset("OPENHUMAN_WORKSPACE");
let _backend_url_guard = EnvVarGuard::unset("BACKEND_URL");
let _vite_backend_guard = EnvVarGuard::unset("VITE_BACKEND_URL");
let (mock_addr, mock_join) = serve_on_ephemeral(mock_upstream_router()).await;
let mock_origin = format!("http://{}", mock_addr);
write_min_config(&openhuman_home, &mock_origin);
let (rpc_addr, rpc_join) = serve_on_ephemeral(build_core_http_router(false)).await;
let rpc_base = format!("http://{}", rpc_addr);
tokio::time::sleep(Duration::from_millis(100)).await;
let snapshot = post_json_rpc(&rpc_base, 1004, "openhuman.app_state_snapshot", json!({})).await;
let result = assert_no_jsonrpc_error(&snapshot, "app_state_snapshot");
let body = result.get("result").unwrap_or(result);
assert!(
body.get("auth").and_then(Value::as_object).is_some(),
"expected auth object: {body}"
);
assert!(
body.get("localState").and_then(Value::as_object).is_some(),
"expected localState object: {body}"
);
let runtime = body.get("runtime").expect("expected runtime object");
assert!(
runtime
.get("screenIntelligence")
.and_then(Value::as_object)
.is_some(),
"expected runtime.screenIntelligence object: {runtime}"
);
assert!(
runtime.get("localAi").and_then(Value::as_object).is_some(),
"expected runtime.localAi object: {runtime}"
);
assert!(
runtime
.get("autocomplete")
.and_then(Value::as_object)
.is_some(),
"expected runtime.autocomplete object: {runtime}"
);
assert!(
runtime.get("service").and_then(Value::as_object).is_some(),
"expected runtime.service object: {runtime}"
);
mock_join.abort();
rpc_join.abort();
}
#[tokio::test]
async fn json_rpc_screen_intelligence_vision_recent_returns_empty_without_session() {
let _env_lock = json_rpc_e2e_env_lock();