Files
openhuman/CLAUDE.md
T
Steven EnamakelandGitHub 14dc860a21 Skills runtime, onboarding, deep links, and core RPC refinements (#95)
* feat(telegram): implement Telegram channel and attachment handling

- Added `TelegramChannel` struct for managing Telegram Bot API interactions, including user management and message handling.
- Introduced attachment parsing with `TelegramAttachment` and `TelegramAttachmentKind` to support various media types.
- Implemented functions for parsing attachment markers and validating URLs, enhancing message processing capabilities.
- Created a new module structure for Telegram, including `attachments`, `channel`, and `text` for better organization and maintainability.

* feat(skills): implement skills registry management and E2E testing

- Added functionality for fetching, searching, installing, and uninstalling skills from a remote registry.
- Introduced new modules for registry operations and types, enhancing the skills management system.
- Implemented E2E tests for skills registry interactions, ensuring robust functionality and integration.
- Updated documentation to reflect new skills registry features and usage instructions.

* refactor(coreRpcClient): remove socket RPC handling and streamline HTTP request logging

- Eliminated socket-based RPC handling to simplify the core RPC client logic.
- Updated logging to use a unified debug logger for both HTTP requests and errors.
- Improved error handling for HTTP responses to ensure clarity in error reporting.

* feat(skills): enhance skill setup handling and improve error management

- Updated SkillActionButton to directly open the setup modal for skills requiring OAuth, bypassing the QuickJS runtime.
- Enhanced SkillSetupWizard to handle OAuth configuration more effectively, ensuring smoother transitions during skill setup.
- Improved error handling during skill startup and setup processes, providing clearer logging for failures.
- Refactored skills loading logic in the Skills page to prioritize registry-based skill fetching, with fallback to runtime discovery.
- Added skill installation handling in the Skills page, allowing for better user feedback during installation processes.

* feat(deep-link): enhance OAuth handling and streamline token management

- Updated desktopDeepLinkListener to improve skill connection handling after OAuth completion.
- Introduced setSkillSetupComplete action to mark skills as connected immediately post-OAuth.
- Refactored token fetching logic to ensure encrypted tokens are stored correctly, enhancing error handling and reducing redundant checks.
- Added new permissions in default.json for improved window management capabilities.

* feat(skills): enhance skill management with global engine and runtime controllers

- Added global engine management for skill runtime access, allowing RPC handlers to interact with the runtime engine.
- Introduced new runtime controllers for skills, including start, stop, status, setup_start, list_tools, sync, and call_tool, enhancing skill lifecycle management.
- Updated schemas to include new skill controller functionalities, improving the overall skills management system.
- Enhanced documentation and comments for clarity on new features and usage.

* refactor(skills): update global engine management and enhance documentation

- Replaced OnceLock with RwLock for the global RuntimeEngine, allowing for better testability and flexibility in engine management.
- Updated the global_engine and require_engine functions to return cloned Arc references, improving usability.
- Enhanced documentation comments for clarity on the global engine's usage and behavior in production and testing scenarios.

* chore(todos): update TODO list with removal of Tauri from Rust core

- Added a new item to the TODO list indicating the need to remove Tauri from the OpenHuman Rust core, streamlining the project structure.

* feat(onboarding): revamp onboarding steps and introduce local AI model consent

- Replaced the PrivacyStep with a new ScreenPermissionsStep to handle accessibility permissions.
- Added LocalAIStep for user consent on local AI model usage and download initiation.
- Introduced SkillsStep and ToolsStep for selecting skills and enabling tools during onboarding.
- Updated onboarding state management to include local model consent, download status, and enabled tools.
- Enhanced the overall onboarding flow with new components and improved user experience.

* feat(onboarding): enhance onboarding flow with new WelcomeStep and updated LocalAIStep

- Introduced a new WelcomeStep to guide users through the onboarding process.
- Updated LocalAIStep to clarify local AI model usage and consent, including improved messaging on privacy and resource impact.
- Enhanced ScreenPermissionsStep to emphasize local processing of accessibility data.
- Adjusted total steps in onboarding to reflect the addition of the WelcomeStep, improving user experience.

* refactor(tray): remove tray integration and related functionalities

- Deleted the tray module and its associated operations, streamlining the project structure.
- Removed references to Tauri app handle in various components, transitioning to a memory client for skill data persistence.
- Updated skill instances and event loops to eliminate dependencies on tray functionalities, enhancing modularity.
- Improved documentation to reflect the removal of tray-related features and clarify the new architecture.

* feat(onboarding): introduce OnboardingOverlay and enhance onboarding flow

- Added OnboardingOverlay component to display the onboarding process as a full-screen overlay when the user is not onboarded.
- Updated the Onboarding component to include a new MnemonicStep for recovery phrase management.
- Enhanced onboarding state management to track workspace onboarding flags and user onboarding status.
- Refactored AppRoutes to streamline routing and integrate the new onboarding flow.
- Removed deprecated onboarding logic from previous steps, improving overall user experience.

* refactor(sidebar): simplify hidden paths and update ProtectedRoute tests

- Removed '/onboarding' from the hiddenPaths in MiniSidebar to streamline route visibility.
- Updated ProtectedRoute tests to reflect changes in onboarding handling, ensuring children render correctly when authenticated.

* chore: format, fix E2E lint, and onboarding step polish

Made-with: Cursor

* style(onboarding): update background color for onboarding steps

- Changed background color from black/30 to stone-900 for improved visual consistency across LocalAIStep, MnemonicStep, ScreenPermissionsStep, SkillsStep, ToolsStep, and WelcomeStep components.
- Enhanced overall aesthetics of the onboarding flow.

* fix(tests): update variable naming and comment out unused JavaScript content

- Renamed workspace variable to `_ws` to indicate it is unused in the `test_registry_cache_ttl_expired` test.
- Commented out the `js_content` variable to prevent unused variable warnings in the test setup.

* refactor(tests): streamline JSON-RPC test setup and remove unused backend URL handling

- Updated the JSON-RPC end-to-end test to always use the in-process Axum mock for backend settings, ensuring consistent test behavior.
- Removed the conditional logic for external backend URLs, simplifying the test setup.
- Ensured proper cleanup of mock join handles after test execution.

* refactor(runtime): update skill startup process to use core RPC

- Replaced the direct call to `runtimeStartSkill` with a `callCoreRpc` method for starting skills, enhancing the integration with the core RPC system.
- Updated comments to reflect the new implementation details.
- Made minor adjustments to the schema organization in Rust for better clarity on runtime controllers.
2026-03-30 14:32:18 -07:00

20 KiB
Raw Blame History

OpenHuman

AI-powered assistant for communities — React + Tauri v2 desktop app with a Rust core (JSON-RPC / CLI) and sandboxed QuickJS skills.

This file orients contributors and coding agents. Authoritative narrative architecture: docs/ARCHITECTURE.md. Frontend layout: docs/src/README.md. Tauri shell: docs/src-tauri/README.md.


Repository layout

Path Role
app/ Yarn workspace openhuman-app: Vite + React (app/src/), Tauri desktop host (app/src-tauri/), Vitest tests
Repo root src/ Rust library openhuman_core and openhuman CLI binary entrypoint (src/main.rs) — core_server, openhuman::* domains, skills runtime (QuickJS / rquickjs), MCP routing in the core process
Skills registry tinyhumansai/openhuman-skills on GitHub — canonical skill packages and TS build; not vendored in this tree (see blurb below).
Cargo.toml (root) Core crate; cargo build --bin openhuman produces the sidecar the UI stages via apps core:stage
docs/ Architecture and module guides (numbered pages under docs/src/, docs/src-tauri/)

Commands in documentation assume the repo root unless noted: yarn dev runs the app workspace.

Skills registry: Skill sources and the bundler live in github.com/tinyhumansai/openhuman-skills. Clone that repository to author or change skills (yarn install, yarn build). The desktop apps skills catalog defaults to that GitHub slug; override with VITE_SKILLS_GITHUB_REPO (see app/src/utils/config.ts).


Runtime scope

  • Shipped product: desktop — Windows, macOS, Linux (see docs/ARCHITECTURE.md “Platform reach”).
  • Tauri host (app/src-tauri): desktop-only (compile_error! for non-desktop targets). Do not add Android/iOS branches inside app/src-tauri.
  • Core binary (openhuman): spawned/staged as a sidecar; the Web UI talks to it over HTTP (core_rpc_relay + core_rpc client), not by re-implementing domain logic in the shell.

Where logic lives

  • Rust (openhuman / repo root src/): Business logic and execution—domains, skills runtime, RPC, persistence, and CLI behavior. This is the authoritative place for rules and side effects.
  • Tauri + React (app/): Interaction and UX—screens, navigation, input, accessibility, windowing, and bridging to the core. The shell presents and orchestrates; it does not duplicate core business rules.

Commands (from repository root)

# Frontend + Tauri dev (workspace delegates to app/)
yarn dev

# Desktop with Tauri (loads env via scripts/load-dotenv.sh)
yarn tauri dev

# Production UI build (app workspace)
yarn build

# Typecheck / lint / format (app workspace)
yarn typecheck
yarn lint
yarn format
yarn format:check

# Stage openhuman core binary next to Tauri resources (required for core RPC)
cd app && yarn core:stage

# Skills — develop in the GitHub registry repo, then build (see tinyhumansai/openhuman-skills).
# If you keep a local clone path wired in app scripts, you can also run:
yarn workspace openhuman-app skills:build
yarn workspace openhuman-app skills:watch

# Rust — core library + CLI (repo root)
cargo check --manifest-path Cargo.toml
cargo build --manifest-path Cargo.toml --bin openhuman

# Rust — Tauri shell only
cargo check --manifest-path app/src-tauri/Cargo.toml

Tests: Vitest in app/ (yarn test, yarn test:coverage). Rust tests via cargo test at repo root as wired in app/package.json.

Quality: ESLint + Prettier + Husky in the app workspace.


Testing Guide (Unit + E2E)

Unit tests (Vitest)

  • Where tests live: co-locate as *.test.ts / *.test.tsx under app/src/**.
  • Runner/config: Vitest with app/test/vitest.config.ts and shared setup in app/src/test/setup.ts.
  • Run:
yarn test:unit
yarn test:coverage
  • Authoring rules:
    • Prefer testing behavior over implementation details.
    • Use existing helpers from app/src/test/ (test-utils.tsx, shared mock backend) before adding new harness code.
    • Keep tests deterministic: avoid real network calls, time-sensitive flakes, or hidden global state.

Shared mock backend (app + Rust tests)

  • Core implementation: scripts/mock-api-core.mjs
  • Standalone server entrypoint: scripts/mock-api-server.mjs
  • E2E wrapper: app/test/e2e/mock-server.ts
  • Vitest unit setup: app/src/test/setup.ts starts the shared mock server by default on http://127.0.0.1:5005.

Key admin endpoints:

  • GET /__admin/health
  • POST /__admin/reset
  • POST /__admin/behavior
  • GET /__admin/requests

Run manually:

yarn mock:api
curl -s http://127.0.0.1:18473/__admin/health

E2E tests (WDIO + Appium mac2)

  • Where specs live: app/test/e2e/specs/*.spec.ts

  • Shared harness:

    • Helpers: app/test/e2e/helpers/*
    • Mock backend: app/test/e2e/mock-server.ts
    • WDIO config: app/test/wdio.conf.ts
  • Build + run:

# Build desktop app bundle + stage core sidecar
yarn test:e2e:build

# Run one spec
bash app/scripts/e2e-run-spec.sh test/e2e/specs/smoke.spec.ts smoke

# Run all flow specs
yarn test:e2e:all:flows
  • Authoring rules:
    • Ensure each spec is runnable in isolation.
    • Use helper waits (waitForAppReady, waitForWebView, etc.) instead of ad hoc long sleeps.
    • Assert both UI outcomes and backend/mock effects when relevant.
    • Add failure diagnostics (request logs, accessibility tree dump) for faster debugging by agents.

Deterministic core-sidecar reset

By default, app/scripts/e2e-run-spec.sh creates and cleans a temp OPENHUMAN_WORKSPACE automatically when the variable is not provided.

If you need a fixed workspace for debugging, provide one explicitly:

export OPENHUMAN_WORKSPACE="$(mktemp -d)"
yarn test:e2e:build
bash app/scripts/e2e-run-spec.sh test/e2e/specs/smoke.spec.ts smoke
rm -rf "$OPENHUMAN_WORKSPACE"
  • OPENHUMAN_WORKSPACE redirects core config + workspace storage away from ~/.openhuman.
  • Default reset strategy:
    • Rebuild/stage sidecar once per E2E run (yarn test:e2e:build).
    • Isolate state per test case with a fresh temp workspace (default behavior in e2e-run-spec.sh).

Rust tests with mock backend

Use the shared mock backend runner so Rust unit/integration tests get deterministic API behavior:

yarn test:rust
# or targeted
bash scripts/test-rust-with-mock.sh --test json_rpc_e2e

Example per-test-case pattern inside a harness script:

run_case() {
  export OPENHUMAN_WORKSPACE="$(mktemp -d)"
  bash app/scripts/e2e-run-spec.sh "$1" "$2"
  rm -rf "$OPENHUMAN_WORKSPACE"
}

Test authoring checklist

  • Add/update unit tests for logic changes before stacking additional features.
  • Add/update E2E coverage for user-visible flows and cross-process integration behavior.
  • Keep new tests independent, deterministic, and debuggable from logs alone.
  • When touching core/sidecar behavior, validate both:
    • yarn test:unit
    • targeted E2E spec(s) via app/scripts/e2e-run-spec.sh

Frontend (app/src/)

Provider chain (app/src/App.tsx)

Order matters for auth and realtime:

Redux ProviderPersistGateUserProviderSocketProviderAIProviderSkillProviderHashRouterAppRoutes.

There is no TelegramProvider in the current tree; Telegram may appear in UI copy or legacy settings, but MTProto is not an active provider here.

State (app/src/store/)

Redux Toolkit slices include auth, user, socket, ai, skills, team, and related modules. Prefer Redux (and persist where configured) over ad hoc localStorage for app state; see project rules for exceptions.

Services (app/src/services/)

Singleton-style modules include apiClient, socketService, coreRpcClient (HTTP bridge to the core process), and domain api/* clients. There is no mtprotoService in this tree.

MCP (app/src/lib/mcp/)

Transport, validation, and types for JSON-RPC-style messaging over Socket.io — not a large Telegram tool pack. Tooling for agents is driven by the skills system and backend; see agentToolRegistry.ts and core RPC.

Routing (app/src/AppRoutes.tsx)

Hash routes include /, /onboarding, /mnemonic, /home, /intelligence, /skills, /conversations, /invites, /agents, /settings/*, plus DefaultRedirect. No dedicated /login route in AppRoutes (auth flows use the welcome/onboarding paths).

AI configuration

Bundled prompts live under src/openhuman/agent/prompts/ at the repository root (also bundled via app/src-tauri/tauri.conf.json resources). Loaders under app/src/lib/ai/ use ?raw imports, optional remote fetch, and in Tauri ai_get_config / ai_refresh_config for packaged content.


Tauri shell (app/src-tauri/)

Thin desktop host: window management, daemon health bridging, core process lifecycle (core_process, CoreProcessHandle), and JSON-RPC relay to the openhuman sidecar (core_rpc_relay, core_rpc).

Registered IPC commands (see docs/src-tauri/02-commands.md) include greet, write_ai_config_file, ai_get_config, ai_refresh_config, core_rpc_relay, window commands, and OpenHuman service / daemon host helpers (openhuman_*).

Deep link plugin is registered where supported; behavior is platform-specific (see platform notes below).


Rust core (repo root src/)

  • openhuman/ — Domain logic (skills, memory, channels, config, …). RPC controllers live in rpc.rs files per domain; use RpcOutcome<T> pattern per AGENTS.md / internal rules.
  • src/openhuman/ module layout: New functionality must live in a dedicated subdirectory (its own folder/module, e.g. openhuman/my_domain/mod.rs plus related files, or a new subfolder under an existing domain). Do not add new standalone *.rs files directly at src/openhuman/ root; place new code in a module directory and declare it from mod.rs (or merge into an existing domain folder).
  • Controller schema contract: Shared controller metadata types live in src/core/mod.rs (ControllerSchema, FieldSchema, TypeSchema) and are consumed by adapters (RPC/CLI) in different ways.
  • Domain schema files: For each domain, define controller schema metadata in a dedicated module inside the domain folder (example: src/openhuman/cron/schemas.rs) and export from the domain mod.rs.
  • Light mod.rs rule: Keep domain mod.rs files light and export-focused. Put operational code in sibling files (example: ops.rs, store.rs, schedule.rs, types.rs), then re-export the public API from mod.rs.
  • core_server/ — Transport only: Axum/HTTP, JSON-RPC envelope, CLI parsing, dispatch (core_server::dispatch) — no heavy business logic here.
  • Layering: Implementation in openhuman::<domain>/, controllers in openhuman::<domain>/rpc.rs, routes in core_server/.

Skills runtime uses QuickJS (rquickjs) in src/openhuman/skills/ (e.g. qjs_skill_instance.rs, qjs_engine.rs), not V8/deno_core in this repository.

Controller migration checklist

  • src/openhuman/<domain>/mod.rs: keep export-focused, add mod schemas; and re-export:
    • all_controller_schemas as all_<domain>_controller_schemas
    • all_registered_controllers as all_<domain>_registered_controllers
  • src/openhuman/<domain>/schemas.rs must define:
    • schemas(function: &str) -> ControllerSchema
    • all_controller_schemas() -> Vec<ControllerSchema>
    • all_registered_controllers() -> Vec<RegisteredController>
    • domain handler fns fn handle_*(_: Map<String, Value>) -> ControllerFuture
  • Handlers should delegate to existing domain rpc.rs functions during migration.
  • Wire domain exports into src/core/all.rs for both declared schemas and registered handlers.
  • Keep adapters generic: do not add domain-specific logic to src/core/cli.rs or src/core/jsonrpc.rs.
  • Remove migrated method branches from src/rpc/dispatch.rs once registry coverage is in place.

App theme & design system

Design intent: Premium, calm visual language — ocean primary (#4A83DD), sage / amber / coral semantic colors, Inter + Cabinet Grotesk + JetBrains Mono, Tailwind with custom radii/spacing/shadows. Details: docs/DESIGN_GUIDELINES.md.

Desktop shell (Tauri) vs application code

In the parent OpenHuman desktop app, Tauri / Rust is a delivery vehicle: windowing, process lifecycle, IPC to the core sidecar, and other host concerns. Keep as much UI behavior and product logic as practical in TypeScript/React (app/). Avoid growing Rust in the shell for flows that belong in the web layer unless there is a hard platform or security reason.

Git workflow


Coding philosophy

  • Unix-style modules: Prefer individual modules with a single, sharp responsibility—each should do one thing really well. Compose behavior through small, well-named units and clear boundaries instead of monolithic code.
  • Tests before the next layer: Ship enough unit tests and coverage for the behavior you are adding or changing before building additional features on top of it. Treat untested code as incomplete; do not accumulate depth on a shaky base.

Feature design workflow (new capabilities)

Follow this order so behavior is specified, proven in Rust, proven over RPC, then surfaced in the UI with matching tests.

  1. Specify against the current codebase — Ground the design in existing domains, controller/registry patterns, and JSON-RPC naming (openhuman.<namespace>_<function>). Reuse or extend documented flows in docs/ARCHITECTURE.md and sibling guides; avoid parallel architectures.
  2. Implement in Rust — Add domain logic under src/openhuman/<domain>/, wire schemas + registered handlers into the shared registry, and land unit tests in the crate (cargo test -p openhuman, focused modules) until the feature is correct in isolation.
  3. JSON-RPC E2E — Add or extend integration-style tests that call the real HTTP JSON-RPC surface (e.g. tests/json_rpc_e2e.rs, mock backend / scripts/test-rust-with-mock.sh as appropriate) so methods, params, and outcomes match what the UI will call.
  4. UI in the Tauri app — Build React screens, state, and core_rpc_relay / coreRpcClient usage in app/; keep business rules in the core, not duplicated in the shell.
  5. App unit tests — Cover components, hooks, and clients with Vitest (yarn test / yarn test:unit in app/).
  6. App E2E — Add desktop E2E specs where the feature is user-visible (yarn test:e2e*, isolated workspace — see Testing Guide (Unit + E2E)) so the full stack (UI → Tauri → sidecar) behaves as intended.

Debug logging (throughout) — Add lots of development-oriented logging as you build, not as an afterthought. In Rust, use log / tracing at debug or trace on RPC entry and exit, error paths, state transitions, and any branch that is hard to infer from tests alone. In app/, follow existing patterns (e.g. the debug npm package with a namespace per area) plus dev-only detail where useful. Prefer grep-friendly prefixes ([feature], domain name, or JSON-RPC method) so terminal output from sidecar, Tauri, and WebView can be correlated during yarn dev / tauri dev. Never log secrets, raw JWTs, API keys, or full PII—redact or omit.

Planning rule: When scoping a feature, define the E2E scenarios (core RPC + app) up front. Those scenarios should cover the full intended scope—happy paths, failure modes, auth or policy gates, and regressions you care about. If a scenario is not testable end-to-end, the spec is incomplete or the cut is too large; split or add harness support first.


Key patterns (concise)

  • Debug logging: Ship heavy debug/trace (Rust) and namespaced debug / dev logs (app/) on new flows so sidecar + WebView output is easy to grep; see Feature design workflow. Never log secrets or raw tokens.
  • src/openhuman/: New features go in a folder/module, not new root-level src/openhuman/*.rs files (see Rust core section).
  • File size: Prefer ≤ ~500 lines per source file; split modules when growing.
  • Pre-merge checks (when touching code): Prettier, ESLint, tsc --noEmit in app/; cargo fmt + cargo check for changed Rust (Cargo.toml at root and/or app/src-tauri/Cargo.toml as appropriate).
  • No dynamic imports in app code (static import only); use try/catch around Tauri APIs where needed.
  • Type-only imports: import type where appropriate.
  • Dual socket / tool sync: If you change realtime protocol, keep frontend (socketService / MCP transport) and core socket behavior aligned (see docs/ARCHITECTURE.md dual-socket section).

Platform notes

  • macOS deep links: Often require a built .app bundle; not only tauri dev. See docs/telegram-login-desktop.md if applicable.
  • window.__TAURI__: Not assumed at module load; guard Tauri usage accordingly.
  • Core sidecar: Must be staged/built so core_rpc can reach the openhuman binary (see scripts/stage-core-sidecar.mjs).

Last aligned with monorepo layout (app/ + root src/), QuickJS skills in openhuman_core, skills catalog on GitHub (tinyhumansai/openhuman-skills), and Tauri shell IPC as of repo state.