diff --git a/src/openhuman/about_app/README.md b/src/openhuman/about_app/README.md new file mode 100644 index 000000000..1ac8dc144 --- /dev/null +++ b/src/openhuman/about_app/README.md @@ -0,0 +1,78 @@ +# about_app + +The single source of truth for the OpenHuman desktop app's **user-facing capability catalog**. It enumerates every capability the app exposes to end users — what each one does, where it lives in the UI (`how_to`), its maturity (`stable` / `beta` / `coming_soon` / `deprecated`), and a per-capability privacy disclosure (what data, if any, leaves the device and where it goes). The catalog is a compile-time static table; the module exposes read-only list / lookup / search over it via JSON-RPC. It is stateless — no persistence, no event subscribers, no agent tools. + +## Responsibilities + +- Define the canonical, hard-coded list of user-facing capabilities (`CAPABILITIES` in `catalog.rs`). +- Classify each capability by `CapabilityCategory` (conversation, intelligence, skills, local_ai, team, settings, auth, screen_intelligence, channels, automation, mobile) and `CapabilityStatus`. +- Attach optional `CapabilityPrivacy` disclosures (`leaves_device`, `data_kind`, `destinations`) so the in-app Privacy surface can render "what leaves my computer". +- Provide read APIs: list all (optionally filtered by category), look up one by stable id, keyword search across id/name/domain/category/description/how_to/status. +- Validate catalog integrity at first access (no empty ids, no duplicate ids) via a `OnceLock` guard. +- Expose those reads to CLI + JSON-RPC through controller schemas. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/about_app/mod.rs` | Export-only module root + docstring. Re-exports catalog reads, ops entry points, schema registry hooks, and types. | +| `src/openhuman/about_app/types.rs` | Serde domain types: `Capability`, `CapabilityCategory` (with `as_str` / `FromStr` incl. aliases), `CapabilityStatus`, `CapabilityPrivacy`, `PrivacyDataKind`. Inline serde/roundtrip tests. | +| `src/openhuman/about_app/catalog.rs` | The static `CAPABILITIES` table plus shared `CapabilityPrivacy` constants. Implements `all_capabilities`, `capabilities_by_category`, `lookup`, `search`, and the `ensure_validated` integrity check. | +| `src/openhuman/about_app/ops.rs` | RPC-facing logic returning `RpcOutcome`: `list_capabilities`, `lookup_capability`, `search_capabilities`. Thin wrappers over `catalog.rs` with summary logs. | +| `src/openhuman/about_app/schemas.rs` | Controller schemas + `handle_*` async handlers for the three RPC methods; param structs; the `all_about_app_controller_schemas` / `all_about_app_registered_controllers` registry pair. | +| `src/openhuman/about_app/catalog_tests.rs` | Sibling test module (`#[path]`-included by `catalog.rs`) covering catalog behavior. | + +## Public surface + +Re-exported from `mod.rs`: + +- **Catalog reads** (`catalog`): `all_capabilities()`, `capabilities_by_category(CapabilityCategory)`, `lookup(&str)`, `search(&str)`. +- **Ops** (`ops`): `list_capabilities(Option) -> RpcOutcome>`, `lookup_capability(&str) -> Result, String>`, `search_capabilities(&str) -> RpcOutcome>`. +- **Schema registry** (`schemas`): `about_app_schemas(&str)`, `all_about_app_controller_schemas()`, `all_about_app_registered_controllers()`. +- **Types**: `Capability`, `CapabilityCategory`, `CapabilityPrivacy`, `CapabilityStatus`, `PrivacyDataKind`. + +## RPC / controllers + +Namespace `about_app`, registered into the global controller registry via `src/core/all.rs`: + +| Method | Inputs | Output | Description | +| --- | --- | --- | --- | +| `about_app.list` | `category` (optional enum) | `capabilities: Capability[]` | List all capabilities, optionally filtered by category. | +| `about_app.lookup` | `id` (string, required) | `capability: Capability` | Look up one capability by stable id (e.g. `local_ai.download_model`); errors on unknown id. | +| `about_app.search` | `query` (string, required) | `capabilities: Capability[]` | Keyword search; empty query returns all. | + +Handlers deserialize params, log at `debug`, and emit `RpcOutcome` via `into_cli_compatible_json()`. The `category` input is schema-typed as an `Option` of all `CapabilityCategory` wire names. + +## Agent tools + +None. This module owns no `tools.rs`. + +## Events + +None. No `bus.rs`; the module neither publishes nor subscribes to `DomainEvent`s. + +## Persistence + +None. No `store.rs`. The catalog is a compile-time `&'static [Capability]` constant; the only runtime state is a `OnceLock<()>` (`VALIDATED`) that runs the duplicate/empty-id integrity check once. + +## Dependencies + +- `crate::rpc::RpcOutcome` — return-type contract for ops/handlers. +- `crate::core::all::{ControllerFuture, RegisteredController}` — controller registration types (schemas.rs). +- `crate::core::{ControllerSchema, FieldSchema, TypeSchema}` — controller schema definitions (schemas.rs). + +No dependencies on other `openhuman` domains — capability metadata for other domains is hand-authored text in `catalog.rs`, not imports. + +## Used by + +- `src/core/all.rs` — registers the controllers/schemas into the global RPC/CLI registry and supplies the `about_app` namespace description. +- `src/openhuman/memory_sync/composio/periodic.rs` — references this catalog only in a doc comment, as the place to add the user-visible status for that flow (no code dependency). + +## Notes / gotchas + +- **`privacy: None` means "unknown", not "safe"** (per `types.rs` doc). UI must not treat an unannotated capability as local-only. +- **Adding/renaming/removing a user-facing feature requires editing `CAPABILITIES`** — this is the capability catalog that CLAUDE.md's "Capability catalog" rule points at. Keep ids stable; duplicate or empty ids panic at first catalog access via `ensure_validated`. +- Privacy constants encode real third-party destinations (Hugging Face, GitHub Releases, Composio `backend.composio.dev`, Polymarket, SearXNG, configured embedding providers, ElevenLabs, etc.) — the inline comments document why several were corrected away from the generic `DERIVED_TO_BACKEND` / `LOCAL_CREDENTIALS` defaults; mirror that diligence when adding network-touching capabilities. +- `Capability` fields are all `&'static str` / copy types, so `Capability` is `Copy` and the read APIs cheaply return owned `Vec`s by copying. +- A capability's `domain` is a free-text label and does not always equal its `category` wire name (e.g. `embeddings`, `wallet`, `runtime_python`, `devices`, `desktop_companion`, `security`, `tools`, `memory`). +- `CapabilityCategory::FromStr` is lenient (case-insensitive, accepts `local-ai`/`local ai`/`localai` and `screen-intelligence`/`screen intelligence` aliases); `as_str` emits the canonical snake_case wire name used by serde. diff --git a/src/openhuman/agent_experience/README.md b/src/openhuman/agent_experience/README.md new file mode 100644 index 000000000..131a7efee --- /dev/null +++ b/src/openhuman/agent_experience/README.md @@ -0,0 +1,93 @@ +# agent_experience + +Hermes-style **procedural experience memory** for agents. Captures what tool sequences worked (or failed) during a chat turn, redacts secrets, persists them as structured records in the shared memory store, and ranks/injects relevant past experiences back into future turns as a compact "Relevant Operating Experience" prompt block. The goal is cross-turn procedural learning: the agent remembers *how* it solved similar tasks before, not just facts. + +## Responsibilities + +- Define the `AgentExperience` record (task summary, tool sequence, outcome, lesson, reuse/avoid hints, confidence, tags). +- Persist experiences (upsert by stable id) into the memory store under the `agent_experience` namespace, redacting secret-like text first. +- Retrieve and rank experiences for a given task query via lexical/tool/tag overlap scoring. +- Mark experiences as dismissed so retrieval skips them. +- Auto-derive experience candidates from a completed turn's tool calls (multi-tool success, repeated failures, partial recovery) via a `PostTurnHook`. +- Render ranked hits into a byte-capped markdown block and prepend it to the enriched user message before a turn. +- Expose capture/retrieve/list/dismiss over JSON-RPC. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/agent_experience/mod.rs` | Export-focused module root; re-exports the public surface. | +| `src/openhuman/agent_experience/types.rs` | Serde types (`AgentExperience`, `ExperienceHit`, `ExperienceSource`, `ExperienceOutcome`), `redact_text` (Bearer / `sk-` / `token=secret` masking), and `stable_experience_id` (SHA-256 over summary + tool sequence + outcome). | +| `src/openhuman/agent_experience/store.rs` | `AgentExperienceStore` over `Arc`: `put`/`list`/`dismiss`/`retrieve`, `ExperienceQuery`, the `AGENT_EXPERIENCE_NAMESPACE` const, and the lexical/tool/tag overlap scoring (`score_experience`). | +| `src/openhuman/agent_experience/capture.rs` | `AgentExperienceCaptureHook` — a `PostTurnHook` that mines `TurnContext.tool_calls` into experience candidates (`successful_multi_tool_experience`, `repeated_failure_experiences`, `partial_success_experience`) and persists them. | +| `src/openhuman/agent_experience/prompt.rs` | `render_experience_hits` (byte-capped markdown under `AGENT_EXPERIENCE_HEADING = "## Relevant Operating Experience"`) and `prepend_experience_block`. | +| `src/openhuman/agent_experience/ops.rs` | RPC entry points returning `RpcOutcome` (`capture`/`retrieve`/`list`/`dismiss`); `open_store()` resolves the memory client (lazy-init from config if not ready). | +| `src/openhuman/agent_experience/schemas.rs` | Controller schemas + `handle_*` dispatchers; `all_controller_schemas` / `all_registered_controllers`. | + +## Public surface + +From `mod.rs` re-exports: + +- `AgentExperienceCaptureHook` (capture) +- `prepend_experience_block`, `render_experience_hits`, `AGENT_EXPERIENCE_HEADING` (prompt) +- `all_agent_experience_controller_schemas`, `all_agent_experience_registered_controllers` (schemas) +- `AgentExperienceStore`, `ExperienceQuery`, `AGENT_EXPERIENCE_NAMESPACE` (store) +- `redact_text`, `stable_experience_id`, `AgentExperience`, `ExperienceHit`, `ExperienceOutcome`, `ExperienceSource` (types) + +## RPC / controllers + +Namespace `agent_experience` (registered into `src/core/all.rs`): + +| Method | Inputs | Output | +| --- | --- | --- | +| `agent_experience.capture` | `experience: AgentExperience` | Stored `AgentExperience` (upserted, redacted). | +| `agent_experience.retrieve` | `query` (req), `tools[]`, `tags[]`, `agent_id?`, `entrypoint?`, `max_hits?` (default 5) | `hits: ExperienceHit[]` ranked. | +| `agent_experience.list` | none | `experiences: AgentExperience[]` ordered by most-recent update. | +| `agent_experience.dismiss` | `id` | `{ id, dismissed }`. | + +All handlers delegate to `ops.rs` and wrap results in `RpcOutcome::single_log`. + +## Agent hooks (not a tool) + +This module owns no `tools.rs` agent tool. Instead it registers `AgentExperienceCaptureHook` as a **`PostTurnHook`** (`name() == "agent_experience_capture"`). On `on_turn_complete` it extracts candidates from the turn's tool calls and persists them when enabled. Candidate heuristics: + +- **Multi-tool success**: ≥2 successful tool calls → `ExperienceOutcome::Success`, confidence 0.72. +- **Repeated failure**: a tool that failed ≥2 times in one turn → `Failure`, confidence 0.68, with an error class parsed from the output summary (`...(error_class)`). +- **Partial success**: a failure followed by a later success → `Partial`, confidence 0.62. + +## Events + +None — no `bus.rs`; this module does not publish or subscribe to `DomainEvent`s. + +## Persistence + +Records are stored through the shared `Memory` abstraction (no dedicated DB): + +- Namespace: `agent_experience` (`AGENT_EXPERIENCE_NAMESPACE`). +- Key: `experience/`; id is `stable_experience_id(...)` (`exp_<24 hex>`) when not supplied. +- Value: full `AgentExperience` JSON, `MemoryCategory::Custom("agent_experience")`. +- `put` preserves the original `created_at_ms` on update, stamps `updated_at_ms`, and redacts `task_summary` / `lesson` / `reuse_hint` / `avoid_hint` before write. Dismiss is a soft flag (`dismissed = true`), retained in `list`, filtered out of `retrieve`. + +## Dependencies + +- `crate::openhuman::memory` — `Memory` trait, `MemoryCategory`, and `memory::global` client (storage backend; lazy-init via `Config`). +- `crate::openhuman::config` — `Config::load_or_init` to resolve `workspace_dir` when the memory client isn't ready. +- `crate::openhuman::agent::hooks` — `PostTurnHook`, `TurnContext`, `ToolCallRecord` (capture hook contract / turn inputs). +- `crate::core::all` — `ControllerFuture`, `RegisteredController` for RPC registration. +- `crate::core` — `ControllerSchema`, `FieldSchema`, `TypeSchema` (schema types); `crate::rpc::RpcOutcome`. +- `crate::openhuman::memory_tools::test_helpers::MockMemory` — tests only. + +## Used by + +- `src/core/all.rs` — registers controllers/schemas and the namespace description. +- `src/openhuman/agent/harness/session/builder.rs` — constructs `AgentExperienceCaptureHook::new(...)` and registers it for the learning/capture flow. +- `src/openhuman/agent/harness/session/turn.rs` — imports from this module and `inject_agent_experience_context` to retrieve + prepend the experience block into the enriched user message before a turn runs. +- `src/openhuman/mod.rs` — declares the module. +- `src/openhuman/memory_sync/workspace/mod.rs` — references it (doc comment) as a peer memory writer. + +## Notes / gotchas + +- **Redaction is applied at write time**, both in `store::put` and again in `capture::build_experience`; secret-like substrings (`Bearer …`, `sk-…`, `token=/password:` pairs) are masked before persistence. +- Retrieval scoring is **lexical, not embedding-based**: term sets keep only tokens length > 2, normalized lowercase; score combines tool overlap (weighted highest), tag overlap, query-term overlap over summary+lesson+hints, plus small agent/entrypoint match boosts and a confidence prior. `max_hits == 0` short-circuits to empty. +- `render_experience_hits` is hard byte-capped (`max_bytes`) with UTF-8-boundary-safe truncation, so the injected prompt block can't blow the context budget. +- The capture hook is gated by an `enabled` flag passed at construction; when disabled `on_turn_complete` is a no-op, and capture failures only `log::warn!` (never fail the turn). diff --git a/src/openhuman/agent_tool_policy/README.md b/src/openhuman/agent_tool_policy/README.md new file mode 100644 index 000000000..b7d4a1ebc --- /dev/null +++ b/src/openhuman/agent_tool_policy/README.md @@ -0,0 +1,53 @@ +# agent_tool_policy + +Profiles and enforces the tool boundary for a single agent session, keeping the prompt-visible tool set and runtime execution decisions aligned with the channel's configured permission ceiling. Given the active agent, channel, configured per-channel permission map, the available tool registry, and an optional set of explicitly-visible tool names, it produces a deterministic, immutable `ToolPolicySession` snapshot: per-tool decisions (allow / deny / hide), the allowed/blocked/hidden name sets, and a coarse task risk level. It also renders a compact system-prompt section describing the active boundary. This domain is pure logic — no persistence, no RPC, no events. + +## Responsibilities + +- Resolve a channel's `PermissionLevel` ceiling from a `channel -> permission` string map (`permission_for_channel`), with fallbacks. +- Classify every tool in the registry against that ceiling and the optional visibility set, producing a `ToolPolicyAction` (Allow / RequireApproval / Deny / HideFromPrompt) per tool. +- Build an immutable `ToolPolicySession` snapshot (profile, capabilities, allowed/blocked/hidden tool-name sets, decision map) attached to an agent session. +- Derive a coarse `TaskRiskLevel` (Low/Medium/High/Critical) from the highest allowed permission. +- Render a bounded `## Tool Policy Boundary` system-prompt section listing the active agent/channel/entrypoint, allowed permission, risk, allowed tools, and a restricted-count summary. +- Provide a fail-closed default decision (`Deny`) for unknown or unlisted tool names at runtime. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/agent_tool_policy/mod.rs` | Export-only: module docstring + `mod` decls + `pub use` re-exports of the engine, prompt renderer, and types. | +| `src/openhuman/agent_tool_policy/types.rs` | Serde-free domain types: `TaskRiskLevel`, `TaskProfile`, `ToolPolicyAction`, `ToolPolicyDecision`, `ToolCapability`, `ToolPolicySession` (with query helpers). Holds the `NO_TOOLS_ALLOWED_SENTINEL`. | +| `src/openhuman/agent_tool_policy/engine.rs` | `ToolPolicyEngine::build_session` — the classification logic; private `permission_for_channel` / `parse_permission_level` helpers. Includes inline `#[cfg(test)]` suite. | +| `src/openhuman/agent_tool_policy/prompt.rs` | `render_tool_policy_boundary` + `TOOL_POLICY_BOUNDARY_HEADING`; UTF-8-safe `truncate_utf8`. Includes inline `#[cfg(test)]` suite. | + +## Public surface + +Re-exported from `mod.rs`: + +- `ToolPolicyEngine` — `build_session(agent_id, channel, entrypoint, channel_permissions: &HashMap, tools: &[Box], visible_tool_names: &HashSet) -> ToolPolicySession`. +- `render_tool_policy_boundary(session: &ToolPolicySession, max_bytes: usize) -> Option` — `None` when the session has no restrictions; otherwise a truncated prompt section. +- Types: `TaskProfile`, `TaskRiskLevel`, `ToolCapability`, `ToolPolicyAction`, `ToolPolicyDecision`, `ToolPolicySession`. + +`ToolPolicySession` helpers: `is_allowed(name)`, `has_restrictions()`, `restricted_tool_count()`, `visible_tool_names_for_prompt()`, `decision_for(name)` (defaults to `Deny`). `ToolPolicyDecision::is_denied()` is true for anything other than `Allow`. + +## Dependencies + +- `crate::openhuman::tools` — `PermissionLevel` (the ceiling/ordering, parsed and compared) and the `Tool` trait (`name()`, `permission_level()`). The only openhuman/core dependency. +- stdlib `std::collections` (`BTreeSet`/`HashMap`/`HashSet`) and `log` for grep-friendly `[tool-policy]` diagnostics under target `openhuman::agent_tool_policy`. + +## Used by + +- `src/openhuman/agent/harness/session/builder.rs` — builds the `ToolPolicySession` (`ToolPolicyEngine`, `ToolPolicySession`). +- `src/openhuman/agent/harness/session/runtime.rs` — uses `ToolPolicyEngine`. +- `src/openhuman/agent/harness/session/turn.rs` — calls `render_tool_policy_boundary` to inject the boundary into the prompt. +- `src/openhuman/agent/harness/session/types.rs` — carries `ToolPolicySession` on the session. + +## Notes / gotchas + +- **Legacy escape hatch**: an empty `channel_permissions` map yields `PermissionLevel::Dangerous` (fully unrestricted), preserving pre-policy behavior. Once *any* channel policy exists, channels missing from the map (or with an unparseable value) fall back to `PermissionLevel::ReadOnly`, not unrestricted. +- **Permission parsing** (`parse_permission_level`) is lenient: trims, lowercases, strips `-`/`_`, and accepts aliases (`read`/`readonly`, `exec`/`execute`, `danger`/`dangerous`). Unrecognized tokens fall back to read-only. +- **Two independent restriction axes**: a tool can be `Deny`ed (exceeds the permission ceiling → `blocked_tool_names`) or `HideFromPrompt` (not in the non-empty `visible_tool_names` set → `hidden_tool_names`). Hidden takes precedence over deny in the classification order. +- `ToolPolicyAction::RequireApproval` is defined and handled in the match (routed to `blocked_tool_names`) but `build_session` never currently produces it. +- `visible_tool_names_for_prompt()` inserts `NO_TOOLS_ALLOWED_SENTINEL` when restrictions exist but nothing is allowed, so prompt rendering can signal an empty-but-restricted surface rather than an unrestricted one. +- `render_tool_policy_boundary` returns `None` for unrestricted sessions, and `truncate_utf8` guarantees the output stays within `max_bytes` on a char boundary (appending `\n[...truncated]` only when there is room). +- Snapshots are immutable and deterministic per session; there is no mutation API. diff --git a/src/openhuman/approval/README.md b/src/openhuman/approval/README.md new file mode 100644 index 000000000..23268d27c --- /dev/null +++ b/src/openhuman/approval/README.md @@ -0,0 +1,94 @@ +# approval + +Interactive approval workflow for supervised mode (issue #1339). `ApprovalGate` is async middleware sitting between the agent and any tool whose `Tool::external_effect` returns `true` (Slack post, email send, calendar create, shell, …). It intercepts the call, checks the user's "Always allow" allowlist, persists a pending row in SQLite, publishes an `ApprovalRequested` event so the UI can surface a prompt, parks the tool-call future on a `oneshot`, and resumes when the UI (or a typed chat yes/no) dispatches a decision via the `approval_decide` RPC. Denials and timeouts (10-min TTL) fail closed. The module also redacts PII/chat content out of anything it persists or broadcasts, and records a terminal execution-outcome audit trail after the allowed tool finishes (issue #2135). + +## Responsibilities + +- Intercept external-effect tool calls and gate them behind explicit user consent. +- Short-circuit to `Allow` when the tool is on the user's `autonomy.auto_approve` allowlist (read live via `security::live_policy`). +- Allow through (never park) when there is no live chat context — background/triage/cron turns carry no `ApprovalChatContext` and are pre-authorized. +- Persist pending requests in SQLite so they survive a core restart; lazily expire stale rows; keep a durable decided/executed audit trail. +- Resolve a parked call on a user decision (`approve_once` / `approve_always_for_tool` / `deny`), TTL timeout, or channel drop — failing closed in every non-approve path. +- Redact arguments (`redact_args`) and build safe action summaries (`summarize_action`) before anything leaves the gate. +- Route a thread's yes/no chat reply back to a parked approval (`pending_for_thread` + `parse_approval_reply`). +- On `approve_always_for_tool`, persist the tool onto `autonomy.auto_approve` (config save + live-policy reload) so it skips prompting next time. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/approval/mod.rs` | Export-focused: module docstring, `pub mod` decls, `pub use` re-exports including the controller-schema pair. | +| `src/openhuman/approval/gate.rs` | `ApprovalGate` — the singleton coordinator. `init_global`/`try_global`, `intercept`/`intercept_audited`, `decide`, `record_execution`, `list_pending`, `list_recent_decisions`, the thread→request routing map, `ApprovalChatContext` task-local, and `parse_approval_reply`. | +| `src/openhuman/approval/store.rs` | SQLite persistence (`pending_approvals` table). `insert_pending`, `decide`, `get_decision`, `record_execution`, `list_pending`, `list_recent_decisions`, `purge_session`, `expire_stale`, plus idempotent column migration for the v1 schema. | +| `src/openhuman/approval/types.rs` | Serde domain types: `PendingApproval`, `ApprovalAuditEntry`, `ApprovalDecision`, `GateOutcome`, `ExecutionOutcome`. | +| `src/openhuman/approval/redact.rs` | `redact_args` (PII/chat-content key scrubbing + home-path stripping) and `summarize_action` (safe-field summary). | +| `src/openhuman/approval/rpc.rs` | Domain RPC entry points returning `RpcOutcome`: `approval_list_pending`, `approval_list_recent_decisions`, `approval_decide`. | +| `src/openhuman/approval/schemas.rs` | Controller schemas + `handle_*` fns wiring the RPC into the registry. | + +## Public surface + +Re-exported from `mod.rs`: + +- Gate: `ApprovalGate`, `ApprovalChatContext`, `APPROVAL_CHAT_CONTEXT` (task-local), `parse_approval_reply`. +- Redaction: `redact_args`, `summarize_action`. +- Types: `PendingApproval`, `ApprovalAuditEntry`, `ApprovalDecision`, `ExecutionOutcome`, `GateOutcome`. +- Controller registry: `all_approval_controller_schemas`, `all_approval_registered_controllers`. + +`ApprovalGate::try_global()` returns `None` when no gate is installed; tools/harness branches treat `None` as "no gating". + +## RPC / controllers + +Namespace `approval` (registered via `all_approval_registered_controllers`, consumed by `src/core/all.rs`): + +| Method | Inputs | Output | +| --- | --- | --- | +| `approval.list_pending` | — | `pending: PendingApproval[]` | +| `approval.list_recent_decisions` | `limit?: u64` (1-500, default 50) | `decisions: ApprovalAuditEntry[]` | +| `approval.decide` | `request_id: string`, `decision: string` (`approve_once` / `approve_always_for_tool` / `deny`) | `decided: PendingApproval` | + +`list_pending` / `list_recent_decisions` return empty (not an error) when the gate is not installed; `decide` errors when the gate is absent or the `request_id` is unknown/already decided. + +## Agent tools + +None. This module gates other domains' tools; it owns no tools of its own (no `tools.rs`). + +## Events + +Published via `publish_global` (domain `approval`, defined in `src/core/event_bus/events.rs`): + +- `DomainEvent::ApprovalRequested { request_id, tool_name, action_summary, args_redacted, session_id, thread_id, client_id }` — emitted when a call is parked. Bridged to the `approval_request` web-channel socket event by `ApprovalSurfaceSubscriber` (defined in `src/openhuman/channels/providers/web.rs`). +- `DomainEvent::ApprovalDecided { request_id, tool_name, decision }` — emitted when a decision is applied. + +No `bus.rs` in this module — it only publishes; the subscriber lives in the `channels` web provider. + +## Persistence + +SQLite DB at `{workspace_dir}/approval/approval.db`, table `pending_approvals` (opened per-call via `with_connection`, schema + column migration applied idempotently). Columns: `request_id` (PK), `tool_name`, `action_summary`, `args_redacted` (JSON), `session_id`, `created_at`, `expires_at`, `decided_at`, `decision`, plus the after-action audit columns `executed_at`, `execution_outcome`, `execution_error` (added by `migrate_columns` for v1 DBs). Pending rows survive restart; expired rows are lazily transitioned to a terminal `deny` decision; `record_execution` is write-once (`executed_at IS NULL` guard) and sanitizes/caps error text to 512 chars to keep secrets/PII out of the durable log. + +## Dependencies + +- `crate::core::event_bus` — `publish_global` + `DomainEvent` to surface approval prompts/decisions. +- `crate::core::all` — `ControllerFuture` / `RegisteredController` for the controller registry. +- `crate::core` (`ControllerSchema`, `FieldSchema`, `TypeSchema`) — schema definitions. +- `crate::rpc::RpcOutcome` — RPC return contract. +- `crate::openhuman::config::Config` — workspace dir (DB path) + the boot-time `autonomy.auto_approve` snapshot; `config::ops::add_auto_approve_tool` to persist "Always allow". +- `crate::openhuman::security` — `live_policy::current()` for the live "Always allow" list and `POLICY_DENIED_MARKER` for deny reasons. +- `crate::openhuman::memory_store::safety::sanitize_text` — scrub secrets out of stored execution-error strings. + +## Used by + +- `src/core/jsonrpc.rs` — installs the global gate (`ApprovalGate::init_global`) at startup; wires the approval RPCs. +- `src/core/all.rs` — registers the controller schemas. +- `src/openhuman/agent/harness/tool_loop.rs` (+ subagent_runner) — routes external-effect tool calls through the gate before `execute()`. +- `src/openhuman/channels/providers/web.rs` — sets `APPROVAL_CHAT_CONTEXT`, hosts `ApprovalSurfaceSubscriber`, and routes typed yes/no replies to `approval_decide`. +- `src/openhuman/channels/proactive.rs`, `src/openhuman/agent/triage/escalation.rs`, `src/openhuman/tools/impl/system/install_tool.rs`, `src/openhuman/wallet/execution.rs` — interact with the gate / approval types. + +## Notes / gotchas + +- **Interactive only.** With no `ApprovalChatContext` task-local in scope, `intercept` returns `Allow` immediately (no row, no event) so autonomous turns don't stall on a prompt nobody can answer. +- **Fail-closed everywhere.** Persist failure, channel drop, and TTL timeout all return `Deny` (with a `POLICY_DENIED_MARKER`-prefixed reason). The TTL path re-reads the persisted decision to honor an approve that committed in the timeout race (PR #2367). +- **Waiter registered before persist** so a fast `approval_decide` can't mark a request approved while no waiter exists (PR #2149). +- **Orphan rows are intentionally preserved** across launches (issue #1339); deciding one is a DB-only audit update — no side effect can fire across processes, so the security invariant holds. +- **`approve_always_for_tool` persistence is the RPC handler's job**, not the gate's — `gate.decide` only resolves the parked future and emits the audit event; `rpc::approval_decide` appends to `autonomy.auto_approve` + reloads the live policy (best-effort; failure degrades to prompting again). +- `OPENHUMAN_APPROVAL_GATE=0`/`false` skips installing the gate (handled in `src/core/jsonrpc.rs`), in which case `Prompt`-class calls run unprompted. +- A prior list-based `ApprovalManager` was removed; the gate is now the sole control reading the `autonomy.auto_approve` allowlist. diff --git a/src/openhuman/artifacts/README.md b/src/openhuman/artifacts/README.md new file mode 100644 index 000000000..480ea5713 --- /dev/null +++ b/src/openhuman/artifacts/README.md @@ -0,0 +1,78 @@ +# artifacts + +Metadata domain for agent-generated artifacts (presentations, documents, images, and other files the agent produces). It owns the on-disk layout under `/artifacts/`, persists per-artifact `meta.json` records, and exposes read/delete RPC controllers in the `ai` namespace. It does **not** generate artifact content itself — it is a thin, sandboxed metadata store + listing/retrieval/deletion surface over a workspace subdirectory. + +## Responsibilities + +- Define artifact metadata types (`ArtifactMeta`, `ArtifactKind`, `ArtifactStatus`) with case-insensitive string parsing and lowercase serde. +- Persist artifact metadata to `/artifacts//meta.json` (`save_artifact_meta`). +- List artifacts with pagination, sorted by `created_at` descending, skipping corrupt/unreadable `meta.json` entries (`list_artifacts`). +- Retrieve a single artifact by ID, returning metadata plus a computed `absolute_path` (`get_artifact` / `ai_get_artifact`). +- Delete an artifact directory and all its contents (`delete_artifact`). +- Enforce path-traversal sandboxing on artifact IDs and resolved paths so callers cannot escape the artifacts root. +- Expose `ai.list_artifacts`, `ai.get_artifact`, `ai.delete_artifact` RPC controllers. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/artifacts/mod.rs` | Export-only: `mod` decls + re-exports of `ArtifactKind`/`ArtifactMeta`/`ArtifactStatus` and the controller-schema/registry pair. | +| `src/openhuman/artifacts/types.rs` | Serde domain types: `ArtifactKind` (presentation/document/image/other), `ArtifactStatus` (pending/ready/failed), `ArtifactMeta`. Each enum has `as_str`/`parse` (case-insensitive, fall back to `Other`/`Pending`). | +| `src/openhuman/artifacts/ops.rs` | Business logic returning `RpcOutcome`: `ai_list_artifacts`, `ai_get_artifact`, `ai_delete_artifact`. Validates non-empty IDs, computes/guards `absolute_path`. `DEFAULT_LIMIT=50`, `MAX_LIMIT=200`. | +| `src/openhuman/artifacts/store.rs` | Persistence over `tokio::fs`: `artifacts_root`, `save_artifact_meta`, `list_artifacts`, `get_artifact`, `delete_artifact`, plus `validate_artifact_id` / `assert_within_root` sandboxing helpers. | +| `src/openhuman/artifacts/schemas.rs` | Controller schemas (`all_controller_schemas`), registry (`all_registered_controllers`), and `handle_*` fns delegating to `ops.rs`; param-parsing helpers (`read_required`, `read_optional_u64`, `type_name`). | +| `src/openhuman/artifacts/ops_tests.rs` | Sibling test suite for `ops.rs` (via `#[path]`). | +| `src/openhuman/artifacts/store_tests.rs` | Sibling test suite for `store.rs` (via `#[path]`). | + +## Public surface + +- `ArtifactKind`, `ArtifactMeta`, `ArtifactStatus` (re-exported from `types`). +- `all_artifacts_controller_schemas` / `all_artifacts_registered_controllers` (re-exported from `schemas`, wired into the core registry). +- `ops::{ai_list_artifacts, ai_get_artifact, ai_delete_artifact}` (called by the schema handlers; referenced by fully-qualified path). +- `store::*` functions are `pub(crate)` — usable inside the crate (e.g. a future producer calling `save_artifact_meta`). + +## RPC / controllers + +All in the `ai` namespace: + +| Method | Inputs | Output | +| --- | --- | --- | +| `ai.list_artifacts` | `offset?: u64` (default 0), `limit?: u64` (default 50, cap 200) | `{ artifacts: ArtifactMeta[], total, offset, limit }` | +| `ai.get_artifact` | `artifact_id: string` (required) | flat `ArtifactMeta` fields + `absolute_path` | +| `ai.delete_artifact` | `artifact_id: string` (required) | `{ artifact_id, deleted: bool }` | + +`get_artifact` output is intentionally flat (no opaque `artifact` wrapper). Handlers load config via `config::rpc::load_config_with_timeout()` and trim the `artifact_id`. + +## Agent tools + +None. This module owns no `tools.rs`. + +## Events + +None. No `bus.rs`; the module publishes/subscribes to no `DomainEvent`s. + +## Persistence + +- Root: `/artifacts/` (auto-created via `create_dir_all` in `artifacts_root`). +- Per artifact: `/artifacts//meta.json` (pretty-printed `ArtifactMeta`). +- `workspace_dir` is read from `Config` (`config.workspace_dir`). +- No database; entirely filesystem-backed via `tokio::fs`. Listing scans subdirectories and skips entries whose `meta.json` is missing or corrupt (logged at `warn`). + +## Dependencies + +- `crate::openhuman::config::Config` / `config::rpc` — source of `workspace_dir` and config loading in handlers. +- `crate::core::all::{ControllerFuture, RegisteredController}` and `crate::core::{ControllerSchema, FieldSchema, TypeSchema}` — controller registry + schema types. +- `crate::rpc::RpcOutcome` — ops return type / handler JSON conversion. +- External crates: `tokio::fs`, `serde`/`serde_json`, `chrono` (timestamps). + +## Used by + +- `src/core/all.rs` — registers `all_artifacts_registered_controllers()` (line ~141) and `all_artifacts_controller_schemas()` (line ~307) into the global controller/schema registries. No other in-crate consumer of `save_artifact_meta` was found, so artifact creation is not yet wired from a producer domain. + +## Notes / gotchas + +- **Path-traversal hardening is layered**: `validate_artifact_id` rejects empty/`.`/`..`, `/` or `\`, absolute paths, and Windows drive-letter paths; `assert_within_root` re-checks the resolved path stays under the artifacts root before any write/read/delete; `ai_get_artifact` independently re-validates that `meta.path` does not escape the root when computing `absolute_path` (defends against a corrupt/adversarial stored `meta.path`). +- `list_artifacts` is **lossy by design** — bad `meta.json` files are skipped (warned), not surfaced as errors, so a single corrupt artifact never breaks the whole listing. +- Enum `parse` never errors: unknown `kind` → `Other`, unknown `status` → `Pending`. +- `store.rs` carries a dead-code `_assert_status_used` helper to keep `ArtifactStatus` referenced outside tests. +- Verbose `[artifacts]` `log::debug!`/`warn!` prefixes throughout, per repo logging conventions. diff --git a/src/openhuman/audio_toolkit/README.md b/src/openhuman/audio_toolkit/README.md new file mode 100644 index 000000000..64e383db7 --- /dev/null +++ b/src/openhuman/audio_toolkit/README.md @@ -0,0 +1,87 @@ +# audio_toolkit + +Text-to-speech "podcast" toolkit. Synthesizes text into a workspace audio file via the `voice` TTS providers, optionally emails that file as an attachment (over SMTP or as a workspace capture in test/e2e mode), and exposes both as agent tools and JSON-RPC controllers. The combined `generate_and_email` flow chains both steps so the agent can turn arbitrary text into a listen-later audio email in one call. + +## Responsibilities + +- Synthesize text → audio (`mp3`/`wav`) using a configured or requested TTS provider (`cloud` default, or `piper`). +- Resolve and harden output paths: workspace-relative only, no absolute paths, no `..` traversal; default path is `artifacts/audio/-.`. +- Enforce provider/format compatibility (`piper` → `wav` only; `cloud` → `mp3` only) and verify the provider's returned MIME matches the requested format. +- Build a multipart email (plain body + audio attachment) and deliver it via the `EmailChannel` over SMTP. +- In `e2e-test-support` builds (or when `OPENHUMAN_EMAIL_CAPTURE_DIR` is set), capture the `.eml` to a workspace file instead of sending. +- Surface all three operations as agent tools and JSON-RPC controllers. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/audio_toolkit/mod.rs` | Export-only module surface: re-exports ops fns, schema registries, and types. | +| `src/openhuman/audio_toolkit/types.rs` | Serde domain types: `AudioFormat`, request/result structs. | +| `src/openhuman/audio_toolkit/ops.rs` | Business logic: `generate_podcast`, `email_podcast`, `generate_and_email_podcast`, `resolve_email_capture_dir`, plus private helpers (path/format/voice resolution, base64 decode, MIME enforcement, slugify, email build/capture). Contains the unit-test suite. | +| `src/openhuman/audio_toolkit/schemas.rs` | RPC controller schemas + `handle_*` handlers delegating to `ops.rs`; loads `Config` via `config::rpc`. | +| `src/openhuman/audio_toolkit/tools.rs` | Re-exports the three agent tool structs from `tools/podcast.rs`. | +| `src/openhuman/audio_toolkit/tools/podcast.rs` | `Tool` impls: `AudioGeneratePodcastTool`, `AudioEmailPodcastTool`, `AudioGenerateAndEmailPodcastTool`; security gating + arg parsing. | + +## Public surface + +From `mod.rs`: + +- **ops**: `generate_podcast`, `email_podcast`, `generate_and_email_podcast`, `resolve_email_capture_dir` — all take `&Config` and return `Result, String>` (except `resolve_email_capture_dir → Option`). +- **schemas**: `all_audio_toolkit_controller_schemas`, `all_audio_toolkit_registered_controllers`. +- **types**: `AudioFormat` (`Mp3`/`Wav`, with `extension()` / `mime()`), `AudioGenerateRequest`, `EmailPodcastRequest`, `AudioGeneratedArtifact`, `AudioEmailDeliveryResult`, `AudioToolkitGenerateAndEmailResult`. +- **tools** (`pub mod tools`): `AudioGeneratePodcastTool`, `AudioEmailPodcastTool`, `AudioGenerateAndEmailPodcastTool`. + +## RPC / controllers + +Namespace `audio_toolkit` (invoked as `openhuman.audio_toolkit_`): + +| Method | Inputs | Output | +| --- | --- | --- | +| `audio_toolkit_generate_podcast` | `text` (req); optional `title`, `output_path`, `provider`, `voice`, `format` | `audio` (artifact JSON) | +| `audio_toolkit_email_podcast` | `to`, `subject`, `body`, `audio_path` (req); optional `attachment_name` | `email` (delivery JSON) | +| `audio_toolkit_generate_and_email_podcast` | `text`, `to`, `subject`, `body` (req); optional `title`, `output_path`, `provider`, `voice`, `format`, `attachment_name` | `result` (combined JSON) | + +Registered into the global controller registry via `src/core/all.rs` (both `all_audio_toolkit_registered_controllers` and `all_audio_toolkit_controller_schemas`). + +## Agent tools + +From `tools/podcast.rs` — all `PermissionLevel::Execute`, each calls `SecurityPolicy::enforce_tool_operation(ToolOperation::Act, ...)` before running: + +| Tool name | Backing op | +| --- | --- | +| `audio_generate_podcast` | `generate_podcast` | +| `audio_email_podcast` | `email_podcast` | +| `audio_generate_and_email_podcast` | `generate_and_email_podcast` | + +Tools are constructed with `Arc` + `Arc` and registered in `src/openhuman/tools/ops.rs`; re-exported via `src/openhuman/tools/mod.rs`. + +## Persistence + +No durable domain store. Side effects are filesystem writes within the workspace: + +- Audio files → `artifacts/audio/` (or caller-supplied `output_path`). +- Captured emails (test/e2e) → `artifacts/email-capture/` (or `OPENHUMAN_EMAIL_CAPTURE_DIR`) as `.eml` files named `podcast-email--.eml`. + +## Dependencies + +- `crate::openhuman::voice` — `create_tts_provider`, `DEFAULT_PIPER_VOICE`; actual speech synthesis. +- `crate::openhuman::channels::email_channel::EmailChannel` — SMTP delivery of the built message. +- `crate::openhuman::config::Config` / `config::rpc` — workspace dir, `local_ai.tts_provider`, `channels_config.email`; `load_config_with_timeout` in RPC handlers. +- `crate::openhuman::security` — `SecurityPolicy` / `ToolOperation` to gate the agent tools. +- `crate::openhuman::tools::traits` — `Tool`, `ToolResult`, `PermissionLevel`. +- `crate::core::all` / `crate::core` — `RegisteredController`, `ControllerFuture`, `ControllerSchema`, `FieldSchema`, `TypeSchema` for RPC registration. +- `crate::rpc::RpcOutcome` — controller/op return contract. +- External crates: `lettre` (email message + attachment), `base64`, `chrono`, `uuid`. + +## Used by + +- `src/core/all.rs` — registers the controllers/schemas into the JSON-RPC + CLI surface. +- `src/openhuman/tools/ops.rs` — instantiates the three tools into the agent tool registry; `src/openhuman/tools/mod.rs` re-exports them. + +## Notes / gotchas + +- `provider`/`format` are coupled: `piper` only emits `wav`, `cloud` only emits `mp3` — mismatches are hard errors (`resolve_format`). After synthesis the returned MIME is re-checked against the requested format (`enforce_audio_format`). +- Default voice is only injected for `piper` (`DEFAULT_PIPER_VOICE`); `cloud` defaults to no explicit voice. +- `email_podcast` re-validates `audio_path` (workspace-relative, no `..`) independently of `generate_podcast`; the combined flow overwrites the email request's `audio_path` with the freshly generated file's path. +- Email capture mode is feature/env-gated: enabled under `feature = "e2e-test-support"` or when `OPENHUMAN_EMAIL_CAPTURE_DIR` is non-empty; otherwise SMTP send requires `channels_config.email` to be configured (`from_address` falls back to `openhuman@localhost.test`). +- `AudioEmailDeliveryResult.mode` is `"capture"` or `"smtp"` to indicate which path ran. diff --git a/src/openhuman/autocomplete/README.md b/src/openhuman/autocomplete/README.md new file mode 100644 index 000000000..8acc26d90 --- /dev/null +++ b/src/openhuman/autocomplete/README.md @@ -0,0 +1,100 @@ +# autocomplete + +System-wide inline text autocomplete for macOS. The engine captures the user's currently-focused text field via the accessibility (AX) middleware, runs **local** (on-device) inference to generate a short single-line continuation, renders it in a floating overlay badge, and applies it on Tab (or rejects on Escape). Accepted completions are persisted as personalisation examples that feed back into later inference. macOS-only at runtime; on other platforms `start` returns an error and most pieces compile to no-ops. There is also an in-app path (context override) used by the OpenHuman composer that bypasses AX capture. + +## Responsibilities + +- Run a background tokio loop that, on each debounce tick, captures focused-field context, calls local inference, and updates engine state with a sanitized suggestion. +- Detect Tab (accept) and Escape (reject) key edges and apply/clear the suggestion via the accessibility middleware (with focused-target re-validation before insertion). +- Render/hide the overlay badge anchored to the focused element, with an osascript notification fallback. +- Special-case terminal apps (extract just the input line) and skip blocked/disabled apps and OpenHuman's own window (the in-app React path handles that). +- Filter low-quality suggestions (too short, no alphanumerics, echo of typed tail). +- Auto-stop after `MAX_CONSECUTIVE_ERRORS` (5) consecutive failures to avoid notification floods; self-heal stuck `generating` phase; short-circuit Apple Events automation denial (`-1743`) to avoid re-firing the macOS consent popup. +- Persist accepted completions to the local KV store and a local memory-doc namespace; surface, list, and clear that history. +- Expose status/start/stop/current/debug_focus/accept/set_style/history/clear_history over the JSON-RPC + CLI controller registry, plus a CLI serve/spawn runner. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/autocomplete/mod.rs` | Export-focused root. Re-exports `core::*`, history helpers, `ops` (also aliased as `rpc`), and the controller schema/registry pair. | +| `src/openhuman/autocomplete/ops.rs` | Controller/CLI surface: `autocomplete_*` async fns returning `RpcOutcome` with structured `[autocomplete]` logs; defines history params/results; `autocomplete_start_cli` (spawn/serve/foreground modes). Holds the ops unit tests. | +| `src/openhuman/autocomplete/schemas.rs` | `all_controller_schemas`, `all_registered_controllers`, per-function `ControllerSchema`, and `handle_*` thunks delegating to `ops`. | +| `src/openhuman/autocomplete/history.rs` | Persistence + personalisation: `save_accepted_completion`, `save_completion_to_local_docs`, `query_relevant_examples`, `load_recent_examples`, `list_history`, `clear_history`, and the `AcceptedCompletion` type. | +| `src/openhuman/autocomplete/core/mod.rs` | Engine submodule root; re-exports the engine and public types. | +| `src/openhuman/autocomplete/core/engine.rs` | `AutocompleteEngine` + global singleton (`AUTOCOMPLETE_ENGINE`, `global_engine`, `start_if_enabled`). Owns `EngineState`, the background refresh loop, `refresh`, `accept`, Tab/Escape handlers, quality/tab-artifact heuristics. | +| `src/openhuman/autocomplete/core/types.rs` | Serde DTOs (`AutocompleteStatus`, `Autocomplete{Start,Stop,Current,Accept,SetStyle}Params/Result`, `AutocompleteSuggestion`, `AutocompleteDebugFocusResult`); re-exports `FocusedTextContext`; `MAX_SUGGESTION_CHARS = 64`. | +| `src/openhuman/autocomplete/core/focus.rs` | Thin re-export of accessibility focus/insert/key-probe fns. | +| `src/openhuman/autocomplete/core/overlay.rs` | `show_overflow_badge` / `overlay_helper_quit` — overlay rendering via accessibility middleware with osascript notification fallback (macOS-gated). | +| `src/openhuman/autocomplete/core/terminal.rs` | Re-exports accessibility terminal detection/extraction helpers. | +| `src/openhuman/autocomplete/core/text.rs` | `sanitize_suggestion`, `truncate_head`, `is_no_text_candidate_error` and re-exported `truncate_tail`. | +| `src/openhuman/autocomplete/core/engine_tests.rs` | Engine unit tests (`#[path]`-included). | + +## Public surface + +- Engine: `AutocompleteEngine`, `AUTOCOMPLETE_ENGINE`, `global_engine()`, `start_if_enabled(&Config)`. +- Types: `AutocompleteStatus`, `AutocompleteSuggestion`, and the `*Params`/`*Result` DTOs listed above. +- Ops (also re-exported as `rpc`): `autocomplete_status`, `autocomplete_start`, `autocomplete_stop`, `autocomplete_current`, `autocomplete_debug_focus`, `autocomplete_accept`, `autocomplete_set_style`, `autocomplete_history`, `autocomplete_clear_history`, `autocomplete_start_cli`. +- History: `AcceptedCompletion`, `list_history`, `clear_history`, `load_recent_examples`, `query_relevant_examples`, `save_accepted_completion`, `save_completion_to_local_docs`. +- Schema pair: `all_autocomplete_controller_schemas`, `all_autocomplete_registered_controllers`. + +## RPC / controllers + +Namespace `autocomplete` (RPC methods `openhuman.autocomplete_` / CLI `autocomplete `): + +| Function | Inputs | Output type | Purpose | +| --- | --- | --- | --- | +| `status` | — | `AutocompleteStatus` | Engine status + latest suggestion metadata. | +| `start` | `debounce_ms?` | `AutocompleteStartResult` | Start engine (clamped 50–2000 ms). | +| `stop` | `reason?` | `AutocompleteStopResult` | Stop engine, abort loop, quit overlay. | +| `current` | `context?` | `AutocompleteCurrentResult` | Compute suggestion for captured or explicit context (in-app path). | +| `debug_focus` | — | `AutocompleteDebugFocusResult` | Inspect focused element/text diagnostics. | +| `accept` | `suggestion?`, `skip_apply?` | `AutocompleteAcceptResult` | Apply (or mark accepted) and persist a completion. | +| `set_style` | `enabled?`, `debounce_ms?`, `max_chars?`, `style_preset?`, `style_instructions?`, `style_examples?`, `disabled_apps?`, `accept_with_tab?`, `overlay_ttl_ms?` | `AutocompleteSetStyleResult` | Update `[autocomplete]` config; auto-starts engine when `enabled=true`. | +| `history` | `limit?` (default 20) | `{ entries: [AcceptedCompletion] }` | List recent accepted completions, newest first. | +| `clear_history` | — | `{ cleared: u64 }` | Delete all history (KV + local docs). | + +Schemas/handlers wired into the registry via `src/core/all.rs` (no domain branches in `cli.rs`/`jsonrpc.rs`). CLI help/serve/spawn flags are adapted by `src/core/autocomplete_cli_adapter.rs`. + +## Agent tools + +None. This domain owns no agent tools (`tools.rs` absent). + +## Events + +None. No `bus.rs`; the engine drives itself via an internal tokio loop and does not publish/subscribe to `DomainEvent`s. + +## Persistence + +History (`history.rs`) writes through `MemoryClient::new_local()` (local KV / docs under the default OpenHuman dir), in two layers: + +- **KV namespace `autocomplete`** — `AcceptedCompletion` JSON keyed by zero-padded timestamp (`accepted:{ts:018}`) so lexical order == reverse-chronological; trimmed to `MAX_HISTORY_ENTRIES` (50). Powers the settings list (`list_history`) and recency examples (`load_recent_examples`). +- **Doc namespace `autocomplete-memory`** — formatted `"[app] ...tail → suggestion"` documents (source_type `autocomplete`, priority `low`, category `daily`), trimmed to `MAX_DOC_ENTRIES` (200), queried semantically by `query_relevant_examples`. + +Config (`[autocomplete]` in the TOML `Config`) is the durable settings store, mutated by `set_style`. + +## Dependencies + +- `crate::openhuman::accessibility` — focused-text capture, text insertion, Tab/Escape/modifier key probes, terminal detection, overlay rendering, Swift helper precompile, and Apple Events automation-denial flags. The core engine's platform muscle. +- `crate::openhuman::config` (`Config`, `AutocompleteConfig`) — load/save settings; the gate for `enabled`, debounce, style, disabled apps, overlay TTL. +- `crate::openhuman::inference::local` (`local_ai`) — on-device inference via `inline_complete_interactive` (interactive variant bypasses the scheduler LLM permit for low keystroke latency). +- `crate::openhuman::memory_store` (`MemoryClient`, `NamespaceDocumentInput`) — local KV + doc persistence for accepted-completion history. +- `crate::core::all` (`ControllerFuture`, `RegisteredController`) and `crate::core::{ControllerSchema, FieldSchema, TypeSchema}` + `crate::rpc::RpcOutcome` — controller registry plumbing. + +## Used by + +- `src/core/all.rs` — registers the controller schemas/handlers. +- `src/core/autocomplete_cli_adapter.rs` — CLI parsing/help/serve-spawn adapter for the `autocomplete` namespace; `src/core/cli.rs` and `src/core/logging.rs` consume it (including the `--autocomplete-logs` run scope). +- `src/openhuman/app_state/ops.rs` — app-state/startup wiring. +- `src/openhuman/accessibility/*` and `src/openhuman/credentials/ops.rs` — adjacent references (automation/permission flow). + +## Notes / gotchas + +- **macOS-only at runtime.** `start` returns `Err("autocomplete is only supported on macOS")`; `platform_supported` in status is `cfg!(target_os = "macos")`. Overlay/notification/focus-validation code is `#[cfg(target_os = "macos")]`-gated. +- **Single process-global engine.** RPC and the embedded startup share `AUTOCOMPLETE_ENGINE`, so tests serialize on a mutex; history integration tests are `#[ignore]` because they hit the real on-disk KV store. +- **OpenHuman-self skip vs in-app override.** The background loop skips AX refresh when OpenHuman is frontmost (React composer handles it); passing `context` to `current` forces inference even for OpenHuman and skips the (latency-costly) history example lookups. +- **Tab artifact cleanup.** Before applying on Tab, the engine detects and backspaces a trailing tab/indentation the app may have inserted (`detect_tab_artifact_suffix`). +- **Confidence is a placeholder** (`0.75`) until `inline_complete` surfaces a real score. +- **Debounce clamped 50–2000 ms; `max_chars` clamped 64–2048; `overlay_ttl_ms` clamped 300–10000.** `MAX_SUGGESTION_CHARS` (64) caps the displayed/applied suggestion regardless of `max_chars` (which bounds *input* context). +- **`set_style` with `enabled=true` auto-starts** the engine and logs the result; disabling aborts the loop and quits the overlay. +- **Refresh has a 120 s timeout**; on timeout with metadata drift the stale suggestion is cleared. A stale `generating` phase older than 12 s self-heals. diff --git a/src/openhuman/billing/README.md b/src/openhuman/billing/README.md new file mode 100644 index 000000000..a730a9070 --- /dev/null +++ b/src/openhuman/billing/README.md @@ -0,0 +1,88 @@ +# billing + +Thin RPC adapter domain over the hosted backend's payment API. It exposes plan lookup, Stripe/Coinbase purchase and top-up flows, credit balance/transactions, auto-recharge + saved-card management, and coupon redemption through the standard controller registry (`openhuman.billing_*`). It holds **no payment logic or state of its own** — every operation forwards an authenticated HTTPS request to the backend (`/payments/*`, `/coupons/*`) and surfaces the JSON response verbatim. Authorization, plan ownership, tenant isolation, and payment policy are enforced backend-side. + +## Responsibilities + +- Authenticate each call with the stored app-session JWT (`Authorization: Bearer …`) and reject calls with no session. +- Fetch current plan/entitlements (`/payments/stripe/currentPlan`) and credit balance (`/payments/credits/balance`). +- Create Stripe Checkout sessions (`purchase_plan`), the Stripe customer portal session, and Stripe SetupIntents for adding cards. +- Initiate credit top-ups via Stripe or Coinbase, and create Coinbase Commerce charges (crypto / annual billing). +- Page credit transaction history; read/update Stripe auto-recharge settings; list/update/delete saved cards. +- Redeem coupon codes and list the current user's redeemed coupons. +- Do **pre-HTTP input validation** (non-empty plan/code/paymentMethodId, finite positive `amountUsd`, gateway whitelist `stripe`/`coinbase`) before any network call. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/billing/mod.rs` | Export-focused module root; re-exports `ops::*` and the schemas/controllers pair. | +| `src/openhuman/billing/ops.rs` | Business logic: one async fn per backend endpoint; auth helper (`require_token`, `get_authed_value`); input validation + gateway normalization. Returns `RpcOutcome`. | +| `src/openhuman/billing/schemas.rs` | Controller schemas, `all_billing_controller_schemas` / `all_billing_registered_controllers`, param structs, and `handle_billing_*` handlers delegating to `ops`. | +| `src/openhuman/billing/schemas_tests.rs` | Sibling test suite for `schemas.rs` (wired via `#[path]` mod). | + +## Public surface + +From `mod.rs`: + +- `ops::*` — async handlers: `get_current_plan`, `get_balance`, `get_transactions`, `get_auto_recharge`, `update_auto_recharge`, `get_cards`, `create_setup_intent`, `update_card`, `delete_card`, `purchase_plan`, `create_portal_session`, `top_up_credits`, `create_coinbase_charge`, `redeem_coupon`, `get_user_coupons`. Each takes `&Config` (plus typed params) and returns `Result, String>`. +- `all_billing_controller_schemas()`, `all_billing_registered_controllers()`, `billing_schemas(function: &str)` — registry wiring. + +## RPC / controllers + +Namespace `billing` (15 methods, exposed as `openhuman.billing_*`): + +| Method | Backend endpoint | +| --- | --- | +| `billing_get_current_plan` | `GET /payments/stripe/currentPlan` | +| `billing_get_balance` | `GET /payments/credits/balance` | +| `billing_get_transactions` | `GET /payments/credits/transactions?limit&offset` | +| `billing_get_auto_recharge` | `GET /payments/credits/auto-recharge` | +| `billing_update_auto_recharge` | `PATCH /payments/credits/auto-recharge` | +| `billing_get_cards` | `GET /payments/credits/auto-recharge/cards` | +| `billing_create_setup_intent` | `POST /payments/credits/auto-recharge/cards/setup-intent` | +| `billing_update_card` | `PATCH /payments/credits/auto-recharge/cards/{paymentMethodId}` | +| `billing_delete_card` | `DELETE /payments/credits/auto-recharge/cards/{paymentMethodId}` | +| `billing_purchase_plan` | `POST /payments/stripe/purchasePlan` | +| `billing_create_portal_session` | `POST /payments/stripe/portal` | +| `billing_top_up` | `POST /payments/credits/top-up` | +| `billing_create_coinbase_charge` | `POST /payments/coinbase/charge` | +| `billing_redeem_coupon` | `POST /coupons/redeem` | +| `billing_get_coupons` | `GET /coupons/me` | + +Handlers load `Config` via `config::rpc::load_config_with_timeout()`, deserialize camelCase params, call the matching `ops` fn, and emit CLI-compatible JSON via `RpcOutcome::into_cli_compatible_json()`. + +## Agent tools + +None. This domain has no `tools.rs`; it is RPC/CLI-only. + +## Events + +None. No `bus.rs`; the domain neither publishes nor subscribes to `DomainEvent`s. + +## Persistence + +None of its own; stateless adapter. The only state it reads is the backend session JWT, which lives in the `api` layer (see `auth_store_session` / `get_session_token`), not in this module. + +## Dependencies + +- `crate::api::config::effective_backend_api_url` — resolves the backend base URL from `config.api_url`. +- `crate::api::jwt::get_session_token` — reads the stored app-session JWT. +- `crate::api::BackendOAuthClient` — performs the authenticated JSON HTTP request (`authed_json`). +- `crate::openhuman::config::Config` — config struct (`api_url`); `config::rpc::load_config_with_timeout` in handlers. +- `crate::rpc::RpcOutcome` — standard RPC return/logging wrapper. +- `crate::core::all::{ControllerFuture, RegisteredController}` and `crate::core::{ControllerSchema, FieldSchema, TypeSchema}` — controller registry types. +- External crates: `reqwest` (`Method`), `serde`/`serde_json`, `urlencoding` (path-segment encoding for `paymentMethodId`). + +## Used by + +- `src/core/all.rs` — registers `all_billing_registered_controllers()` (controllers, ~L215) and `all_billing_controller_schemas()` (schemas, ~L346) into the global registry. This is the sole in-tree consumer. + +## Notes / gotchas + +- **No local authorization** — security is delegated entirely to the backend; a missing/invalid session yields the backend 401/403 surfaced verbatim as an RPC error string. JWTs/API keys are never logged. +- `top_up_credits` and `create_coinbase_charge` default gateway/interval (`stripe`, `annual`); `normalize_gateway` lowercases and whitelists `stripe`/`coinbase`, treating empty/whitespace as default-to-stripe. +- `amountUsd` must be finite and `> 0` (NaN/±Inf/≤0 rejected pre-HTTP). +- `paymentMethodId` is `urlencoding::encode`'d into the path; empty/whitespace ids are rejected. +- `get_transactions` defaults to `limit=20`, `offset=0`; the handler tolerates an empty params map. +- Input-validation unit tests live inline in `ops.rs` and run without network/session/filesystem state. diff --git a/src/openhuman/codegraph/README.md b/src/openhuman/codegraph/README.md new file mode 100644 index 000000000..6d1e86426 --- /dev/null +++ b/src/openhuman/codegraph/README.md @@ -0,0 +1,75 @@ +# codegraph + +Content-addressed code retrieval for coding subagents — the seed engine behind the issue-crusher / pr-reviewer skills. Given a checked-out git worktree, it indexes the tree's code files and answers "which files are most relevant to this query" by fusing a lexical (BM25) arm with a dense (structural-aug embedding) arm via reciprocal-rank fusion. Indexing is content-addressed: every file's `{BM25 tokens, structural-doc embedding}` is cached by its git **blob SHA** (+ embedding-model signature), and a branch's index is just a per-`(repo_id, ref)` **manifest** joined to that shared blob cache at query time — so a branch switch, new commit, or pull only (re)processes the blobs that actually changed. Pure Rust: git CLI for tree enumeration, `rusqlite` for storage, the `embeddings` domain (cloud-default) for vectors. No Python, no extra services. + +## Responsibilities + +- Enumerate tracked code files at a checkout (`git ls-files -s`, filtered to a fixed code-extension set, `≤ 100 KB`/file). +- Extract per-file content: identifier-split BM25 tokens (camelCase / snake_case → sub-words) and a heuristic "structural doc" (definition signatures + imports + called-symbol identifiers + leading doc/comments). +- Embed structural docs in batches (`≤ 128`/call) via the injected `EmbeddingProvider`, L2-normalising each vector. +- Cache `{tokens, emb}` by `(blob_sha, model)` in SQLite; rewrite the `(repo_id, ref)` manifest to the current tree (handles deletes/renames). +- Two index modes: `Lexical` (BM25-only, no embedder call) and `Dense` (structural-aug vectors + BM25). A size-gated `auto` picks between them. +- Search: hydrate the working set, rank with BM25-Okapi and cosine, RRF-fuse top-`PER_ARM` of each arm into top-`k`, and report a `Coverage` flag (`full`/`partial`/`none`). +- Expose `codegraph_index` and `codegraph_search` agent tools, workspace-sandboxed. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/codegraph/mod.rs` | Module docstring + `pub mod` decls + `pub use` re-exports. No logic. | +| `src/openhuman/codegraph/index.rs` | Indexing pipeline: tree enumeration (`tree_blobs`), `code_tokens`, `structural_doc`, `current_ref`, `count_code_files`, and the 3-phase `index_ref` (extract uncached blobs → batch-embed → persist + rewrite manifest). Defines `IndexMode`, `IndexReport`, `LEXICAL_MODEL`. | +| `src/openhuman/codegraph/search.rs` | Retrieval: `search_ref` hydrates the working set, runs `bm25_rank` + `dense_rank`, `rrf`-fuses, computes `Coverage`. Defines `SearchOutcome`, `Coverage`. | +| `src/openhuman/codegraph/store.rs` | SQLite persistence (`CodegraphStore`): `blob(sha,model,…)` content cache + `manifest(repo_id,git_ref,path,sha)`. `open`, `has_blob`, `put_blob(s)`, `set_manifest`, `hydrate`, `manifest_size`, `refs`, `gc`. Defines `BlobEntry`. | +| `src/openhuman/codegraph/tools.rs` | Agent tools `CodegraphIndexTool` / `CodegraphSearchTool` — arg parsing, workspace sandboxing, size-gated `auto` mode, provider/store wiring. | + +## Public surface + +Re-exported from `mod.rs`: + +- **index**: `code_tokens`, `count_code_files`, `current_ref`, `index_ref`, `structural_doc`, `IndexMode` (`Lexical`/`Dense`), `IndexReport`, `LEXICAL_MODEL`. +- **search**: `search_ref`, `Coverage` (`Full`/`Partial`/`None`), `SearchOutcome` (`hits`, `coverage`, `indexed`, `total`). +- **store**: `CodegraphStore`, `BlobEntry` (`path`, `tokens`, `emb`). +- **tools**: `tools::{CodegraphIndexTool, CodegraphSearchTool}` (re-exported through `openhuman::tools::mod`). + +## Agent tools + +Owned in `tools.rs`, registered in `src/openhuman/tools/ops.rs`: + +| Tool | Args | Returns | +| --- | --- | --- | +| `codegraph_index` | `path` (required), `ref?` (default current checkout), `mode?` (`auto`\|`lexical`\|`dense`) | `{mode, files, computed, cached, skipped}` | +| `codegraph_search` | `query` (required), `path` (required), `ref?`, `k?` (default 10) | `{hits:[paths], coverage:full\|partial\|none, indexed, total}` | + +`codegraph_search` is index-first: if the `(repo, ref)` has no manifest, it auto-indexes synchronously (size-gated mode) before searching. Both tools canonicalize `path` and refuse any repo outside the workspace (`resolve_repo_dir`). `mode`/`auto` threshold is governed by `OPENHUMAN_CODEGRAPH_DENSE_MIN_FILES` (default 400 files → dense, else lexical). + +## Persistence + +SQLite DB at `/codegraph/index.db` (WAL journal, `synchronous=NORMAL` — a rebuildable cache). Two tables: + +- `blob(sha, model, tokens, emb, dim)` PK `(sha, model)` — shared content cache: one row per unique file content per embedding-model key. `tokens` is the space-joined BM25 stream; `emb` is the L2-normalised vector as little-endian `f32` bytes. Shared across every repo/branch, so renames and unchanged files are free. +- `manifest(repo_id, git_ref, path, sha)` PK `(repo_id, git_ref, path)`, with index `manifest_repo_ref` — one row per file per branch/commit. A ref's index is its rows joined to `blob`. `gc()` drops blobs no live manifest references. + +`repo_id` is the canonical worktree path; the `model` key is `LEXICAL_MODEL` (`codegraph:lexical:v1`) for lexical indexes or the embedding provider's `signature()` for dense. + +## Dependencies + +- `crate::openhuman::embeddings` — `EmbeddingProvider` trait (injected `&dyn`, unit-tested with fakes) for embedding structural docs/queries; `provider_from_config` builds the configured cloud-default provider in the tools. The provider's `signature()` is the blob-cache model key. +- `crate::openhuman::config::Config` — read by the tools to build the embedding provider. +- `crate::openhuman::tools::traits::{Tool, ToolResult}` — the agent-tool trait the two tools implement. +- External crates: `rusqlite` (+FTS5-era schema; current ranking is in-memory BM25, not FTS5), `anyhow`, `serde`/`serde_json`, `async_trait`, `tracing`. Git is shelled out via `std::process::Command` (no libgit2). + +## Used by + +- `src/openhuman/tools/mod.rs` re-exports the tools (`pub use crate::openhuman::codegraph::tools::*`); `src/openhuman/tools/ops.rs` constructs `CodegraphIndexTool` / `CodegraphSearchTool` into the agent tool registry. +- Registered as the domain `pub mod codegraph;` in `src/openhuman/mod.rs`. + +## Notes / gotchas + +- **No RPC controller / no event-bus subscriber / no config block of its own.** Surface is agent tools only; there is no `schemas.rs`, `rpc.rs`, or `bus.rs`. Behavior tunables come from env vars (`OPENHUMAN_CODEGRAPH_DENSE_MIN_FILES`), not the TOML config. +- The structural extractor is a **dependency-free heuristic**, not tree-sitter — the docstrings note a tree-sitter upgrade is intended to slot in behind `structural_doc` (the cached *content* would then change, so the `model` key isolates it). +- Empty-doc guard: a structure-less file (empty `__init__.py`, `x = 1`) would yield an empty structural doc, which cloud embedders reject (400 on empty input). `index_ref` falls back to the lexical tokens (or `"(no extractable content)"`) so embed inputs are never empty. +- `LEXICAL_MODEL` is deliberately a separate cache key from any embedder signature, so a later dense pass indexes fresh rather than colliding with embedding-less rows. +- BM25 runs **in-memory** over the hydrated per-repo working set (not SQLite FTS5) — the working set is one repo's files, kept small; the module docstring's "SQLite FTS5" framing describes the design lineage, the implemented lexical arm is `bm25_rank`. +- `Coverage` is `indexed / total` (manifest size): files whose blob isn't cached (skipped/oversized/pending) are omitted from `hydrate`, dropping coverage to `Partial`. On non-`Full` coverage the tool description tells the agent to treat hits as hints and also use grep. +- Search auto-detects the index mode: it first hydrates under the embedder signature (dense), and if empty falls back to the lexical key — lexical search makes no embedder round-trip. +- Several live/benchmark tests in `index.rs` are `#[ignore]` (need `OPENHUMAN_WORKSPACE` + a valid backend session, or `CODEGRAPH_BENCH_REPO`). diff --git a/src/openhuman/composio/README.md b/src/openhuman/composio/README.md new file mode 100644 index 000000000..807885101 --- /dev/null +++ b/src/openhuman/composio/README.md @@ -0,0 +1,138 @@ +# composio + +Backend-proxied (and optionally direct/BYO-key) access to Composio's 1000+ OAuth integrations (Gmail, Notion, GitHub, Slack, Google Calendar, …). The Rust counterpart to the backend routes under `src/routes/agentIntegrations/composio.ts`. In **backend mode** the openhuman backend owns the Composio API key, billing/margin, the toolkit allowlist, HMAC webhook verification, and Socket.IO trigger fan-out — the core never hits the Composio API directly. In **direct mode** (`composio.mode = "direct"`, gated by a user-supplied API key in the encrypted keychain) the core talks to Composio's v3 API with the user's own key against their personal tenant. This domain exposes toolkit/connection/tool/trigger management over JSON-RPC, model-facing agent tools for discovery + execution, an OAuth handoff flow, a persistent trigger-event archive, and a per-action execute pipeline (prepare → retry → error classification). + +## Responsibilities + +- List toolkits, connections, agent-ready toolkits, and a local capability matrix. +- Begin OAuth handoffs (`authorize`) and delete connections (with optional source-scoped memory cleanup). +- Discover Composio action tool schemas (`list_tools`) and execute actions (`execute`), mode-aware over the backend/direct split. +- Manage triggers: list available/active, create, enable, disable, plus a persistent trigger-event history archive. +- Fetch normalized per-toolkit user profiles and refresh stored identities; dispatch sync passes to per-toolkit providers. +- Gate agent action visibility/execution by per-toolkit user scope preferences (read/write/admin) and curated catalogs. +- Manage the Composio routing mode and the direct-mode API key (`get_mode` / `set_api_key` / `clear_api_key`). +- Classify execute failures into stable error classes; funnel op-layer errors to Sentry under `domain="composio"`. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/composio/mod.rs` | Export-focused module root; re-exports types, ops, schemas, agent tools, trigger-history, and (via `memory_sync::composio`) bus/periodic/providers. | +| `src/openhuman/composio/types.rs` | Serde domain types mirroring backend response envelopes (toolkits, connections, tools, execute, triggers, trigger events/history). Includes drift-tolerant `de_string_or_object` deserializers. | +| `src/openhuman/composio/ops.rs` | RPC-facing `composio_*` operations returning `RpcOutcome`; mode-aware client routing, memory-cleanup targets on delete, provider dispatch for profile/sync, mode + api-key ops, Sentry funnel `report_composio_op_error`. | +| `src/openhuman/composio/schemas.rs` | Controller schemas + `handle_*` handlers; `all_controller_schemas` / `all_registered_controllers`. | +| `src/openhuman/composio/client.rs` | `ComposioClient` (thin HTTP wrapper over `IntegrationClient` for backend routes) + `ComposioClientKind` (Backend/Direct), `create_composio_client`, and direct-mode v3 helpers (`direct_list_connections`, `direct_list_tools`, `direct_execute`, `direct_authorize`). | +| `src/openhuman/composio/tools.rs` | Agent tools (`ComposioListToolkitsTool`, `ComposioListConnectionsTool`, `ComposioAuthorizeTool`, `ComposioListToolsTool`, `ComposioExecuteTool`) + `all_composio_agent_tools`; scope/visibility gating (`resolve_action_scope`, `evaluate_tool_visibility`). | +| `src/openhuman/composio/tools/direct.rs` | Direct-mode Composio tool provider hitting Composio v2/v3 APIs with the user's key (`ComposioTool`, `ComposioAction`, `ComposioConnectedAccount`). | +| `src/openhuman/composio/action_tool.rs` | `ComposioActionTool` — a `Tool` wrapping exactly one Composio action, constructed dynamically when `integrations_agent` is spawned with a toolkit. | +| `src/openhuman/composio/execute_dispatch.rs` | Centralized execute path: prepare args → retry policy (auth/rate-limit) → error mapping; mode-aware over backend/direct. | +| `src/openhuman/composio/execute_prepare.rs` | Local pre-flight argument validation/preparation for action calls. | +| `src/openhuman/composio/auth_retry.rs` | Single-shot retry for the post-OAuth token-propagation gap ("Connection error, try to authenticate"). | +| `src/openhuman/composio/error_mapping.rs` | `ComposioErrorClass` + classifier/formatter so tool failures aren't bucketed as generic gateway 502s (#1797). | +| `src/openhuman/composio/oauth_handoff.rs` | OAuth handoff helpers + Meta (Instagram/Facebook) rate-limit mitigations (#1952); rate-limit error wrapping. | +| `src/openhuman/composio/googlecalendar_args.rs` | Default-args transformer for Google Calendar list/find slugs (timezone/`singleEvents` defaults, #1714). | +| `src/openhuman/composio/identity.rs` | Resolves the connected account username for a toolkit via the provider `fetch_user_profile` path (used by skill preflight identity gate). | +| `src/openhuman/composio/trigger_history.rs` | Persistent JSONL trigger-event archive partitioned by UTC day; global `OnceLock` store (`init_global`/`global`). | +| `src/openhuman/composio/bus.rs` | Compatibility shim re-exporting `memory_sync::composio::bus`. | +| `src/openhuman/composio/periodic.rs` | Compatibility shim re-exporting `memory_sync::composio::periodic` (periodic sync loop). | +| `src/openhuman/composio/providers/mod.rs` | Compatibility shim re-exporting `memory_sync::composio::providers` (native per-toolkit provider registry, curated catalogs, user-scope prefs). | +| `*_tests.rs` | Sibling test suites for each file. | + +## Public surface + +From `mod.rs` re-exports: + +- **Client**: `ComposioClient`, `ComposioActionTool`. +- **Ops**: `cached_active_integrations`, `connected_set_hash`, `fetch_connected_integrations`, `fetch_connected_integrations_status`, `FetchConnectedIntegrationsStatus`, `fetch_toolkit_actions`, `invalidate_connected_integrations_cache`. +- **Schemas**: `all_composio_controller_schemas`, `all_composio_registered_controllers`. +- **Agent tools**: `all_composio_agent_tools`. +- **Identity**: `connection_identity`. +- **Trigger history**: `init_composio_trigger_history`, `global_composio_trigger_history`. +- **Types**: `ComposioConnection`, `ComposioConnectionsResponse`, `ComposioToolkitsResponse`, `ComposioToolSchema`/`ComposioToolFunction`, `ComposioToolsResponse`, `ComposioAuthorizeResponse`, `ComposioExecuteResponse`, `ComposioDeleteResponse`, `ComposioCapability`/`ComposioCapabilitiesResponse`, `ComposioAgentReadyToolkitsResponse`, `ComposioTriggerEvent`/`ComposioTriggerMetadata`, `ComposioTriggerHistoryEntry`/`ComposioTriggerHistoryResult`. +- **Re-exported from `memory_sync::composio`**: `ComposioProvider`, `ProviderContext`, `ProviderUserProfile`, `SyncOutcome`, `SyncReason`, `all_composio_providers`, `get_composio_provider`, `init_default_composio_providers`; `ComposioTriggerSubscriber`, `ComposioConfigChangedSubscriber`, `register_composio_trigger_subscriber`; `record_sync_success`, `start_periodic_sync`. + +## RPC / controllers + +Namespace `composio`, exposed as `openhuman.composio_*`: + +| Method | Purpose | +| --- | --- | +| `composio.list_toolkits` | Backend allowlist of enabled toolkits (empty in direct mode). | +| `composio.list_capabilities` | Local capability matrix (no signed-in session needed). | +| `composio.list_agent_ready_toolkits` | Toolkit slugs that ship a curated agent catalog (#2283). | +| `composio.list_connections` | Active OAuth connections (mode-aware; reconciles integrations cache). | +| `composio.authorize` | Begin OAuth handoff; returns `connectUrl` + `connectionId`. | +| `composio.delete_connection` | Delete connection; optional source-scoped memory cleanup. | +| `composio.list_tools` | OpenAI function-calling tool schemas (optional toolkit/tag filter). | +| `composio.execute` | Execute an action slug with `{tool, arguments}`. | +| `composio.list_github_repos` | Repos for an authorized GitHub connection. | +| `composio.create_trigger` | Create a trigger instance for a connection. | +| `composio.list_available_triggers` | Catalog of enableable triggers for a toolkit. | +| `composio.list_triggers` | Currently enabled triggers. | +| `composio.enable_trigger` / `composio.disable_trigger` | Enable / delete a trigger. | +| `composio.list_trigger_history` | Recent archived trigger events + JSONL archive paths. | +| `composio.get_user_profile` | Normalized provider profile for a connection. | +| `composio.refresh_all_identities` | Re-fetch + persist identities for all active connections (#1365). | +| `composio.sync` | Spawn a background provider sync pass (`manual`/`periodic`/`connection_created`). | +| `composio.get_user_scopes` / `composio.set_user_scopes` | Read/write per-toolkit read/write/admin scope prefs. | +| `composio.get_mode` | Current routing mode + whether a direct-mode key is set (never returns the key). | +| `composio.set_api_key` / `composio.clear_api_key` | Store/clear direct-mode Composio API key (key never logged/returned). | + +Handlers delegate to `ops.rs`; scope handlers delegate to `providers::user_scopes`. Exports wired into `src/core/all.rs`. + +## Agent tools + +From `tools.rs` (`all_composio_agent_tools`): `composio_list_toolkits`, `composio_list_connections`, `composio_authorize`, `composio_list_tools`, `composio_execute`. Plus `ComposioActionTool` (one tool per action, spawned for `integrations_agent`) and the direct-mode `ComposioTool` provider. Scope elevation is deliberately NOT an agent tool — the user toggles it in the UI. Visibility/execution is gated by curated catalogs + per-toolkit user-scope prefs and sandbox mode; unparseable slugs default to `Write` (fail-closed). + +## Events + +Subscribers/handlers live in `memory_sync::composio::bus` (re-exported here): + +- **`ComposioTriggerSubscriber`** — reacts to `DomainEvent::ComposioTriggerReceived` (published by `socket::event_handlers` when the backend emits `composio:trigger`); archives to trigger history and drives memory ingestion. +- **`ComposioConnectionCreatedSubscriber`** — reacts to `DomainEvent::ComposioConnectionCreated` (published by `composio_authorize`); eagerly warms the integrations cache. +- **`ComposioConfigChangedSubscriber`** — reacts to `DomainEvent::ComposioConfigChanged` (mode/api-key changes). + +Published from `ops.rs`: `DomainEvent::ComposioConnectionCreated` (authorize), `DomainEvent::ComposioConnectionDeleted` (delete), `DomainEvent::ComposioActionExecuted` (execute success/failure, with cost + elapsed). + +## Persistence + +- **Trigger history** (`trigger_history.rs`): JSONL records under `/state/triggers/YYYY-MM-DD.jsonl`, partitioned by UTC day, behind a process-global `OnceLock` store (file-locked via `fs2`; a process-local mutex on Windows). Exposed via `composio.list_trigger_history`. +- **Direct-mode API key**: stored in the encrypted keychain (via `credentials`); never logged/returned. +- **User scope prefs** + **identity facets / sync state**: persisted through the memory layer (`memory`, `memory_store`, `memory_tree`) by the providers in `memory_sync::composio` and by delete-time cleanup in `ops.rs`. +- **Integrations cache**: in-process cache of active connections (`cached_active_integrations` / `invalidate_connected_integrations_cache`), reconciled on each `list_connections`. + +## Dependencies + +- `crate::openhuman::integrations` — shared `IntegrationClient` (Bearer JWT, timeouts, envelope parsing, proxy) backing backend-mode calls. +- `crate::openhuman::config` — `Config` / `ComposioConfig` (`mode`, `entity_id`), `config::rpc` config loading. +- `crate::openhuman::memory`, `memory_store`, `memory_tree` — memory client + chunk store/tree for ingestion and connection-scoped memory cleanup. +- `crate::openhuman::memory_sync::composio` — the actual home of providers, periodic sync, and bus subscribers (this module re-exports them via shims). +- `crate::openhuman::agent::harness` — sandbox mode (`current_sandbox_mode` / `SandboxMode`) for tool gating. +- `crate::openhuman::tools::traits` — `Tool`, `ToolResult`, `ToolCategory`, `PermissionLevel`, `ToolCallOptions`. +- `crate::openhuman::security` — `SecurityPolicy` / `ToolOperation` for direct-tool gating. +- `crate::openhuman::credentials` — encrypted store for the direct-mode API key. +- `crate::openhuman::context::prompt`, `agent::prompts` — prompt/profile injection of connected identities. +- `crate::core::all` — `ControllerFuture` / `RegisteredController` registry types. +- `crate::core::event_bus` — `DomainEvent`, `publish_global`. +- `crate::core::observability` — Sentry error classification/reporting. +- `crate::rpc` — `RpcOutcome`. + +## Used by + +- `src/core/all.rs` — registers the controllers. +- `src/openhuman/tools/{mod,ops,schemas}.rs` — wires agent tools into the tool registry. +- `src/openhuman/agent/**` — harness/session/subagent spawning (`integrations_agent`), triage escalation, debug. +- `src/openhuman/socket/event_handlers.rs` — parses `composio:trigger` and publishes `ComposioTriggerReceived`. +- `src/openhuman/heartbeat/planner/collectors.rs`, `subconscious/situation_report` — read connected integrations / calendar. +- `src/openhuman/learning/{linkedin_enrichment,profile_md_renderer}.rs`, `memory/read_rpc.rs`, `memory_tree/score/store.rs` — profile/identity + memory consumers. +- `src/openhuman/skills/preflight.rs` — identity gate via `connection_identity`. +- `src/openhuman/credentials/ops.rs`, `channels/runtime/dispatch.rs`, `src/bin/slack_backfill.rs`. + +## Notes / gotchas + +- **`bus.rs`, `periodic.rs`, `providers/mod.rs` are compatibility shims** — the real implementations live under `src/openhuman/memory_sync/composio/`. Edit there. +- **Mode-aware routing (#1710)**: `authorize`/`execute`/`list_*` go through `create_composio_client` so a `composio.mode` toggle is honoured per call; direct mode never surfaces backend-tenant data (empty allowlist, user's own connections). Backend-only ops (`delete_connection`, `list_github_repos`, the `triggers/*` family, provider profile/sync) use `resolve_client` and require a backend session token. +- **`ops.rs` is large (~2380 lines)** — only the first ~1363 lines were read in full when authoring this doc; mode/api-key ops and `sync_cache_with_connections` were located by grep at lines ~1574–2354. +- **Error classification matters for the UI**: `execute` may return pre-classified `[composio:error:] …` strings (parsed by `app/src/lib/composio/formatters.ts`); `ops.rs` preserves them rather than re-wrapping. +- **Sentry funnel**: `report_composio_op_error` re-tags op-layer failures under `domain="composio"` with `failure="non_2xx"|"transport"` (+ extracted backend status) so transient 5xx leaks are dropped by `before_send` while genuine bugs surface. +- **Type drift tolerance**: trigger types use `de_string_or_object` / `de_opt_string_or_object` to accept upstream fields that flip between string and object shapes. diff --git a/src/openhuman/connectivity/README.md b/src/openhuman/connectivity/README.md new file mode 100644 index 000000000..4bca33629 --- /dev/null +++ b/src/openhuman/connectivity/README.md @@ -0,0 +1,65 @@ +# connectivity + +Diagnostics for the local core's reachability and the live backend Socket.IO state, plus the listen-port selection logic the core uses when it boots its embedded HTTP listener. The frontend has three independent connectivity channels — browser internet, backend Socket.IO websocket, and the local core HTTP — and issue #1527 split them in the UI so users see *which* channel is broken instead of one conflated "Disconnected" pill. This module exposes a cheap `openhuman.connectivity_diag` RPC that snapshots in-memory backend-socket state plus the local core's PID and listening port (no I/O beyond a single TCP probe), suitable for poll-based health checks. + +## Responsibilities + +- Answer `openhuman.connectivity_diag` with a flat snapshot: backend socket state, last websocket error, sidecar PID, configured listen port, and whether that port currently has a listener bound. +- Resolve the configured core RPC port from the environment (`OPENHUMAN_CORE_RPC_URL` then `OPENHUMAN_CORE_PORT`, defaulting to `7788`). +- Snapshot the backend Socket.IO connection state from the global `SocketManager` (reports `"uninitialized"` when the manager singleton isn't registered yet). +- Probe whether a TCP port on loopback is already bound (`is_port_in_use`). +- Pick a listen port for the embedded core HTTP listener (`pick_listen_port` / `pick_listen_port_for_host`): try preferred, retry transient `AddrInUse` races, request stale-listener takeover when another OpenHuman core owns the port (#1130), otherwise fall back to a port pool. +- Handle Windows OS-excluded port ranges (`WSAEACCES` / os error 10013, Sentry OPENHUMAN-TAURI-500) by routing straight to fallback ports instead of failing. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/connectivity/mod.rs` | Module docstring + exports; re-exports schema entry points as `all_connectivity_controller_schemas` / `all_connectivity_registered_controllers`. | +| `src/openhuman/connectivity/ops.rs` | Pure helper `is_port_in_use(port)` — binds an ephemeral loopback listener to detect an occupied port; unit-tested in isolation. | +| `src/openhuman/connectivity/rpc.rs` | Core logic: `ConnectivityDiagResponse`, `snapshot()`, `diag()` RPC, env port resolution, socket-state snapshot, and the `pick_listen_port*` family with fingerprinting (`identify_listener`) and OS-exclusion handling. | +| `src/openhuman/connectivity/schemas.rs` | Controller schema for the single `connectivity_diag` controller + `handle_diag` delegating to `rpc::diag()`. | + +## Public surface + +- `connectivity::all_connectivity_controller_schemas()` / `all_connectivity_registered_controllers()` — registry entry points (re-exported from `schemas.rs`). +- `connectivity::ops::is_port_in_use(port: u16) -> bool`. +- `connectivity::rpc::ConnectivityDiagResponse` — serialized diag payload (`socket_state`, `last_ws_error`, `sidecar_pid`, `listen_port`, `listen_port_in_use`). +- `connectivity::rpc::snapshot() -> ConnectivityDiagResponse` and `connectivity::rpc::diag() -> Result, String>`. +- `connectivity::rpc::pick_listen_port(preferred)` / `pick_listen_port_for_host(host, preferred)` → `Result`. +- `connectivity::rpc::PickListenPortResult` (`listener`, `port`, `fallback_from`) and `PickListenPortError` (`WouldTakeOver` / `NoAvailablePort` / `BindFailed`). + +## RPC / controllers + +- **`openhuman.connectivity_diag`** (namespace `connectivity`, function `diag`) — read-only, no inputs. Returns a single output field `diag` (JSON) containing the `ConnectivityDiagResponse` snapshot. Described as cheap and safe to poll. Registered via `all_registered_controllers` → `handle_diag` → `rpc::diag()`. + +Restart/mutate operations are intentionally **not** here — they live in the Tauri shell (`restart_core_process` in `app/src-tauri/src/lib.rs`) because they touch the host process tree and can't be answered from inside the core itself. + +## Persistence + +None — the module holds no state. The diag snapshot reads only the environment, the in-memory `SocketManager`, and a live TCP probe. + +## Dependencies + +- `crate::openhuman::socket::manager::global_socket_manager` — read the live backend Socket.IO `ConnectionStatus` and last error for the diag snapshot. +- `crate::core::all::{ControllerFuture, RegisteredController}` — controller registration types. +- `crate::core::{ControllerSchema, FieldSchema, TypeSchema}` — controller schema shape. +- `crate::rpc::RpcOutcome` — RPC return envelope (`RpcOutcome::single_log`). +- External crates: `reqwest` (HTTP fingerprint probe of a listener's `GET /` root), `tokio` (async `TcpListener`, retry backoff), `serde`/`serde_json`, `url`, `tracing`/`log`. + +## Used by + +- `src/core/all.rs` — registers the controller (`all_connectivity_registered_controllers`) and schema (`all_connectivity_controller_schemas`), and routes the `"connectivity"` namespace. +- `src/core/jsonrpc.rs` — calls `connectivity::rpc::pick_listen_port_for_host(...)` during core bind to select the embedded HTTP listener port; afterward syncs `OPENHUMAN_CORE_RPC_URL` to the actual bound port so `resolve_listen_port()` (and thus `connectivity_diag`) reports the live listener after a fallback. +- `src/openhuman/mod.rs` — declares the module. + +## Notes / gotchas + +- The `pick_listen_port` flow is the real workhorse despite the module's "diag" framing; it owns the core's startup port selection and the stale-listener takeover decision (#1130). +- Port resolution priority is `OPENHUMAN_CORE_RPC_URL` (port component) > `OPENHUMAN_CORE_PORT` > default `7788`. Invalid `OPENHUMAN_CORE_PORT` logs a warning and falls back to the default rather than failing. +- Fallback pool: when preferred is the default `7788`, fallbacks are `7789..=7798`; otherwise `preferred+1..=preferred+10` (saturating-checked). +- `WouldTakeOver` is only returned when something is actually listening (`AddrInUse`) **and** the listener fingerprints as an OpenHuman core (its `GET /` returns JSON with `"name":"openhuman"`). OS-excluded ports (Windows `WSAEACCES` / os error 10013) skip the takeover probe and route directly to fallbacks; `is_port_excluded_bind_error` matches on the raw OS code (10013) because Rust's `ErrorKind` mapping for it isn't stable across releases. +- IPv6 probe hosts are bracketed per RFC 3986 before building the fingerprint URL so live cores on IPv6 aren't misclassified as `Other`. +- `socket_state` is funneled through `serde_json` (not `Debug`) so the lowercased wire shape stays stable; `"uninitialized"` is reported when the `SocketManager` singleton isn't installed (early startup / tests). +- `is_port_in_use` returns `false` on non-`AddrInUse` bind errors (e.g. permission denied) so a port isn't misreported as occupied. +- No agent tools and no event-bus subscribers in this module. diff --git a/src/openhuman/context/README.md b/src/openhuman/context/README.md new file mode 100644 index 000000000..64c06b335 --- /dev/null +++ b/src/openhuman/context/README.md @@ -0,0 +1,95 @@ +# context + +Global context management for agent sessions: the single home for everything that shapes what an LLM sees during a conversation. It owns (1) **system-prompt assembly** (via a re-export of `agent::prompts` plus a bespoke channel-runtime prompt builder), (2) **mechanical history reduction** — a layered pipeline (tool-result budget → trim → microcompact → autocompact signal → session-memory trigger) that keeps the in-flight conversation inside the provider's context window, and (3) **summarization execution** — dispatching the LLM compaction call when the pipeline asks for autocompaction. Agents hold one `ContextManager` per session; all shared logic lives here so new agent archetypes and delegation tools do not re-wire any of it. This is a **pure logic / state-tracking domain** — no RPC controllers, no agent tools, no event-bus subscribers, no persisted store. + +## Responsibilities + +- Assemble opening system prompts (delegates to `agent::prompts` via `prompt.rs`; provides a separate bespoke builder for channel runtimes). +- Track context-window utilisation per session and decide when reduction must run (`ContextGuard`). +- Apply an inline per-tool-result byte cap before output enters history (`tool_result_budget`). +- Run microcompact: replace older `ToolResults` payloads with a placeholder while preserving the `AssistantToolCalls ⇔ ToolResults` API invariant. +- Orchestrate the ordered reduction chain and signal when prose autocompaction is needed (`ContextPipeline`). +- Dispatch the LLM summarization call on autocompaction and feed success/failure into a circuit breaker (`ContextManager` + `Summarizer`). +- Optionally back compaction with the archivist's rolling segment recap, falling back to a plain summarizer (`SegmentRecapSummarizer`). +- Track session-memory extraction thresholds (token growth, tool calls, turns) and report when a background archivist extraction should fire — without spawning it itself (`session_memory`). + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/context/mod.rs` | Module docstring + `pub mod` decls + `pub use` re-exports. Export-focused, no logic. | +| `src/openhuman/context/manager.rs` | `ContextManager` — per-session handle agents hold. Owns the default prompt builder, the pipeline, the summarizer `Arc`, and budget/markdown config. Entry point `reduce_before_call` maps `PipelineOutcome` → `ReductionOutcome` and dispatches the summarizer. | +| `src/openhuman/context/pipeline.rs` | `ContextPipeline` orchestrator. Runs the guard, microcompact, autocompact signalling, and session-memory bookkeeping. Owns `ContextGuard` + a shared `SessionMemoryHandle` (`Arc>`). Pure — issues no LLM calls. | +| `src/openhuman/context/guard.rs` | `ContextGuard` — pre-inference utilisation check (soft threshold 0.90, hard 0.95) with a 3-strike compaction circuit breaker. No-op when context window is unknown. | +| `src/openhuman/context/microcompact.rs` | Stage 3. `microcompact()` clears bodies of older `ToolResults` envelopes (keeping the N most recent), idempotent, invariant-preserving. | +| `src/openhuman/context/tool_result_budget.rs` | Stage 1. `apply_tool_result_budget()` — UTF-8-safe per-result byte cap applied inline at tool-execution time (default 16 KiB) with a truncation marker. | +| `src/openhuman/context/summarizer.rs` | `Summarizer` async trait + default `ProviderSummarizer` (wraps `Arc`). Replaces the head of history with one system summary message, keeping `keep_recent` tail messages; `snap_split_forward` (shared `pub(super)`) never splits a tool-call pair. | +| `src/openhuman/context/segment_recap_summarizer.rs` | `SegmentRecapSummarizer` — wraps an inner `Summarizer`, prefers the archivist's rolling segment recap, soft-falls-back to the inner summarizer when no recap is available. Read-only against the archivist. | +| `src/openhuman/context/session_memory.rs` | `SessionMemoryState` / `SessionMemoryConfig` — threshold-gated `should_extract` decision (token growth + tool calls + turns must all cross) and extraction bookkeeping. Holds `ARCHIVIST_EXTRACTION_PROMPT`. State-tracking only; does not spawn the archivist. | +| `src/openhuman/context/prompt.rs` | Compat shim — `pub use crate::openhuman::agent::prompts::*`. Prompt rendering moved to `agent::prompts`; this keeps `context::prompt::...` as a stable import path. | +| `src/openhuman/context/channels_prompt.rs` | Bespoke free-function `build_system_prompt(...)` for channel runtimes (Discord/Slack/Telegram/…). Byte-stable for prefix-cache hits; injects OpenClaw bootstrap files (`SOUL.md`, `IDENTITY.md`, optional `PROFILE.md`/`MEMORY.md`), tools, safety, skills, runtime, and channel-capabilities sections. | +| `src/openhuman/context/manager_tests.rs` · `summarizer_tests.rs` · `segment_recap_summarizer_tests.rs` | Sibling test suites wired via `#[cfg(test)] #[path = ...] mod tests`. Other files use inline `#[cfg(test)] mod tests`. | + +## Public surface + +From `mod.rs` re-exports: + +- **Guard**: `ContextGuard`, `ContextCheckResult`. +- **Manager**: `ContextManager`, `ContextStats`, `ReductionOutcome`. +- **Microcompact**: `microcompact`, `MicrocompactStats`, `CLEARED_PLACEHOLDER`, `DEFAULT_KEEP_RECENT_TOOL_RESULTS`. +- **Pipeline**: `ContextPipeline`, `ContextPipelineConfig`, `PipelineOutcome` (plus `SessionMemoryHandle` type alias). +- **Prompt** (re-exported from `agent::prompts`): `SystemPromptBuilder`, `PromptSection`, `PromptContext`, `PromptTool`, `ArchetypePromptSection`, `DateTimeSection`, `IdentitySection`, `LearnedContextData`, `RuntimeSection`, `SafetySection`, `ToolsSection`, `WorkspaceSection`. +- **Segment recap**: `SegmentRecapSummarizer`. +- **Session memory**: `SessionMemoryConfig`, `SessionMemoryState`, `ARCHIVIST_EXTRACTION_PROMPT`, `DEFAULT_MIN_TOKEN_GROWTH`, `DEFAULT_MIN_TOOL_CALLS`, `DEFAULT_MIN_TURNS_BETWEEN`. +- **Summarizer**: `ProviderSummarizer`, `Summarizer` (trait), `SummaryStats`. +- **Tool-result budget**: `apply_tool_result_budget`, `BudgetOutcome`, `DEFAULT_TOOL_RESULT_BUDGET_BYTES`. + +## RPC / controllers + +None. No `schemas.rs`, no `all_controller_schemas`, no `handle_*`. `ContextStats` doc comments reference an optional `context.get_stats` / `context.get_stats` RPC, but the schema/handler is not defined in this module. + +## Agent tools + +None. No `tools.rs`. (Session-memory extraction uses the `update_memory_md` / `memory_recall` / `memory_search` tools, but those are owned elsewhere; this module only references them in prompt text.) + +## Events + +None. No `bus.rs`; no `DomainEvent`s published or subscribed. + +## Persistence + +No `store.rs`. State is per-session and in-memory: + +- `ContextGuard` holds last token counts, context window, and circuit-breaker state. +- `SessionMemoryState` (behind a shared `Arc>` `SessionMemoryHandle`) tracks cumulative tokens / tool calls / turn counters and extraction-in-progress flag; resets naturally when a session ends. + +The durable long-term substrate session-memory targets is the workspace `MEMORY.md` file, but that file is written by the spawned archivist sub-agent (owned by the agent harness), not by this module. + +## Dependencies + +- `crate::openhuman::config` — reads `ContextConfig` (`config/schema/context.rs`): enabled flag, microcompact/autocompact toggles, `summarizer_model`, `tool_result_budget_bytes`, `prefer_markdown_tool_output`, and embedded `SessionMemoryConfig` thresholds. +- `crate::openhuman::inference::provider` — core types `ConversationMessage`, `ChatMessage`, `ToolCall`, `ToolResultMessage`, `UsageInfo`, and the `Provider` trait the summarizer drives. +- `crate::openhuman::agent::prompts` — `prompt.rs` re-exports the entire prompt-section/builder surface from here (prompt logic lives next to the agents that consume it). +- `crate::openhuman::agent::harness::archivist::ArchivistHook` — `SegmentRecapSummarizer` reads the rolling segment recap (read-only) from the archivist. +- `crate::openhuman::skills::Skill` — `channels_prompt.rs` renders the available-skills section. +- `crate::openhuman::util::floor_char_boundary` — UTF-8-safe truncation in `tool_result_budget`. +- `crate::openhuman::memory` / `memory_store` — used only in the `segment_recap_summarizer` tests (`ChatPrompt`, `fts5`, `segments`). + +## Used by + +- **`agent::harness`** — the primary consumer: `session/builder.rs` constructs the `ContextManager` (and chooses `ProviderSummarizer` vs `SegmentRecapSummarizer` based on `config.learning.unified_compaction_enabled`); `session/turn.rs` calls `reduce_before_call`, drives the session-memory counters, and spawns the archivist extraction when `should_extract_session_memory` says so; `session/runtime.rs`, `fork_context.rs`, `tool_loop.rs`, `subagent_runner/*`, and `tool_filter.rs` consume the prompt builder and reduction surface. +- **`agent::agents/*/prompt.rs`** — every archetype prompt module pulls prompt sections/builder through `context::prompt`. +- **`channels`** — `channels/runtime/startup.rs` (and channel prompt/identity tests) call `channels_prompt::build_system_prompt`. +- **`agent::dispatcher`, `agent::triage`, `agent::tools` (spawn_subagent / spawn_parallel / spawn_worker_thread), `learning::prompt_sections`, `memory_tools::prompt`, `tools::orchestrator_tools`, `composio::ops`** — consume the prompt-building surface. +- **`config::schema`** — `context.rs` embeds `SessionMemoryConfig`. + +## Notes / gotchas + +- **The pipeline issues no LLM calls.** `ContextPipeline::run_before_call` only *signals* `AutocompactionRequested`; the actual summarization is dispatched by `ContextManager::reduce_before_call`. This keeps the pipeline fully testable without a provider. +- **Stage 1 (tool-result budget) and the snip/trim are not pipeline stages.** Tool-result budget is applied inline in `Agent::execute_tool_call` before output enters history; the hard message-count trim is `Agent::trim_history`. Only microcompact (stage 3) and autocompact (stage 4) run inside `run_before_call`. +- **Cache contract**: microcompact and autocompact deliberately mutate previously-sent history (breaking the KV-cache prefix) and run *only* when the guard says the window would otherwise bust; each firing establishes the new smaller prefix as the next cache target. +- **Circuit breaker**: three consecutive summarizer failures trip the breaker (`compaction_disabled`); above the 0.95 hard limit while tripped, the guard returns `ContextExhausted` and the turn should abort. Any successful reduction (microcompact freeing envelopes, or a successful summary) resets the breaker. +- **No partial mutation on failure**: both `ProviderSummarizer` and `SegmentRecapSummarizer` either fully rewrite the head or leave history untouched — so the breaker can safely treat failure as "nothing happened." +- **Session memory is separate from compaction**: it does not mutate in-flight history; it gates a *persistent* `MEMORY.md` extraction. All three thresholds (token growth, tool calls, turns) must be crossed and no extraction may be in flight. `mark_extraction_failed` keeps deltas so the next turn retries; `mark_extraction_complete` resets them. The handle is `Arc`-cloned so a detached background task can flip completion state after the synchronous borrow is released. +- **`prompt.rs` is a compat shim** — do not add prompt logic here; it lives in `agent::prompts`. +- **`channels_prompt::build_system_prompt` deliberately bypasses `SystemPromptBuilder`** to keep production channel prompt bytes stable for prefix-cache hits; it is a standalone free function despite living under `context/`. diff --git a/src/openhuman/cost/README.md b/src/openhuman/cost/README.md new file mode 100644 index 000000000..dcf2a5d63 --- /dev/null +++ b/src/openhuman/cost/README.md @@ -0,0 +1,85 @@ +# cost + +API-usage cost tracking and budget enforcement for the agent. Records per-call token usage and computed USD cost to an append-only JSONL file, maintains in-memory daily/monthly aggregates, exposes a budget gate (`check_budget`) and a 7-day dashboard surface over JSON-RPC. A process-global singleton tracker is shared by the agent turn loop (telemetry) and the dashboard RPC handlers so each provider call is persisted exactly once. + +## Responsibilities + +- Compute per-call cost in USD from token counts and per-million prices (`TokenUsage::new`), clamping non-finite/negative prices to `0.0`. +- Persist each usage event as a `CostRecord` line in `costs.jsonl` (durable: write + `sync_all`). +- Maintain cached current-day / current-month spend aggregates, rebuilt on day/month rollover. +- Enforce daily and monthly budget limits with warn-threshold signalling (`check_budget` → `BudgetCheck::{Allowed, Warning, Exceeded}`) — only when `cost.enabled`. +- Capture dashboard telemetry **unconditionally** (independent of `cost.enabled`) via `record_usage_unconditional`, so history exists before a user opts into enforcement. +- Aggregate a 7-day daily history (zero-filling gap days), monthly pace projection, budget utilisation/status, and per-model breakdown for the dashboard. +- Serve the dashboard / daily-history / summary over JSON-RPC, with a cached read-only fallback tracker when the global is uninitialised. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/cost/mod.rs` | Export-focused module root; re-exports tracker, types, global helpers, and the `all_cost_*` controller schema/registry pair. | +| `src/openhuman/cost/types.rs` | Serde domain types: `TokenUsage`, `CostRecord`, `UsagePeriod`, `BudgetCheck`, `CostSummary`, `ModelStats`, `DailyCostEntry`, `BudgetStatus`, `CostDashboard`. Cost-calc logic lives in `TokenUsage::new`. | +| `src/openhuman/cost/tracker.rs` | `CostTracker` (budget checks, recording, summaries, daily history, dashboard build) plus the private `CostStorage` JSONL persistence + aggregate-cache layer. Functions as both `ops` and `store`. | +| `src/openhuman/cost/global.rs` | Process-global `OnceCell>` singleton: `init_global`, `try_global`, `record_provider_usage`, and `build_token_usage` (provider `UsageInfo` → `TokenUsage`). | +| `src/openhuman/cost/rpc.rs` | RPC-facing handlers (`dashboard`, `daily_history`, `summary`) returning `RpcOutcome`; DTO types; `resolve_tracker` with a cached fallback tracker + error-replay TTL. | +| `src/openhuman/cost/schemas.rs` | Controller schemas + `handle_*` JSON-RPC dispatchers; `all_controller_schemas` / `all_registered_controllers`. | +| `src/openhuman/cost/tracker_tests.rs` | Sibling test suite for `tracker.rs` (`#[path]`-included). | + +## Public surface + +From `mod.rs` re-exports: + +- `CostTracker` — the tracker (`tracker`). +- `init_global`, `try_global`, `record_provider_usage` (`global`). +- `all_cost_controller_schemas`, `all_cost_registered_controllers` (`schemas`). +- Types: `BudgetCheck`, `BudgetStatus`, `CostDashboard`, `CostRecord`, `CostSummary`, `DailyCostEntry`, `ModelStats`, `TokenUsage`, `UsagePeriod`. + +Notable `CostTracker` methods: `new`, `session_id`, `check_budget`, `record_usage`, `record_usage_unconditional`, `get_summary`, `get_daily_cost`, `get_monthly_cost`, `get_daily_history`, `get_dashboard`. + +## RPC / controllers + +Namespace `cost` (methods `openhuman.cost_*` via the registry): + +| Method | Inputs | Output | +| --- | --- | --- | +| `cost_get_dashboard` | none | 7-day dashboard payload: per-day buckets, summary metrics, budget utilisation/status, per-model breakdown. | +| `cost_get_daily_history` | `days?` (u32, default 7, clamped `[1, 366]`) | Ordered daily entries, oldest first, gaps zero-filled. | +| `cost_get_summary` | none | Live session / daily / monthly cost summary. | + +Handlers load config via `config_rpc::load_config_with_timeout`, then delegate to `rpc.rs`. RPC DTOs (`CostDashboardDto`, `DailyCostEntryDto`, `ModelStatsDto`, `CostSummaryDto`) add presentation fields not on the domain types — `provider` (derived from the `provider/model` prefix), `percent_of_total`, and dashboard threshold/`enabled` flags from `cost.dashboard`. + +## Events + +None. The module has no `bus.rs` and no `DomainEvent` publishers/subscribers. + +## Persistence + +- Append-only JSONL at `/state/costs.jsonl`, one `CostRecord` per line. +- Legacy migration: a pre-existing `/.openhuman/costs.db` is moved (rename, copy-fallback) to the new path on first `CostTracker::new`. +- In-memory caches in `CostStorage`: `daily_cost_usd` / `monthly_cost_usd` plus the cached day/year/month they pertain to; rebuilt by full file scan on construction and on period rollover. Malformed lines are skipped with a `warn`. +- Per-session in-memory `Vec` (`session_costs`) backs the session figures in `get_summary`. + +## Dependencies + +- `crate::openhuman::config` — `CostConfig` / `Config` (limits, warn percent, `dashboard` thresholds/currency/enabled, `workspace_dir`); `config::rpc::load_config_with_timeout` in schemas. +- `crate::openhuman::inference::provider::traits::UsageInfo` — provider usage payload translated into `TokenUsage` in `global.rs`. +- `crate::core::all` — `ControllerFuture`, `RegisteredController` for controller registration. +- `crate::core` — `ControllerSchema`, `FieldSchema`, `TypeSchema`. +- `crate::rpc::RpcOutcome` — RPC return wrapper. +- External: `chrono`, `serde`/`serde_json`, `uuid`, `parking_lot`, `once_cell`, `anyhow`, `tempfile` (tests). + +## Used by + +- `src/core/all.rs` — registers `all_cost_registered_controllers` / `all_cost_controller_schemas`. +- `src/core/jsonrpc.rs` — calls `cost::init_global(cfg.cost.clone(), &workspace_dir)` at bootstrap. +- `src/openhuman/agent/harness/session/turn.rs` — calls `cost::record_provider_usage` after provider calls to log per-turn usage. +- `src/openhuman/config/schema/identity_cost.rs` — `CostConfig` definition references `check_budget` / `record_provider_usage` semantics in docs. + +## Notes / gotchas + +- **`cost.enabled` gates enforcement only, not telemetry.** When `false`, `check_budget` returns `Allowed` and `record_usage` is a no-op, but the agent path uses `record_usage_unconditional`, so `costs.jsonl` still grows. This is a deliberate behavioural change (logged with a `warn` on init for upgraders) so spend history exists before turning on hard caps. +- The global tracker is a one-shot `OnceCell`; `init_global` is idempotent and never panics on construction failure (it logs and leaves `try_global() == None`). Callers before bootstrap (e.g. unit tests) must treat the absence as a soft no-op. +- `record_provider_usage` skips all-zero `UsageInfo` payloads (`input==0 && output==0 && charged==0.0`) so providers that don't echo usage don't inflate the request count. +- The RPC fallback tracker (`resolve_tracker`) shares the same JSONL file as the real tracker and is read-effective only; it caches by workspace path and replays a construction error for `FALLBACK_ERROR_TTL` (30s) to avoid hammering a bad workspace on the UI's ~10s poll. +- `budget_utilization` is clamped to `1.0` for display; `budget_status` is computed from the raw (unclamped) utilisation against `warn`/`alert` thresholds. A non-positive monthly limit forces `BudgetStatus::Normal` and `0.0` utilisation. +- All amounts are stored/computed in USD; `currency` is a presentation hint only. +- Time bucketing is UTC throughout (`naive_utc().date()`); model is the bucket key for per-model stats, and `provider` is derived from the `provider/model` slash prefix in DTO mapping. diff --git a/src/openhuman/credentials/README.md b/src/openhuman/credentials/README.md new file mode 100644 index 000000000..39eeb67e0 --- /dev/null +++ b/src/openhuman/credentials/README.md @@ -0,0 +1,105 @@ +# credentials + +Credential management for the OpenHuman app session and provider/OAuth auth profiles. Owns the on-disk **auth-profiles** store (encrypted JSON + OS keychain), the `app-session` JWT lifecycle (login / logout / session-state), per-provider token storage (e.g. API keys, OAuth token sets), the backend OAuth connect/handoff flows, and the Composio direct-mode (BYO key) credential slot. Exposes everything under the `auth.*` JSON-RPC / CLI namespace and runs the canonical session-teardown when a `SessionExpired` event fires. + +## Responsibilities + +- Store and validate the app session JWT (`app-session` provider, `default` profile), including local offline sessions and backend `GET /auth/me` validation. +- On login: activate the user-scoped openhuman directory, purge pre-login (anonymous) conversation threads on first activation, bind memory/conversation persistence, bootstrap subconscious, and start login-gated services (local AI, voice, dictation, screen intelligence, autocomplete). +- On logout / session-expiry: remove the JWT, clear the active-user marker, stop login-gated services, reset subconscious, and flip the scheduler-gate signed-out override. +- Persist arbitrary provider credentials (token + metadata fields) as named auth profiles; list/remove/set-active; prefix-list profiles for grouped namespaces (e.g. `channel:*`). +- Run backend OAuth flows: connect URL, list integrations, fetch integration handoff tokens, fetch one-time client key, revoke integration. +- Store/read/clear the Composio direct-mode API key (`composio-direct` provider). +- Encrypt/decrypt arbitrary secrets via the `SecretStore`. +- Manage the on-disk profile store with secret-at-rest handling: OS keychain when available, ChaCha20-Poly1305 encrypted JSON fallback otherwise, with legacy cipher migration, keychain promotion, corrupt-store quarantine, and crash-safe file locking. + +## Key files + +| File | Role | +| --- | --- | +| `mod.rs` | Export-focused. Re-exports `core::*`, `ops` (also as `rpc`), Composio-direct helpers, schema controllers (`all_credentials_controller_schemas` / `all_credentials_registered_controllers`), and backend OAuth REST types from `crate::api::rest`. | +| `core.rs` | `AuthService` facade over `AuthProfilesStore` — store/get/remove/set-active profiles, resolve bearer token, profile-id selection logic (override → active → default → any-for-provider), provider normalization, state-dir derivation. | +| `profiles.rs` | The persistence engine. `AuthProfile` / `TokenSet` / `AuthProfileKind` / `AuthProfilesData` types and `AuthProfilesStore` — atomic JSON read/write, keychain vs encrypted-JSON secret handling, legacy migration, corrupt-store quarantine, PID-aware stale-lock recovery. | +| `ops.rs` | Business logic + RPC entry points (returns `RpcOutcome`). Session lifecycle (`store_session`/`clear_session`/`auth_get_*`), login/logout service orchestration, provider-credential CRUD, OAuth flows, Composio-direct key helpers, secret encrypt/decrypt. Re-exported as `rpc`. | +| `schemas.rs` | `auth.*` controller schemas + `handle_*` dispatchers delegating to `ops`. Defines `all_controller_schemas` / `all_registered_controllers`. | +| `session_support.rs` | Session/auth helpers: `build_session_state`, `get_session_token`, `load_app_session_profile`, `summarize_auth_profile`, local-session detection/slug, field parsing. Shared by RPC and the HTTP host. | +| `responses.rs` | Response DTOs: `AuthStateResponse`, `AuthProfileSummary`. | +| `cli.rs` | CLI auth entrypoints (`cli_auth_login/logout/status/list`) that branch on `app-session` vs provider; `--field key=value` parsing. | +| `bus.rs` | `SessionExpiredSubscriber` — `EventHandler` for `DomainEvent::SessionExpired`. | +| `ops_tests.rs`, `profiles_tests.rs`, `schemas_tests.rs` | Sibling test suites (`#[path = ...]`). Plus inline `#[cfg(test)]` tests in `core.rs`, `session_support.rs`, `bus.rs`, `cli.rs`. | + +## Public surface + +- **`AuthService`** (`core.rs`) — `from_config`, `new`, `load_profiles`, `store_provider_token`, `set_active_profile`, `remove_profile`, `get_profile`, `get_provider_bearer_token`. +- **Constants** — `APP_SESSION_PROVIDER` (`"app-session"`), `DEFAULT_AUTH_PROFILE_NAME` (`"default"`), `COMPOSIO_DIRECT_PROVIDER` (`"composio-direct"`). +- **Helpers** — `normalize_provider`, `default_profile_id`, `select_profile_id`, `state_dir_from_config`, `profile_id`. +- **Types** (`profiles.rs`) — `AuthProfile`, `AuthProfileKind` (`OAuth`/`Token`), `TokenSet`, `AuthProfilesData`, `AuthProfilesStore`. +- **Ops/RPC** (`ops`, re-exported as `rpc`) — `store_session`, `clear_session`, `auth_get_state`, `auth_get_session_token_json`, `auth_get_me`, `consume_login_token`, `auth_create_channel_link_token`, `store_provider_credentials`, `remove_provider_credentials`, `list_provider_credentials`, `list_provider_credentials_by_prefix`, `oauth_connect`, `oauth_list_integrations`, `oauth_fetch_integration_tokens`, `oauth_fetch_client_key`, `oauth_revoke_integration`, `encrypt_secret`, `decrypt_secret`, `start_login_gated_services`, `stop_login_gated_services`. +- **Composio-direct** — `store_composio_api_key`, `get_composio_api_key`, `clear_composio_api_key`, `rpc_store_composio_api_key`. +- **Backend OAuth re-exports** (from `crate::api::rest`) — `BackendOAuthClient`, `ConnectResponse`, `IntegrationSummary`, `IntegrationTokensHandoff`, `decrypt_handoff_blob`, `user_id_from_auth_me_payload`, `user_id_from_profile_payload`. +- **Schema controllers** — `all_credentials_controller_schemas`, `all_credentials_registered_controllers`. + +## RPC / controllers + +Namespace `auth` (JSON-RPC `openhuman.auth_*` / CLI). Defined in `schemas.rs`: + +| Method | Description | +| --- | --- | +| `auth_store_session` | Store + validate app session JWT. | +| `auth_clear_session` | Remove stored app session credentials. | +| `auth_get_state` | Current auth/session state (`AuthStateResponse`). | +| `auth_get_session_token` | Read stored app session token. | +| `auth_get_me` | Fetch current authenticated backend user profile. | +| `auth_consume_login_token` | Consume one-time login handoff token → session JWT. | +| `auth_create_channel_link_token` | Short-lived channel link token (telegram/discord). | +| `auth_store_provider_credentials` | Store provider credentials for a profile. | +| `auth_remove_provider_credentials` | Remove provider credentials. | +| `auth_list_provider_credentials` | List stored provider credentials (optional provider filter). | +| `auth_oauth_connect` | Create OAuth connect URL for a provider. | +| `auth_oauth_list_integrations` | List OAuth integrations for the session. | +| `auth_oauth_fetch_integration_tokens` | Fetch integration handoff tokens. | +| `auth_oauth_fetch_client_key` | Fetch one-time client key share for an encrypted integration. | +| `auth_oauth_revoke_integration` | Revoke an OAuth integration. | + +Note: `list_provider_credentials_by_prefix` and the Composio-direct/secret helpers are public ops but not registered as `auth.*` controllers here — they are called directly by other domains. + +## Agent tools + +None. This module owns no agent tools (`tools.rs` does not exist). + +## Events + +`bus.rs` — `SessionExpiredSubscriber` (`name() == "credentials::session_expired_handler"`, domain filter `["auth"]`) **subscribes** to `DomainEvent::SessionExpired`. On a non-local session it flips the scheduler gate to signed-out and calls `clear_session`; for a local offline session it re-enables the gate and no-ops. This module does not publish events directly (publishers of `SessionExpired` are 401-detection sites elsewhere). + +## Persistence + +`AuthProfilesStore` (`profiles.rs`) writes `auth-profiles.json` in the config state directory (parent of `config.config_path`, user-scoped after login). Layout: `schema_version` (current = 1), `updated_at`, `active_profiles` (provider → profile-id), `profiles` (id → profile). Secret handling: + +- **OS keychain** when available (`crate::openhuman::keyring::is_available`): all token fields stored under key `auth:{profile_id}` namespaced by a per-user id derived from the state dir; JSON keeps no secret fields. +- **Encrypted-JSON fallback** (headless/CI): token fields encrypted via `SecretStore` (ChaCha20-Poly1305). +- Loads migrate legacy `enc:`/`enc2:` cipher fields and promote secrets into the keychain; unrecoverable (un-decryptable / bad-`kind`) profiles are dropped rather than poisoning the whole store; unparseable files are quarantined to `auth-profiles.corrupt-.json` and reset to empty. +- Mutations are guarded by `auth-profiles.lock` (PID-stamped). Stale/leaked/malformed locks are reclaimed by liveness + age checks to avoid the "stuck on Initializing OpenHuman" hang. + +## Dependencies + +- `crate::openhuman::config` — `Config`, config load (`load_config_with_timeout`), user-dir activation (`default_root_openhuman_dir`, `user_openhuman_dir`, `read/write/clear_active_user`, `pre_login_user_dir`), onboarding state. +- `crate::openhuman::keyring` — `SecretStore` (encrypt/decrypt) and OS keychain `get`/`set`/`delete`/`is_available`. +- `crate::openhuman::scheduler_gate` — signed-out override flipped on login/logout/session-expiry. +- `crate::openhuman::memory_conversations` — purge pre-login threads, bind conversation persistence after login. +- `crate::openhuman::memory` — bind memory client to the active workspace after login. +- `crate::openhuman::subconscious` — post-login bootstrap / user-switch reset. +- `crate::openhuman::inference`, `::voice`, `::screen_intelligence`, `::autocomplete` — login-gated services started/stopped. +- `crate::api::config`, `::jwt`, `::rest` — backend API URL, session-token read, `BackendOAuthClient` + OAuth/handoff types. +- `crate::core::all` (`ControllerFuture`, `RegisteredController`), `crate::core` (`ControllerSchema`/`FieldSchema`/`TypeSchema`), `crate::core::event_bus` (`DomainEvent`/`EventHandler`), `crate::rpc::RpcOutcome` — controller registry + RPC envelope + event bus. + +## Used by + +Many domains consume `AuthService` / session helpers / Composio-direct key, including: `src/core/{all,auth,jsonrpc}.rs` (controller wiring + auth gate), `src/api/jwt.rs`, `app_state/ops.rs` (session snapshot), `channels/*` (managed credentials), `composio/{client,ops}.rs` (BYO key), `config/schema/*`, `embeddings/cloud.rs`, `encryption/ops.rs`, `http_host/auth.rs`, `inference/*` (provider auth, OpenAI OAuth), `migrations/unify_ai_provider_settings.rs`, `referral/ops.rs`, `subconscious/engine.rs`, and `webhooks`. + +## Notes / gotchas + +- `mod.rs` re-exports `ops` both as `ops::*` and as `pub use ops as rpc` — call sites use `credentials::rpc::*`; this is the documented `rpc.rs`-equivalent exception (no separate `rpc.rs` file exists). +- Doc comments in `responses.rs` / `session_support.rs` reference `crate::core_server` / `crate::core_server::types`; those paths are historical — the actual transport crate is `src/core/`. +- `store_session` does heavy orchestration beyond just storing a token (directory activation, thread purge, service startup). Treat it as the login funnel, not a thin setter. +- Local offline sessions are detected purely by the JWT signature segment being literally `local` (`is_local_session_token`); they skip backend validation and are never treated as expired. +- Secrets are never logged — debug lines record only lengths/markers, honoring the CLAUDE.md redaction rule. diff --git a/src/openhuman/cwd_jail/README.md b/src/openhuman/cwd_jail/README.md new file mode 100644 index 000000000..6a2178813 --- /dev/null +++ b/src/openhuman/cwd_jail/README.md @@ -0,0 +1,77 @@ +# cwd_jail + +Cross-platform directory-jail facade. Given a declarative description of a workspace (`Jail`), it picks the strongest available OS sandbox backend and spawns a child process caged into a single read/write root (plus optional read-only paths), with toggles for outbound network and subprocess creation. It is the user-facing complement to `src/openhuman/security/` — the autonomy gate decides *whether* a command may run; `cwd_jail` decides *what filesystem* the approved child process sees. It jails the *child* it spawns, never the core process itself. + +## Responsibilities + +- Describe a jail declaratively via a builder (`Jail::new(root, label)` + `.add_read_only(...)`, `.deny_net()`, `.deny_subprocess()`). +- Auto-detect and cache the strongest backend for the current OS (Landlock / Seatbelt / AppContainer / noop). +- Spawn a `std::process::Command` inside the jail, canonicalizing `root` (and read-only paths) first so backends never see `..` or symlink trickery. +- Provide a persistent registry to manage many jailed workspaces side-by-side, each with a stable id, label, directory, and metadata, indexed in a JSON file. +- Fall back gracefully (`noop`) when no OS-level sandbox is available, while still letting callers rely on application-layer path checks. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/cwd_jail/mod.rs` | Module docstring + the thin facade: `spawn` / `spawn_with` / `default_backend` (cached via `OnceLock`). Re-exports the public surface. Inline tests. | +| `src/openhuman/cwd_jail/jail.rs` | Core types: the `Jail` description struct (builder + `canonicalize`/`canonicalize_or_log`) and the `JailBackend` trait (`name`/`is_available`/`spawn`). | +| `src/openhuman/cwd_jail/detect.rs` | `pick_backend()` — cfg-gated platform selection; returns the first available backend or `NoopBackend`. | +| `src/openhuman/cwd_jail/noop.rs` | `NoopBackend` — no enforcement, plain `Command::spawn`. Always available. | +| `src/openhuman/cwd_jail/linux.rs` | `LandlockBackend` — kernel 5.13+ Landlock LSM applied in `pre_exec` (child-side, after fork, before exec). Gated on the `sandbox-landlock` cargo feature. | +| `src/openhuman/cwd_jail/macos.rs` | `SeatbeltBackend` — wraps the command in `/usr/bin/sandbox-exec -p ''`. Renders an allow-default-reads / deny-default-writes Seatbelt profile. | +| `src/openhuman/cwd_jail/windows.rs` | `AppContainerBackend` — `CreateAppContainerProfile` + DACL grant + `STARTUPINFOEX`/`CreateProcessW` via `windows-sys`. | +| `src/openhuman/cwd_jail/registry.rs` | `JailRegistry` + `JailRecord` — multi-jail manager persisted to `index.json`, with atomic-rename writes and containment checks. | +| `src/openhuman/cwd_jail/registry_tests.rs` | Sibling test suite for the registry (`#[path]`-included from `registry.rs`). | + +## Public surface + +Re-exported from `mod.rs`: + +- `Jail`, `JailBackend` — the declarative jail description and the OS-enforcement trait. +- `NoopBackend` — the unenforced fallback backend. +- `JailRecord`, `JailRegistry` — persisted multi-jail manager. + +Free functions in `mod.rs`: + +- `default_backend() -> Arc` — process-wide cached, lazily auto-detected backend. +- `spawn(jail: &Jail, cmd: Command) -> io::Result` — canonicalize + spawn under the default backend. +- `spawn_with(backend: &dyn JailBackend, jail: &Jail, cmd: Command) -> io::Result` — same, with an explicit backend (tests / forced weaker backend). + +`JailRegistry` methods: `open`, `base`, `create`, `get`, `list`, `find_by_label`, `rename`, `set_notes`, `delete`, `clear`, `spawn_in`, `spawn_in_with`. + +## Persistence + +`JailRegistry` is rooted at a base directory (e.g. `~/.openhuman/jails/` or `/jails/`). Each jail is a `//` directory; metadata for all jails lives in `/index.json`. + +- `JailRecord` fields: `id`, `label`, `dir`, `backend_at_create`, `created_at_unix`, `updated_at_unix`, optional `notes`. +- The on-disk index is the source of truth; in-memory state (`Index`, a `BTreeMap` for deterministic ordering) is rebuilt on every `open()`. +- Writes are atomic (write-temp + rename, with a direct-overwrite fallback if rename-over fails). Mutating ops roll back the in-memory change if persistence fails. +- Ids are generated as `j` (not cryptographically random — used as directory names), with a collision-retry loop on `create`. +- Concurrency is guarded by a `std::sync::Mutex`; multi-*process* access is an explicit non-goal (no OS file locking). + +## Dependencies + +This module is notably self-contained: its own files contain **no** `use crate::openhuman::` or `use crate::core::` imports. Dependencies are external/std only: + +- `std::process` (`Command`/`Child`), `std::fs`, `std::sync` (`Mutex`/`OnceLock`/`Arc`), `std::time`. +- `serde` / `serde_json` — `JailRecord` and the index serialization (registry). +- `landlock` crate — Linux backend, gated on the `sandbox-landlock` cargo feature. +- `windows-sys` — Windows AppContainer FFI (Security/Isolation, Threading, Memory APIs). +- macOS backend shells out to the system binary `/usr/bin/sandbox-exec`. + +The Linux backend's docstring references `crate::openhuman::security::landlock` as conceptual prior art, but the implementation here is self-contained (it does not import that module). + +## Used by + +- Declared at `src/openhuman/mod.rs:37` (`pub mod cwd_jail;`). No other `src/` Rust files currently reference `openhuman::cwd_jail` — it is a self-standing facade not yet wired into a calling domain. + +## Notes / gotchas + +- **Not RPC-facing, no agent tools, no event bus.** There is no `schemas.rs`, `tools.rs`, or `bus.rs`; the module exposes no `openhuman.*` RPC methods, owns no agent tools, and publishes/subscribes to no `DomainEvent`s. +- **Backends differ in what `allow_net` / `read_only` mean.** Landlock does not gate network at all; macOS Seatbelt grants `allow default` (full network) and treats `read_only` as informational since reads are already allowed; Windows AppContainer is the only backend that honors `read_only` as a real read grant and maps `allow_net` to the strictly outbound-only `internetClient` capability (LAN/inbound capabilities deliberately excluded). +- **Windows `spawn` currently returns an error after a successful launch.** `spawn_in_container` creates the process successfully but cannot bridge the raw `HANDLE` into a `std::process::Child` (the needed `FromRawHandle for Child` is unstable), so it closes the handle and returns `io::ErrorKind::Unsupported`. See the TODO in `windows.rs`. The Windows path is compile-checked but flagged as needing real-hardware testing. +- **macOS stdio is inherited** — the Seatbelt wrapper cannot re-apply the original command's `Stdio` config; it uses `sandbox-exec` defaults (inherit). The profile re-allows writes only under canonicalized `root` + `/private/tmp`, so callers must canonicalize first (the `spawn` facade does this automatically) or writes inside `root` may be denied (e.g. `/tmp` → `/private/tmp`). +- **Linux Landlock runs in `pre_exec`** (child-side after fork) so the parent keeps its privileges; read-only paths also get `Execute` so the child can run binaries found there (e.g. `/usr/bin/sh`). +- **Registry containment guard:** both `delete` and `jail_for` (used by `spawn_in`/`spawn_in_with`) refuse to operate on a record whose canonicalized `dir` is not under the canonicalized `base`, defending against a corrupted index pointing at `/`. +- **Free-form input is length-logged, not value-logged** (labels/notes) to avoid leaking arbitrary text into logs. diff --git a/src/openhuman/dashboard/README.md b/src/openhuman/dashboard/README.md new file mode 100644 index 000000000..4557b2493 --- /dev/null +++ b/src/openhuman/dashboard/README.md @@ -0,0 +1,58 @@ +# dashboard + +Aggregate, operator-facing views over local config. Today it owns a single read-only view: the per-model health comparison table rendered in the desktop **Settings → Developer Options → Model Health** panel. The view joins the local `Config::model_registry` with the `dashboard.model_health` thresholds and emits one row per model. Telemetry-driven metric fields (`quality_score`, `hallucination_rate`, `agents_using`, `tasks_evaluated`) are intentional placeholders (`null` / `0`) until a local telemetry pipeline lands — the placeholder contract is documented in `ops.rs` and asserted in its tests. Stateless: no persistence, no event-bus subscribers, no agent tools. + +## Responsibilities + +- Build the model health comparison response by mapping each `model_registry` entry to a `ModelHealthEntry` (id, provider, cost per 1M output tokens, vision flag). +- Surface the `dashboard.model_health` thresholds (`hallucination_threshold`, `min_tasks_for_rating`, `evaluation_window_tasks`) so the frontend can compute status badges. +- Emit telemetry metric fields as explicit placeholders (`None` / `0`) — to be populated here, not at the transport layer, once telemetry exists. +- Reject the request with an error when `dashboard.model_health.enabled` is `false`. +- Expose the view over JSON-RPC via the controller registry. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/dashboard/mod.rs` | Export-only: module docstring + `mod`/`pub use` re-exports of ops, schemas, and types. | +| `src/openhuman/dashboard/types.rs` | Wire types: `ModelHealthEntry`, `ModelHealthConfigView`, `ModelHealthResponse` (serde). | +| `src/openhuman/dashboard/ops.rs` | Business logic: `model_health(&Config)` joins registry + thresholds, returns `RpcOutcome`. Inline tests cover mapping, thresholds, disabled feature, empty registry. | +| `src/openhuman/dashboard/schemas.rs` | Controller schemas + registered controller + `handle_dashboard_model_health` handler (loads config via timeout, delegates to `ops::model_health`). Inline tests assert schema stability and list-length parity. | + +## Public surface + +- `ops::model_health` — `fn model_health(config: &Config) -> Result, String>`. +- `schemas::all_dashboard_controller_schemas`, `schemas::all_dashboard_registered_controllers`, `schemas::dashboard_schemas` — controller-registry entry points. +- `types::ModelHealthEntry`, `types::ModelHealthConfigView`, `types::ModelHealthResponse`. + +## RPC / controllers + +| Method | Inputs | Outputs | +| --- | --- | --- | +| `openhuman.dashboard_model_health` (namespace `dashboard`, function `model_health`) | none | `models: ModelHealthEntry[]`, `config: ModelHealthConfigView` | + +`ModelHealthEntry`: `id`, `provider`, `cost_per_1m_output` (f64), `vision` (bool), `quality_score` (`f64?`, placeholder), `hallucination_rate` (`f64?`, placeholder), `agents_using` (u64, placeholder 0), `tasks_evaluated` (u64, placeholder 0). +`ModelHealthConfigView`: `hallucination_threshold` (f64), `min_tasks_for_rating` (u64), `evaluation_window_tasks` (u64). + +The handler loads config via `crate::openhuman::config::rpc::load_config_with_timeout()` and returns CLI-compatible JSON through `RpcOutcome::into_cli_compatible_json()`. Wired into the registry in `src/core/all.rs` (both `all_dashboard_registered_controllers` and `all_dashboard_controller_schemas`). + +## Persistence + +None. The module reads from in-memory `Config`; it stores no state. + +## Dependencies + +- `crate::openhuman::config` (`Config`, `config::rpc::load_config_with_timeout`) — source of the model registry and `dashboard.model_health` thresholds. `DashboardConfig` / `ModelHealthConfig` are defined in `src/openhuman/config/schema/dashboard.rs`. +- `crate::core::all` (`ControllerFuture`, `RegisteredController`) and `crate::core` (`ControllerSchema`, `FieldSchema`, `TypeSchema`) — controller registry types. +- `crate::rpc::RpcOutcome` — standard RPC result wrapper. +- `serde` / `serde_json` — wire (de)serialization and handler params. + +## Used by + +- `src/core/all.rs` — registers the controller and its schema into the global RPC/CLI registry (lines ~119 and ~297). + +## Notes / gotchas + +- **Placeholder contract is load-bearing.** `quality_score` / `hallucination_rate` are always `None` and `agents_using` / `tasks_evaluated` always `0`. The frontend treats null quality/hallucination as "no signal", collapsing badges to `staging`. When telemetry lands, populate these in `ops::model_health`, not in the transport layer. +- Disabled feature (`dashboard.model_health.enabled == false`) returns `Err("model health disabled")`, which the handler surfaces as an RPC error. +- This is the only view in the domain so far; `mod.rs` is intentionally export-only per the canonical module shape (no `store.rs`, `tools.rs`, or `bus.rs` — the domain is stateless, owns no agent tools, and has no event subscribers). diff --git a/src/openhuman/desktop_companion/README.md b/src/openhuman/desktop_companion/README.md new file mode 100644 index 000000000..688080891 --- /dev/null +++ b/src/openhuman/desktop_companion/README.md @@ -0,0 +1,106 @@ +# desktop_companion + +Clicky-style desktop companion interaction loop. Ties hotkey activation, microphone capture, screen context, LLM reasoning, speech synthesis, and visual pointing into a single product experience. The module is orchestration-only: it composes existing building blocks (`voice` STT/TTS, `meet_agent` LLM/wav helpers, `accessibility` foreground context, `provider_surfaces` queue, the backend chat-completions API) into a per-turn pipeline driven by a single process-global session state machine. `mod.rs` is export-focused; operational code lives in `session.rs`, `pipeline.rs`, `pointing.rs`, and `handoff.rs`. + +## Responsibilities + +- Own a single process-global companion **session** (only one active at a time) with TTL enforcement, consent gating, and conversation history. +- Drive the **state machine**: `Idle → Listening → Thinking → Speaking/Pointing → Idle`, with validated transitions and `Any → Error` / `Error → Idle`. +- Run a single **interaction turn** (text or audio): STT → screen context → LLM → POINT-tag parse → TTS → pointing, cancellable mid-turn via `CancellationToken`. +- Parse `[POINT:x,y:label:screenN]` tags from LLM output and map screen-relative coordinates to absolute multi-monitor desktop coordinates. +- Detect **provider-surface handoff** opportunities (LLM mentions Slack/Discord/email/etc.) and match against the `provider_surfaces` respond queue. +- Broadcast companion state changes over an internal `tokio::broadcast` bus and publish session lifecycle `DomainEvent`s on the global event bus. +- Expose session lifecycle + config over JSON-RPC under the `companion` namespace. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/desktop_companion/mod.rs` | Export-only: module decls + re-export of the controller schema/registry pair. | +| `src/openhuman/desktop_companion/types.rs` | Serde types: `CompanionState`, `ConversationTurn`, `CompanionConfig`, start/stop params + results, `CompanionSessionStatus`, `CompanionStateChangedEvent`. | +| `src/openhuman/desktop_companion/session.rs` | Session lifecycle + state machine over a `Mutex>` process-global singleton. TTL auto-expiry, transition validation, conversation history (capped at 50 turns). | +| `src/openhuman/desktop_companion/pipeline.rs` | Single-turn orchestration (`run_text_turn`, `run_audio_turn`) and real adapters for STT, LLM chat-completions, and TTS. Defines `TurnResult`. | +| `src/openhuman/desktop_companion/pointing.rs` | POINT-tag regex parser; maps screen-relative coords to absolute desktop coords. Defines `PointTarget`, `ScreenGeometry`, `PointingParseResult`. | +| `src/openhuman/desktop_companion/handoff.rs` | Provider keyword matching against the `provider_surfaces` queue; emits `HandoffEvent`. Token-aware matching to avoid substring false positives. | +| `src/openhuman/desktop_companion/bus.rs` | Process-global `tokio::broadcast` channel for `CompanionStateChangedEvent` (`subscribe_state_changed` / `publish_state_changed`). | +| `src/openhuman/desktop_companion/schemas.rs` | JSON-RPC controller registry: 5 controllers under the `companion` namespace + their handlers. | +| `src/openhuman/desktop_companion/{session,pipeline,pointing}_tests.rs` | Sibling test suites via `#[path]`. `handoff.rs` and `bus.rs` keep inline tests. | + +## Public surface + +Re-exported from `mod.rs`: + +- `all_desktop_companion_controller_schemas()` / `all_desktop_companion_registered_controllers()` — RPC registry pair. + +Used by `pipeline.rs` and the Tauri/RPC layer (public within the crate): + +- `session::{start_session, stop_session, session_status, transition_state, push_conversation_turn, conversation_history}` +- `pipeline::{run_text_turn, run_audio_turn, TurnResult}` +- `pointing::{parse_and_map, PointTarget, ScreenGeometry, PointingParseResult}` +- `handoff::{check_handoff, HandoffEvent}` +- `bus::{subscribe_state_changed, publish_state_changed}` +- Types: `CompanionState`, `CompanionConfig`, `CompanionSessionStatus`, `StartCompanionSessionParams`, `StopCompanionSessionParams`, etc. + +## RPC / controllers + +Namespace `companion` (5 controllers), wired into `src/core/all.rs`: + +| Method | Inputs | Notes | +| --- | --- | --- | +| `companion.start_session` | `consent: bool` (required), `ttl_secs: u64?` | Fails if `consent=false` or a non-expired session is already active. | +| `companion.stop_session` | `reason: string?` | Returns `stopped=false` if no session active. | +| `companion.status` | (none) | Auto-expires the session inline if TTL exceeded. | +| `companion.config_get` | (none) | Returns `CompanionConfig::default()` — no persistence yet. | +| `companion.config_set` | `hotkey?`, `activation_mode?`, `ttl_secs?`, `capture_screen?`, `include_app_context?` | **Not implemented** — returns an error; changes are not saved. | + +There is no `companion.activate` RPC; running a turn (`run_text_turn` / `run_audio_turn`) is invoked from the Tauri shell / hotkey bridge, which must perform the `Idle → Listening` transition first (documented precondition). + +## Agent tools + +None. This domain owns no `tools.rs` / agent tools. + +## Events + +Global event bus (`src/core/event_bus`), domain `"companion"`: + +- Publishes `DomainEvent::CompanionSessionStarted { session_id, ttl_secs }` on `start_session`. +- Publishes `DomainEvent::CompanionSessionEnded { session_id, reason, turn_count }` on `stop_session` and on TTL auto-expiry (`reason: "ttl_expired"`). +- `DomainEvent::CompanionStateChanged` is defined in `events.rs` but state changes are currently broadcast on the module-local `bus.rs` channel (`CompanionStateChangedEvent`), not the global bus. + +The module-local `bus.rs` broadcast (`subscribe_state_changed`) is intended to be bridged to the overlay via Socket.IO as `companion:state_changed`. No `EventHandler` subscribers are registered by this module. + +## Persistence + +None durable. The active session is held only in a process-global `Mutex>` in `session.rs` (in-memory, lost on restart). `CompanionConfig` has no store — `config_get` returns defaults and `config_set` is a stub. No `store.rs`. + +## Dependencies + +- `crate::core::all` / `crate::core::{ControllerSchema, FieldSchema, TypeSchema}` — RPC controller registration and schema types. +- `crate::core::event_bus` — publish session lifecycle `DomainEvent`s. +- `crate::openhuman::memory::EmptyRequest` — empty-params deserialization for no-input RPCs. +- `crate::openhuman::voice::cloud_transcribe` — STT (`transcribe_cloud`). +- `crate::openhuman::voice::reply_speech` — TTS (`synthesize_reply`, ElevenLabs `eleven_turbo_v2_5`). +- `crate::openhuman::meet_agent::wav` — PCM16 → WAV packing for STT upload. +- `crate::openhuman::config::ops::load_config_with_timeout` — load core config for backend calls. +- `crate::openhuman::accessibility::foreground_context` — macOS foreground app/window context (screen context). +- `crate::openhuman::provider_surfaces::{store, types::RespondQueueItem}` — handoff queue lookup. +- `crate::api::{config::effective_backend_api_url, jwt::get_session_token, BackendOAuthClient}` — authenticated chat-completions call to the backend (`/openai/v1/chat/completions`, model `agentic-v1`). +- External crates: `parking_lot`, `tokio` broadcast, `tokio_util::sync::CancellationToken`, `regex`, `uuid`, `chrono`, `base64`, `once_cell`, `serde`/`serde_json`. + +## Used by + +- `src/openhuman/mod.rs` — declares `pub mod desktop_companion`. +- `src/core/all.rs` — registers the controllers and extends the schema list. +- The Tauri shell / hotkey bridge is the intended runtime driver (performs the `Idle → Listening` transition and invokes the pipeline), though no in-repo Rust caller of `run_*_turn` was found in the core crate beyond tests. + +## Notes / gotchas + +- **Single active session, process-global.** `start_session` rejects a second session unless the existing one is past TTL (then auto-expired). The `Mutex` itself serializes all session ops — no separate lock. +- **TTL auto-expiry happens in `session_status`** via inline `guard.take()` (not by calling `stop_session`) to avoid a TOCTOU race; it publishes `CompanionSessionEnded` with `reason: "ttl_expired"`. +- **Turn preconditions:** `run_text_turn` / `run_audio_turn` assume the session is already in `Listening`; the caller owns the `Idle → Listening` transition. On cancellation/empty STT the pipeline restores `Idle`. +- **POINT mapping** clamps coords to screen bounds and falls back to screen 0 when the index is out of range or the screens slice is empty (returns raw coords if empty). +- **Handoff is light-touch:** `provider_surfaces` is behaviorally incomplete — handoff just detects matches and emits `HandoffEvent`s; nothing consumes them yet. Single-word keywords use exact token matching ("slacking" won't match "slack"); multi-word ("google meet") use substring; "email"/"gmail" dedupe to provider `gmail`. +- **Screen context is macOS-only** (`gather_screen_context` returns `None` elsewhere). +- **LLM/TTS/STT all require a backend session token**; LLM responses are instructed to avoid markdown (TTS-spoken) and to embed POINT tags. +- Conversation history is capped at 50 turns (oldest drained); the LLM context window uses the last 20 turns. +- `config_set` is a stub and `config_get` returns defaults — config is not yet persisted. diff --git a/src/openhuman/devices/README.md b/src/openhuman/devices/README.md new file mode 100644 index 000000000..12cb4d393 --- /dev/null +++ b/src/openhuman/devices/README.md @@ -0,0 +1,112 @@ +# devices + +Mobile-device pairing domain. Brokers a secure, end-to-end-encrypted tunnel between the Rust core and iOS clients over the tinyhumans backend's `tunnel:*` Socket.IO relay. It registers a pairing channel, generates an X25519 keypair, performs key agreement when the device connects, and persists the resulting paired device. Frame confidentiality/integrity is XChaCha20-Poly1305 over an X25519-derived shared secret. This is the Rust counterpart to the iOS `TunnelTransport` strategy described in the repo's iOS-client notes. + +## Responsibilities + +- Register a new pairing channel with the backend tunnel (`tunnel:register`) and return QR-bound fields (`channel_id`, `pairing_token`, `core_pubkey`, optional `rpc_url`, `expires_at`). +- Generate an X25519 static keypair per pairing; encrypt and persist the private half (via `SecretStore`) so reconnect handshakes survive restart. +- Connect as `role:"core"` on the channel (`tunnel:connect`) and listen for the device. +- On the device's first `tunnel:frame`, complete the X25519 handshake (sealed-handshake or plaintext-pubkey fallback), derive the shared secret, and persist a `PairedDevice`. +- Track live peer-online status from `tunnel:peer-status` and overlay it onto `devices_list` results. +- List non-revoked devices; revoke a device (soft delete) and tear down its in-memory + tunnel state. +- Provide a reusable `TunnelCipher` (seal/open with replay-protection window) for tunnel frame crypto. +- Detect a best-effort LAN `rpc_url` for the direct-HTTP fast path. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/devices/mod.rs` | Export-only: module docstring, `pub mod` decls, re-exports of schema registry fns and public types. | +| `src/openhuman/devices/types.rs` | Serde domain types: `PairedDevice`, `PairingSession`, and the three RPC response payloads. | +| `src/openhuman/devices/rpc.rs` | RPC handler logic for the three methods + module-level in-memory state singletons (`PENDING_KEYPAIRS`, `PERSISTED_KEYPAIRS`, `PENDING_SESSIONS`, `PEER_STATUS`), LAN-URL detection, and `SecretStore`-backed keypair persist/restore. | +| `src/openhuman/devices/schemas.rs` | Controller schemas, `all_controller_schemas`/`all_registered_controllers`, and `handle_*` bridges delegating to `rpc.rs`. Mirrors `cron/schemas.rs`. | +| `src/openhuman/devices/store.rs` | SQLite persistence (`paired_devices` table) via the per-call `with_connection` pattern. | +| `src/openhuman/devices/crypto.rs` | `DeviceKeypair` (X25519 keygen, DH, byte round-trip), `TunnelCipher` (XChaCha20-Poly1305 seal/open with a `WINDOW_SIZE`=128 replay window), and base64url helpers. | +| `src/openhuman/devices/tunnel_client.rs` | Emits/parses `tunnel:*` events over the shared `SocketManager`; wire types; the one-shot ack registry (`PENDING_REGISTER`) that resolves `tunnel:register`. Frame cap 64 KB. | +| `src/openhuman/devices/bus.rs` | `DeviceTunnelSubscriber` event handler — drives handshake completion, persistence, peer-status updates, and register-ack resolution. | + +## Public surface + +Re-exported from `mod.rs`: + +- `all_devices_controller_schemas` / `all_devices_registered_controllers` (alias for `schemas::all_controller_schemas` / `all_registered_controllers`). +- Types: `CreatePairingResponse`, `ListDevicesResponse`, `PairedDevice`, `PairingSession`, `RevokeDeviceResponse`. + +Other notable public items (used across the crate but not re-exported at the domain root): `bus::register_device_tunnel_subscriber`, `crypto::{DeviceKeypair, TunnelCipher, base64url_encode, base64url_decode}`, `tunnel_client::{emit_register, emit_connect, emit_frame, resolve_register_ack, TunnelPeerStatus, TunnelFrame, TunnelRegisterResponse}`. + +## RPC / controllers + +Namespace `devices` (invoked as `openhuman.devices_`): + +| Method | Inputs | Output | Behavior | +| --- | --- | --- | --- | +| `devices_create_pairing` | `label?: string` | `CreatePairingResponse` | Registers a channel, generates+persists keypair, emits `tunnel:connect`, returns QR fields. Pairing token expires in 10 min (backend enforces real TTL). | +| `devices_list` | — | `ListDevicesResponse` | Lists non-revoked devices, overlaying live `peer_online` from `PEER_STATUS`. | +| `devices_revoke` | `channel_id: string` | `RevokeDeviceResponse` | Soft-deletes the device, clears all in-memory state for the channel, publishes `DeviceRevoked`. | + +Wired into the controller registry in `src/core/all.rs` (schemas + registered controllers + the `"devices"` namespace branch). + +## Agent tools + +None. This domain has no `tools.rs` and owns no agent tools. + +## Events + +Subscriber registered at startup from `src/core/jsonrpc.rs` via `register_device_tunnel_subscriber`. `DeviceTunnelSubscriber` (`name() = "device::tunnel"`, `domains() = ["device"]`) **handles**: + +- `DevicePeerOnline` / `DevicePeerOffline` → update `PEER_STATUS`. +- `DeviceTunnelFrame` → complete handshake + persist `PairedDevice`. +- `DeviceTunnelRegistered` → resolve the pending `tunnel:register` ack in `tunnel_client`. + +**Publishes**: + +- `DevicePaired` (after successful handshake + persistence). +- `DeviceRevoked` (from `devices_revoke`). + +Note: the `DevicePeerOnline/Offline`, `DeviceTunnelFrame`, and `DeviceTunnelRegistered` events are *originated* by `src/openhuman/socket/event_handlers.rs` (which parses the raw `tunnel:peer-status` / `tunnel:frame` / `tunnel:registered` / `tunnel:evicted` Socket.IO events and re-publishes them as `DomainEvent`s). This domain consumes them; it does not re-publish peer-status itself. + +## Persistence + +SQLite DB at `{workspace_dir}/devices/devices.db`, table `paired_devices`: + +| Column | Notes | +| --- | --- | +| `channel_id` | PK; 128-bit base32 channel id. | +| `label` | Human-readable label. | +| `device_pubkey` | Base64url X25519 device public key. | +| `core_session_token_hash` | SHA-256 of the core session token. | +| `shared_secret_encrypted` | BLOB, currently always written `NULL`. | +| `created_at` / `last_seen_at` | ISO 8601; `last_seen_at` set by `touch_device`. | +| `revoked` | Soft-delete flag; `list_devices` filters `revoked = 0`. | + +DDL is created idempotently on every connection open (`with_connection`). `peer_online` is **not** persisted — it lives only in the in-memory `PEER_STATUS` map. + +Separately, encrypted X25519 private keys are persisted as `enc2:` strings (via `keyring::SecretStore`, ChaCha20-Poly1305) keyed by `channel_id` in the in-memory `PERSISTED_KEYPAIRS` map, allowing keypair reconstruction (`load_keypair_from_store`) for reconnect handshakes. + +## Dependencies + +- `crate::openhuman::config` (`Config`, `config::rpc::load_config_with_timeout`) — workspace paths and config loading for handlers. +- `crate::openhuman::keyring::SecretStore` — encrypt/decrypt the X25519 private key at rest. +- `crate::openhuman::socket::global_socket_manager` — reuse the shared backend Socket.IO connection to emit `tunnel:*` events (no second WebSocket). +- `crate::core::event_bus` (`publish_global`, `DomainEvent`, `EventHandler`, `SubscriptionHandle`, `subscribe_global`) — pub/sub for device tunnel events. +- `crate::core::all` (`ControllerFuture`, `RegisteredController`) and `crate::core::{ControllerSchema, FieldSchema, TypeSchema}` — controller registry contract. +- `crate::rpc::RpcOutcome` — RPC handler return type. +- External crates: `rusqlite`, `chacha20poly1305`, `x25519-dalek`, `base64`, `sha2`, `chrono`, `once_cell`, `tokio`, `async_trait`, `anyhow`. + +## Used by + +- `src/core/all.rs` — registers the `devices` controllers/schemas and namespace branch. +- `src/core/jsonrpc.rs` — calls `register_device_tunnel_subscriber()` at startup. +- `src/openhuman/socket/event_handlers.rs` — parses raw `tunnel:*` Socket.IO events into `DomainEvent`s that this domain consumes, using this domain's `tunnel_client` wire types (`TunnelPeerStatus`, `TunnelFrame`). + +## Notes / gotchas + +- **Handshake frame format** (`bus::handle_tunnel_frame`): version `0x01` = sealed-handshake (`eph_pub(32) || nonce(24) || ciphertext+tag`; device seals its static pubkey under an ephemeral DH); a non-`0x01`/`0x02` leading byte falls back to treating the whole payload as a plaintext base64url device pubkey (pre-Layer-2 compat). The sealed-handshake decrypt path does **not** reuse `TunnelCipher::open`; it calls `XChaCha20Poly1305` directly on `nonce||ct` after stripping the `eph_pub` prefix. +- **`TunnelCipher` frame format** (`crypto`): `version(1)=0x01 || nonce(24) || ciphertext+tag`, random nonce per frame, replay protection via a 128-entry sliding window of seen nonces. Wrap in a `Mutex`/`RwLock` at the call site (it is `&mut self` on `open`). +- The `label` persisted on pairing currently falls back to the `channel_id` (the pending session stores no real label field; `PairingSession.channel_id` is used as the label source). +- `devices_revoke` only tears down local + in-memory state. There is **no backend revoke endpoint yet** (TODO referencing PR #709 follow-up); the backend channel is left to expire via the pairing-token TTL. +- `rpc_url` LAN detection uses the UDP "connect to 8.8.8.8" trick to read the local IPv4; port comes from `OPENHUMAN_CORE_RPC_PORT` env (default `7788`). Non-fatal if it fails. +- `tunnel:register` has no native Socket.IO ack support, so `tunnel_client` uses a global one-shot (`PENDING_REGISTER`) resolved by the bus on `tunnel:registered`, with a 10-second timeout. +- `PairingSession` and the keypair maps are in-memory only (TTL/cleanup deferred to backend semantics); they are cleared on revoke. +- Outbound `tunnel:frame` payloads are capped at 64 KB; callers are expected to stay ≤ 100 frames/s. diff --git a/src/openhuman/doctor/README.md b/src/openhuman/doctor/README.md new file mode 100644 index 000000000..29bb77518 --- /dev/null +++ b/src/openhuman/doctor/README.md @@ -0,0 +1,85 @@ +# doctor + +Diagnostic / self-check domain for OpenHuman. Runs a synchronous battery of probes against the live `Config`, the workspace directory, the daemon state file, the local environment, the memory-tree SQLite DB, the embedding provider (Ollama), and the Claude Agent SDK binary, then aggregates the findings into a severity-tagged `DoctorReport`. Exposed to CLI and JSON-RPC as `doctor.report` and `doctor.models`. This is what powers `openhuman doctor` / the Settings health surface. + +## Responsibilities + +- Validate config semantics: config file presence, `api_url` (with fallback resolution), app session JWT / sign-in state, `default_model`, temperature range (0.0–2.0), legacy `reliability.fallback_providers`, model & embedding route entries (empty hints/models, invalid embedding providers, `dimensions=0`), `memory.embedding_model` `hint:` cross-reference, at least one channel configured, and delegate-agent models. +- Check workspace integrity: directory existence, write probe (create/write/delete a temp probe file), `memory/` dir, `SYSTEM.md` prompt, and best-effort free disk space (warns under 512 MB; uses `df -m` on Unix, PowerShell `Get-PSDrive` on Windows). +- Inspect daemon state file: presence, JSON validity, heartbeat freshness (stale > 30s = error), scheduler component health (stale > 120s), and per-channel component freshness (stale > 300s). +- Probe environment commands: `git --version`, `curl --version`, `$SHELL`, and `$HOME`/`$USERPROFILE`. +- Probe memory-tree DB: warn on stale `-shm`/`-wal` SQLite side-files or not-yet-created DB; otherwise open the DB and run `SELECT COUNT(*) FROM mem_tree_chunks`. +- Probe embedding-model health: if provider is `ollama`, do a 3s blocking HTTP GET to `/api/tags` and verify the configured model is installed; non-ollama providers report OK without a local probe. +- Probe the Claude Agent SDK: if enabled, run ` --version`. +- Probe provider model availability via `run_models` (currently a stub — see Notes). + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/doctor/mod.rs` | Module docstring + exports. Declares `core`, `ops`, `schemas`; re-exports `core::*`, `ops::*` (also aliased `pub use ops as rpc`), and the schema controller pair. | +| `src/openhuman/doctor/core.rs` | All diagnostic logic + types. `run()` entry point and every `check_*` probe; `run_models()`; severity helpers; OS-specific disk/command helpers. | +| `src/openhuman/doctor/ops.rs` | Async JSON-RPC/CLI controller surface (`doctor_report`, `doctor_models`) wrapping the sync `core` logic in `spawn_blocking` and returning `RpcOutcome`. | +| `src/openhuman/doctor/schemas.rs` | Controller schemas + registry (`all_controller_schemas`, `all_registered_controllers`, `handle_report`/`handle_models`). | +| `src/openhuman/doctor/core_tests.rs` | Test suite for `core.rs` (via `#[path = "core_tests.rs"] mod tests`). | + +## Public surface + +From `mod.rs` re-exports (`core::*`): + +- Types: `Severity` (`Ok`/`Warn`/`Error`), `DiagnosticItem`, `DoctorSummary`, `DoctorReport`, `ModelProbeOutcome`, `ModelProbeEntry`, `ModelProbeSummary`, `ModelProbeReport`. +- Functions: `run(&Config) -> Result` (blocking-only — keep no `.await` inside), `run_models(&Config, use_cache) -> Result`. + +From `ops` (also re-exported as `doctor::rpc`): + +- `doctor_report(&Config) -> Result, String>` +- `doctor_models(&Config, use_cache: bool) -> Result, String>` + +Schema pair re-exported as `all_doctor_controller_schemas` / `all_doctor_registered_controllers`. + +## RPC / controllers + +Namespace `doctor`, two functions: + +| Method | Inputs | Output | +| --- | --- | --- | +| `doctor.report` | none | `DoctorReport` ("Run diagnostics for workspace and runtime configuration.") | +| `doctor.models` | `use_cache: Option` (default `true`) | `ModelProbeReport` ("Probe provider model availability and auth status.") | + +Both handlers load config via `config_rpc::load_config_with_timeout()` and return `RpcOutcome::single_log(...)`. Wired into the global registry in `src/core/all.rs` (controllers, schemas, and the namespace description "Run diagnostics for workspace and runtime health."). + +## Agent tools + +None. This domain owns no agent tools (no `tools.rs`). + +## Events + +None. No `bus.rs` / event-bus subscribers or publishers. + +## Persistence + +None of its own (no `store.rs`). It only **reads** existing state owned by other domains: the daemon state file (`service::daemon::state_file_path`), the memory-tree SQLite DB (`/memory_tree/chunks.db`), config files, and `/SYSTEM.md` / `/memory/`. Its only writes are an ephemeral workspace probe file that is immediately deleted. + +## Dependencies + +- `crate::openhuman::config::{Config, rpc}` — reads the live config for all probes; `config_rpc::load_config_with_timeout` in the handlers. +- `crate::openhuman::service::daemon` — `state_file_path` for the daemon heartbeat/component snapshot. +- `crate::openhuman::memory_store::{chunks::store, factories}` — `with_connection` for the DB probe; `effective_embedding_settings` to resolve the intended embedding provider/model. +- `crate::openhuman::inference::{provider, local}` — `provider::list_providers` (model targets) and `local::ollama_base_url` (embedding probe). +- `crate::api::{config, jwt}` — `effective_api_url` fallback resolution and `get_session_token` for sign-in state. +- `crate::core::all::{ControllerFuture, RegisteredController}`, `crate::core::{ControllerSchema, FieldSchema, TypeSchema}` — controller/schema plumbing. +- `crate::rpc::RpcOutcome` — handler return contract. +- External: `reqwest` (blocking client + URL parse), `serde`/`serde_json`, `chrono`, `anyhow`. + +## Used by + +- `src/core/all.rs` — registers the doctor controllers/schemas into the global RPC + CLI registry (the only in-tree consumer). + +## Notes / gotchas + +- `run()` is **strictly blocking** by contract (file system, sqlite, blocking HTTP). `reqwest::blocking::Client` panics inside a tokio runtime, so `ops::doctor_report` runs the whole thing in `tokio::task::spawn_blocking`. Do not add `.await` inside `core::run`. +- `run_models` / `doctor.models` is effectively a **stub**: it enumerates providers from `inference::provider::list_providers` but marks every entry `Skipped` with message "model catalog refresh removed" (catalog refresh was removed). It never actually probes auth/availability despite the schema description. +- The embedding probe is capped at a 3s timeout to avoid stalling on a slow Ollama daemon; non-ollama providers short-circuit to OK. +- `model_matches` treats `name` vs `name:tag` as a match only when at most one side is tagged; two differently-tagged names are not considered equal. +- Severity rollup: `DoctorSummary` counts `Ok`/`Warn`/`Error` items; checks are intentionally lenient (missing optional dirs/tools → `Warn`, not `Error`). +- OS-specific helpers (`available_disk_space_mb`, `check_command_available`, `check_claude_agent_sdk`) set `CREATE_NO_WINDOW` on Windows to avoid console flashes. diff --git a/src/openhuman/embeddings/README.md b/src/openhuman/embeddings/README.md new file mode 100644 index 000000000..20dcf8bee --- /dev/null +++ b/src/openhuman/embeddings/README.md @@ -0,0 +1,107 @@ +# embeddings + +Embedding providers for the OpenHuman memory system. Converts text into numerical vectors for semantic search, abstracting over multiple backends behind a single `EmbeddingProvider` trait. Owns provider construction (factory + slug catalog), per-endpoint client-side rate limiting, 429/503 retry-with-backoff logic, and the JSON-RPC surface that the Settings UI uses to pick a provider, store API keys, test connectivity, and embed text. The default provider is the OpenHuman backend ("managed" / "cloud", Voyage-backed); other paths include direct Voyage, OpenAI, Cohere, local Ollama, any OpenAI-compatible `custom:` endpoint, and a no-op (keyword-only) fallback. + +## Responsibilities + +- Define `EmbeddingProvider` (async trait) and the canonical embedding-space signature format (`provider=…;model=…;dims=…`) — the single source of truth so config-derived and live-provider signatures stay byte-identical (#1574). +- Implement each provider: cloud/managed, Voyage, OpenAI(-compatible), Cohere, Ollama, noop. +- Construct providers from a slug + model + dims (with or without credentials) via the factory. +- Maintain the static provider/model catalog (slugs, labels, API-key/endpoint requirements, dimension presets) consumed by the frontend picker. +- Throttle outbound cloud-embedding HTTP requests with a process-global, per-endpoint token bucket (loopback exempt). +- Parse `Retry-After` and apply 429/503 exponential backoff in HTTP-based providers. +- Expose RPC handlers for settings, API-key management, embed, and connection testing; trigger memory wipe / re-embed backfill when the embedding signature changes. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/embeddings/mod.rs` | Module docstring + module decls + `pub use` re-exports; re-exports `VectorStore` et al. from `memory_store::vectors`; exposes `all_embeddings_controller_schemas` / `all_embeddings_registered_controllers`. | +| `src/openhuman/embeddings/provider_trait.rs` | `EmbeddingProvider` trait (`name`/`model_id`/`dimensions`/`signature`/`embed`/`embed_one`) and `format_embedding_signature`. | +| `src/openhuman/embeddings/factory.rs` | `create_embedding_provider`, `create_embedding_provider_with_credentials`, `default_embedding_provider` (cloud), `default_local_embedding_provider` (Ollama). Maps provider slugs to concrete impls; unknown slugs error. | +| `src/openhuman/embeddings/catalog.rs` | Static catalog of `EmbeddingProviderEntry` + `EmbeddingModelPreset`; slug constants; `all_providers` / `find_provider` / `find_model` / `default_model_for`. | +| `src/openhuman/embeddings/cloud.rs` | `OpenHumanCloudEmbedding` — default provider; resolves session JWT + API URL per call, delegates HTTP to `OpenAiEmbedding` against `/openai/v1`. `DEFAULT_CLOUD_EMBEDDING_MODEL` = `embedding-v1`, dims 1024. | +| `src/openhuman/embeddings/openai.rs` | `OpenAiEmbedding` — OpenAI-compatible `POST /v1/embeddings`; URL inference, 429/503 retry, rate-limit gating, dimension/count validation. Used directly by cloud, voyage, and custom paths. | +| `src/openhuman/embeddings/voyage.rs` | `VoyageEmbedding` — thin wrapper delegating to `OpenAiEmbedding` against `api.voyageai.com`. | +| `src/openhuman/embeddings/cohere.rs` | `CohereEmbedding` — Cohere-native `POST /v2/embed` wire format (`texts`, `embedding_types`, nested `embeddings.float`) with its own 429/503 retry loop. | +| `src/openhuman/embeddings/ollama.rs` | `OllamaEmbedding` — local Ollama `POST /api/embed`; base-url/model normalization, blank-input zero-vector preservation, NaN-encoding 500 per-text recovery (TAURI-RUST-AZ). Defaults `bge-m3`/1024. | +| `src/openhuman/embeddings/noop.rs` | `NoopEmbedding` — returns empty vectors; keyword-only fallback (`name`/`model_id` = "none", dims 0). | +| `src/openhuman/embeddings/rate_limit.rs` | Process-global, per-endpoint token-bucket request limiter; `set_embedding_rate_limit`, `embedding_rate_limit`, `acquire_embedding_slot`. Default 60/min; loopback exempt; `0` disables. | +| `src/openhuman/embeddings/retry_after.rs` | `parse_retry_after_ms`, `backoff_ms_for_attempt`, and the `MAX_429_RETRIES` / backoff constants. | +| `src/openhuman/embeddings/rpc.rs` | RPC business logic: `get_settings`, `update_settings`, `set_api_key`, `clear_api_key`, `embed`, `test_connection`, plus `provider_from_config` and `resolve_api_key`. | +| `src/openhuman/embeddings/schemas.rs` | Controller schemas + `handle_*` param-deserializing wrappers delegating to `rpc.rs`. | +| `src/openhuman/embeddings/mod_tests.rs` | Module-level tests (via `#[path]`). | +| `src/openhuman/embeddings/openai_tests.rs` / `ollama_tests.rs` | Sibling test suites for the OpenAI and Ollama providers (via `#[path]`). | + +## Public surface + +From `mod.rs` re-exports: + +- Trait + helper: `EmbeddingProvider`, `format_embedding_signature`. +- Providers: `OpenHumanCloudEmbedding`, `OpenAiEmbedding`, `OllamaEmbedding`, `NoopEmbedding` (Voyage/Cohere are public via their submodules through the factory). +- Factory fns: `create_embedding_provider`, `default_embedding_provider`, `default_local_embedding_provider`. +- Config-driven builder: `provider_from_config` (re-exported from `rpc`) — same construction `embed` uses, for callers like `codegraph` that need a provider without a JSON-RPC round-trip. +- Defaults/consts: `DEFAULT_CLOUD_EMBEDDING_MODEL`, `DEFAULT_CLOUD_EMBEDDING_DIMENSIONS`, `DEFAULT_OLLAMA_MODEL`, `DEFAULT_OLLAMA_DIMENSIONS`. +- Vector-store re-exports (moved to `memory_store::vectors`, re-exported here for callers): `store`, `bytes_to_vec`, `cosine_similarity`, `vec_to_bytes`, `SearchResult`, `VectorStore`. +- RPC registry: `all_embeddings_controller_schemas`, `all_embeddings_registered_controllers`. + +## RPC / controllers + +Namespace `embeddings` (6 controllers, registered through `src/core/all.rs`): + +| Method | Description | +| --- | --- | +| `embeddings.get_settings` | Current provider/model/dims/rate-limit + full provider catalog with `has_api_key` flags and `vector_search_enabled`. | +| `embeddings.update_settings` | Update provider/model/dimensions/custom_endpoint/rate_limit. Requires `confirm_wipe=true` when **dimensions** change (returns `EMBEDDINGS_DIMENSION_CHANGE_REQUIRES_WIPE` otherwise); wipes memory on dim change and queues a re-embed backfill on any signature change. Also syncs `config.embeddings_provider` workload routing. | +| `embeddings.set_api_key` | Store an API key under credential provider `embeddings:`. | +| `embeddings.clear_api_key` | Remove the stored key for `embeddings:`. | +| `embeddings.embed` | Embed input texts using the configured provider; returns vectors, dimensions, count. | +| `embeddings.test_connection` | Run a single test embed against the configured or specified provider/model/dims. | + +All handlers return `RpcOutcome` via `into_cli_compatible_json`. + +## Agent tools + +None. This module owns no `tools.rs`; agent-facing memory tools live in `memory/tools/` and consume embeddings indirectly through the memory store. + +## Events + +None. No `bus.rs` / `EventHandler` impls. Cross-module side effects in `update_settings` are direct calls (`memory::read_rpc::wipe_all_rpc`, `memory_queue::ensure_reembed_backfill`), not domain events. + +## Persistence + +No `store.rs`. Settings live in the global `Config` (`config.memory.embedding_provider` / `embedding_model` / `embedding_dimensions` / `embedding_rate_limit_per_min`, plus `config.embeddings_provider`). API keys are persisted via the credentials domain (`AuthService`) under provider key `embeddings:`. Vectors themselves are stored by `memory_store::vectors` (re-exported here, not owned). + +## Dependencies + +- `crate::openhuman::config` — `Config`, `config::ops::load_config_with_timeout`, `build_runtime_proxy_client` (proxy-aware reqwest), config save. +- `crate::openhuman::credentials` — `AuthService`, `APP_SESSION_PROVIDER` for resolving the session JWT (cloud) and storing/loading provider API keys. +- `crate::openhuman::memory_store::vectors` — vector store types re-exported from `mod.rs`. +- `crate::openhuman::memory::read_rpc` — `wipe_all_rpc` on a dimension change. +- `crate::openhuman::memory_queue` — `ensure_reembed_backfill` on a signature change. +- `crate::openhuman::inference::local` — `ollama_base_url()` when constructing the Ollama provider. +- `crate::api::config` — `effective_api_url` for the cloud backend base URL. +- `crate::core::all` — `ControllerFuture`, `RegisteredController` for the RPC registry. +- `crate::core::{ControllerSchema, FieldSchema, TypeSchema}` — schema definitions. +- `crate::core::observability` — `report_error_or_expected` so transient upstream HTTP failures demote to warning breadcrumbs instead of Sentry errors. +- `crate::rpc::RpcOutcome` — handler return contract. + +## Used by + +- `memory_store/*` — factories, vectors store, chunks store, unified store, retrieval, client (primary consumer; constructs providers and uses signatures to partition the embedding space). +- `memory/*` — ingestion queue, preferences, tools (recall/store/forget), `read_rpc`. +- `memory_tree/score/embed/cloud.rs` — scoring path. +- `codegraph/*` — index/search/tools obtain a provider via `provider_from_config` for `signature()` + direct embedding. +- `voice/factory.rs`, `screen_intelligence`, `channels`, `agent` tools — indirect consumers. +- `config/schema/load.rs` — wires `memory.embedding_rate_limit_per_min` into `rate_limit::set_embedding_rate_limit`. +- `core/all.rs` — registers the RPC controllers; `core/observability.rs` — classifies the canonical embedding error strings. + +## Notes / gotchas + +- **Signature drift is the core hazard.** `format_embedding_signature` is the single source of truth; both config-derived and live-provider signatures route through it. A mismatch silently splits one embedding space into two (#1574). `update_settings` keys memory wipe off **dimension** change (vectors stay comparable across provider/model swaps at the same dimensionality) but queues a re-embed backfill on **any** signature change. +- **Rate limiting is account-wide and process-global**, keyed by resolved base URL, with capacity = 1 token (no burst) to stay strictly under a hard per-minute cap. Ephemeral provider instances share one budget per endpoint. Loopback hosts are exempt; `limit==0` disables. The acquire chokepoint sits **inside** the retry loop so retried attempts consume tokens. +- **Cloud provider resolves auth lazily per call** — it can be constructed before login; the first `embed()` errors clearly if unauthenticated. It honors `OPENHUMAN_WORKSPACE` for the auth-profiles directory so non-default workspaces (tests/multi-instance) don't silently lose their session. +- **Ollama NaN recovery (TAURI-RUST-AZ):** a single bad input can poison a whole batch with a 500 `unsupported value: NaN`; the provider re-issues per-text and substitutes empty embeddings for the offending entries. Blank/whitespace inputs are preserved as zero-vectors so result length always matches input length. `local-*` model IDs are rejected (they're virtual routing aliases). +- **Custom endpoints** are encoded in the provider slug as `custom:`; `resolve_api_key` and the RPC handlers normalize this back to the `custom` credential slug. +- **Voyage and cloud reuse `OpenAiEmbedding`** for HTTP plumbing (Voyage's API is an OpenAI superset). **Cohere** speaks a distinct wire format and has its own retry loop. +- Error strings from OpenAI/Cohere providers intentionally preserve the `(429 ` / status substring so `core::observability`'s `TransientUpstreamHttp` classifier downgrades them. diff --git a/src/openhuman/health/README.md b/src/openhuman/health/README.md new file mode 100644 index 000000000..fe7f1471b --- /dev/null +++ b/src/openhuman/health/README.md @@ -0,0 +1,86 @@ +# health + +In-process health registry for the OpenHuman core. Tracks per-component liveness (status, last-ok / last-error timestamps, restart counts) plus process metadata (PID, uptime), exposes a snapshot over JSON-RPC/CLI, and keeps itself current by subscribing to `system`/`channel` domain events on the global event bus. State is purely in-memory (a process-global `OnceLock` registry) — nothing is persisted to disk. It also serves static system info (version/OS/arch/PID). + +## Responsibilities + +- Maintain a process-global registry of named components, each with `status`, `updated_at`, `last_ok`, `last_error`, `restart_count`. +- Provide mutators: `mark_component_ok`, `mark_component_error`, `bump_component_restart`. +- Produce a point-in-time `HealthSnapshot` (PID, uptime seconds, components map) and its JSON form. +- Drive component state automatically from `DomainEvent`s via an event-bus subscriber. +- Expose `health.snapshot` and `health.system_info` RPC/CLI controllers. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/health/mod.rs` | Export-only: declares modules, re-exports `core::*`, `ops::*`, `pub use ops as rpc`, and the controller-schema pair (`all_health_controller_schemas` / `all_health_registered_controllers`). | +| `src/openhuman/health/core.rs` | The registry itself: `ComponentHealth`/`HealthSnapshot` types, the `OnceLock` singleton (`started_at` + `Mutex`), and the mutators/`snapshot`/`snapshot_json` functions. | +| `src/openhuman/health/ops.rs` | RPC handler logic returning `RpcOutcome`: `health_snapshot()` and `system_info()` (+ `SystemInfo` type). | +| `src/openhuman/health/schemas.rs` | Controller schemas + `handle_snapshot`/`handle_system_info` async handlers that delegate to `ops` and serialize via `into_cli_compatible_json`. | +| `src/openhuman/health/bus.rs` | `HealthSubscriber` (`EventHandler`) and `register_health_subscriber()`; maps domain events to registry mutations. | + +## Public surface + +From `core.rs` (re-exported via `pub use core::*`): +- Types: `ComponentHealth`, `HealthSnapshot`. +- Functions: `mark_component_ok(component)`, `mark_component_error(component, error)`, `bump_component_restart(component)`, `snapshot() -> HealthSnapshot`, `snapshot_json() -> serde_json::Value`. + +From `ops.rs` (re-exported via `pub use ops::*`, also aliased `pub use ops as rpc`): +- `health_snapshot() -> RpcOutcome`, `system_info() -> RpcOutcome`, and the `SystemInfo` struct. + +From `bus.rs`: `HealthSubscriber`, `register_health_subscriber()`. + +From `schemas.rs` (re-exported under aliased names): `all_health_controller_schemas`, `all_health_registered_controllers`. + +## RPC / controllers + +Two controllers in the `health` namespace: + +| Method | Inputs | Outputs | +| --- | --- | --- | +| `openhuman.health_snapshot` | none | `snapshot` (JSON): full serialized `HealthSnapshot`. | +| `openhuman.health_system_info` | none | `version`, `os`, `arch`, `pid`. | + +`system_info`'s `version` is `CARGO_PKG_VERSION`; `os`/`arch` come from `std::env::consts`. Legacy callers may send `openhuman.system_info`, which the alias table rewrites to `health_system_info` before dispatch. + +## Events + +Subscriber `HealthSubscriber` (`name = "health::registry"`) registered via `register_health_subscriber()` on the global event bus, filtered to the `system` and `channel` domains. It reacts to: + +| DomainEvent | Action | +| --- | --- | +| `SystemStartup { component }` | `mark_component_ok(component)` | +| `HealthChanged { component, healthy, message }` | OK if `healthy`, else `mark_component_error` with `message` (default `"unknown health error"`) | +| `HealthRestarted { component }` | `bump_component_restart(component)` | +| `ChannelConnected { channel }` | `mark_component_ok("channel:")` | +| `ChannelDisconnected { channel, reason }` | `mark_component_error("channel:", reason)` | + +It only subscribes; it does not publish events. + +## Persistence + +None on disk. State lives in a process-global `OnceLock` (lazy-initialized) holding a `Mutex>` and an `Instant` start time. Cleared on process exit; uptime resets each launch. + +## Dependencies + +- `crate::core::event_bus` (`DomainEvent`, `EventHandler`, `SubscriptionHandle`, `subscribe_global`) — to receive system/channel events. +- `crate::core::all` (`ControllerFuture`, `RegisteredController`) and `crate::core::{ControllerSchema, FieldSchema, TypeSchema}` — controller registry wiring. +- `crate::rpc::RpcOutcome` — RPC handler return contract. +- External crates: `chrono` (RFC3339 timestamps), `parking_lot::Mutex`, `serde`/`serde_json`, `async_trait`. + +## Used by + +- `src/core/all.rs` — registers `all_health_*` controllers into the registry. +- `src/core/jsonrpc.rs` — references health (snapshot/system_info surface). +- `src/openhuman/channels/runtime/{startup,supervision}.rs` and `src/openhuman/channels/tests/health.rs` — channel runtime updates component health. +- `src/openhuman/cron/scheduler.rs`, `src/openhuman/update/scheduler.rs` — emit/consume health signals. + +## Notes / gotchas + +- The registry is a global singleton; tests use UUID-suffixed component names to avoid cross-test contention rather than resetting shared state. +- `upsert_component` creates entries lazily with initial status `"starting"` and always refreshes `updated_at` after the update closure. +- `mark_component_ok` clears `last_error`; `mark_component_error` leaves `last_ok` intact (so the last-known-good time survives a failure). +- `restart_count` uses `saturating_add` (won't overflow). +- The `system_info` schema declares `pid` as a `String` (`TypeSchema::String`) even though the `SystemInfo` struct serializes `pid` as a numeric `u32` — schema type vs. wire type differ here. +- `bus.rs` short-circuits double registration via a `OnceLock` and warns (does not panic) if the bus isn't initialized. diff --git a/src/openhuman/heartbeat/README.md b/src/openhuman/heartbeat/README.md new file mode 100644 index 000000000..aa22d87e4 --- /dev/null +++ b/src/openhuman/heartbeat/README.md @@ -0,0 +1,92 @@ +# heartbeat + +Periodic background scheduler that wakes on a configurable interval to do two things: (1) run the **planner** — evaluate upcoming meetings, cron reminders, and urgent notifications and dispatch deduplicated proactive notifications; and (2) optionally drive the **subconscious** engine for task-driven evaluation via local model inference. The loop reloads config before every tick so UI setting changes apply without an app restart, and it sleeps before the first tick so a fresh login never burns budget immediately. + +## Responsibilities + +- Run a long-lived tick loop (`HeartbeatEngine::run`) gated by `[heartbeat]` config; clamps the interval to a 5-minute floor. +- Reload config each tick and re-emit a settings line only when the relevant settings change. +- On each tick, run the planner (`evaluate_and_dispatch`) to collect → plan → dedupe → persist → notify for three categories: meetings, reminders, important events. +- When `inference_enabled`, fetch the shared global subconscious engine and call `engine.tick()`; otherwise (legacy mode) just count tasks in `HEARTBEAT.md`. +- Collect calendar meetings via Composio (mode-aware: backend vs direct), cron reminders, and unread/urgent integration notifications. +- Pick a delivery stage + message text per event based on lead time (`plan_delivery_for_event`). +- Dedupe deliveries both in-tick (content-based `overlap_key`) and across ticks (durable SQLite store), prune dedupe rows older than 14 days. +- Persist each alert into the notifications store and emit a core notification; optionally emit a `ProactiveMessageRequested` event for external channel delivery. +- Expose RPC to read/update heartbeat settings and to force one immediate planner tick. +- Seed a default `HEARTBEAT.md` in the workspace (`ensure_heartbeat_file`). + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/heartbeat/mod.rs` | Module docstring + exports; re-exports planner/rpc/engine and the controller-schema pair. | +| `src/openhuman/heartbeat/engine.rs` | `HeartbeatEngine` — the tick loop, config reload, planner-tick + subconscious dispatch, `collect_tasks`/`parse_tasks`/`ensure_heartbeat_file`. | +| `src/openhuman/heartbeat/rpc.rs` | RPC handlers `settings_get` / `settings_set` / `tick_now`; `HeartbeatSettingsPatch` / `HeartbeatSettingsView`; clamps and applies settings, bootstraps/stops the loop. | +| `src/openhuman/heartbeat/schemas.rs` | Controller schemas + `handle_*` thunks for the three RPC methods. | +| `src/openhuman/heartbeat/planner/mod.rs` | `evaluate_and_dispatch` — orchestrates collect → plan → dedupe → persist → notify; owns the cross-tick + in-tick dedupe logic. | +| `src/openhuman/heartbeat/planner/types.rs` | `HeartbeatCategory`, `PendingEvent`, `PlannedDelivery`, public `PlannerRunSummary`. | +| `src/openhuman/heartbeat/planner/collectors.rs` | Source collectors: `collect_cron_reminders`, `collect_calendar_meetings` (Composio fan-out + rotation), `collect_relevant_notifications`, calendar payload extraction. | +| `src/openhuman/heartbeat/planner/plan.rs` | `plan_delivery_for_event` — stage selection (`heads_up`/`final_call`/`starting_now`, `soon`/`due`, `important_now`) and message text per category and lead time. | +| `src/openhuman/heartbeat/planner/persistence.rs` | `persist_heartbeat_alert` — writes a durable `IntegrationNotification` (provider `"heartbeat"`, `triage_action="react"`) into the notifications store. | +| `src/openhuman/heartbeat/planner/store.rs` | SQLite dedupe store (`heartbeat_notification_state`): `mark_sent`, `prune_old`, `SentMarker`. | +| `src/openhuman/heartbeat/planner/utils.rs` | Pure helpers: `sanitize_preview`, `stable_key` (SHA-256), `compute_overlap_key` (category + normalized title + 15-min bucket). | + +## Public surface + +- `engine::HeartbeatEngine` — `new(config, workspace_dir)`, `run()`, `collect_tasks()`, `parse_tasks()`, `ensure_heartbeat_file(dir)`. +- `planner::evaluate_and_dispatch(config, now) -> PlannerRunSummary`. +- `planner::PlannerRunSummary` — `{ source_events, deliveries_attempted, deliveries_sent, deliveries_skipped_dedup }`. +- `rpc::{settings_get, settings_set, tick_now}`, `rpc::{HeartbeatSettingsPatch, HeartbeatSettingsView}`. +- `all_heartbeat_controller_schemas` / `all_heartbeat_registered_controllers` (re-exported from `schemas.rs`). + +## RPC / controllers + +Namespace `heartbeat` (wired into the registry in `src/core/all.rs`): + +| Method | Inputs | Output | Notes | +| --- | --- | --- | --- | +| `heartbeat.settings_get` | — | `settings` (JSON `HeartbeatSettingsView`) | Read current settings. | +| `heartbeat.settings_set` | optional `enabled`, `interval_minutes`, `inference_enabled`, `notify_meetings`, `notify_reminders`, `notify_relevant_events`, `external_delivery_enabled`, `meeting_lookahead_minutes`, `max_calendar_connections_per_tick`, `reminder_lookahead_minutes` | `settings` (updated view) | Clamps (`interval_minutes`≥5, lookaheads/cap≥1), saves config, then bootstraps (`subconscious::global::bootstrap_after_login`) or stops (`stop_heartbeat_loop`) the loop. | +| `heartbeat.tick_now` | — | `summary` (JSON `PlannerRunSummary`) | Runs one immediate planner tick. | + +## Events + +- **Publishes** `DomainEvent::ProactiveMessageRequested` (via `core::event_bus::publish_global`) when `external_delivery_enabled` and the planned delivery sets `allow_external` — routes the proactive message to active external channels. +- **Publishes** core notifications via `notifications::bus::publish_core_notification` (`CoreNotificationEvent`) for every delivered alert. +- This module defines no `EventHandler`/subscriber of its own (no `bus.rs`). + +## Persistence + +- **Dedupe store** (`planner/store.rs`): SQLite at `{workspace_dir}/heartbeat/heartbeat_state.db`, table `heartbeat_notification_state` keyed by `dedupe_key` (`INSERT OR IGNORE` for atomic dedupe). Rows older than 14 days are pruned each tick. +- **Durable alerts** (`planner/persistence.rs`): written into the shared notifications store (provider `"heartbeat"`) — not owned by this module's schema. +- The dedupe marker is written **after** the durable notification persists, so a failed persist never permanently suppresses retries. + +## Dependencies + +- `crate::openhuman::config` — `Config` / `HeartbeatConfig`; reads the `[heartbeat]` block, `load_or_init` / `load_config_with_timeout` / `save`. +- `crate::openhuman::subconscious::global` — `get_or_init_engine` (shared engine for inference ticks), `bootstrap_after_login`, `stop_heartbeat_loop`. +- `crate::openhuman::cron` — `list_jobs`, `CronJob` to surface reminder-like jobs. +- `crate::openhuman::composio` — `client` (mode-aware factory, backend/direct list+execute), `types`, `googlecalendar_args` to poll Google Calendar events. +- `crate::openhuman::notifications` — `store` (read unread/urgent items, `insert_if_not_recent`), `bus::publish_core_notification`, `types` (`IntegrationNotification`, `NotificationStatus`, `CoreNotificationEvent`/`CoreNotificationCategory`). +- `crate::core::event_bus` — `publish_global` / `DomainEvent` for proactive-message dispatch. +- `crate::core::all` / `crate::core::{ControllerSchema, FieldSchema, TypeSchema}` — controller registration. +- `crate::rpc::RpcOutcome` — RPC return contract. +- External crates: `rusqlite` (dedupe store), `sha2` + `hex` (stable keys), `chrono`, `serde`/`serde_json`. + +## Used by + +- `src/core/all.rs` — registers the heartbeat controllers + schemas into the RPC registry. +- `src/openhuman/subconscious/global.rs` — constructs `HeartbeatEngine` and owns its run lifecycle. +- `src/openhuman/workspace/ops.rs` — calls `HeartbeatEngine::ensure_heartbeat_file` during workspace setup. + +## Notes / gotchas + +- **5-minute floor** on `interval_minutes` is enforced both at the RPC clamp and at runtime in `run()`. +- **First tick is delayed** — the loop sleeps for the interval before the first evaluation (fresh-login budget safety). +- **Cross-source dedupe** uses `overlap_key` = `category | normalized_title | 15-min bucket`, so the same meeting surfaced by both a cron job and a calendar connection only notifies once; the disk dedupe key folds in the delivery `stage`. +- **All-day calendar events are intentionally skipped** — extraction only accepts `start.dateTime`, never `start.date` (avoids birthdays/OOO/holidays being promoted to meetings). +- **Self-escalation guard**: `collect_relevant_notifications` filters out `provider == "heartbeat"` to avoid a feedback loop where each tick re-escalates its own alerts. +- **Calendar fan-out is rotated** across ticks (`select_calendar_connections_for_tick`) and capped by `max_calendar_connections_per_tick` so a user with many calendar connections doesn't hammer Composio in a single tick. +- **Timezone correctness**: calendar start times are parsed via RFC3339 offset parsing and normalized to UTC (regression-pinned for non-UTC offsets, issue #1714); direct-mode users see their own calendar, not the backend tenant's (#1710). +- Grace windows extend up to ~10 minutes past an anchor so tick alignment doesn't miss a meeting/reminder. +- `mark_sent` returns `false` on an existing key (dedupe hit) rather than erroring. diff --git a/src/openhuman/http_host/README.md b/src/openhuman/http_host/README.md new file mode 100644 index 000000000..e46ff40c7 --- /dev/null +++ b/src/openhuman/http_host/README.md @@ -0,0 +1,77 @@ +# http_host + +Static directory hosting over ad-hoc, in-process HTTP listeners owned by the core. Lets trusted callers (RPC/CLI) start, inspect, list, and stop lightweight file servers that expose a chosen directory on a chosen TCP port. Each server runs as an in-process `axum` task sharing the core's lifetime, and defaults to HTTP Basic authentication using the active user's identity plus a randomly generated password. There is no on-disk persistence — the registry of running servers lives in process memory and is torn down on shutdown. + +## Responsibilities + +- Start an `axum` static file server bound to a requested `bind_host:port`, serving a canonicalized directory tree. +- Default-on HTTP Basic auth: derive a username from the active session (falling back to env), generate a random password per server. +- Serve files (streamed) and directory listings (auto `index.html`, otherwise a generated HTML listing), with MIME type inference by extension. +- Enforce path-traversal safety: reject `..`, absolute, URL-encoded escape, and out-of-root resolved paths. +- Track running servers in an in-process registry keyed by a UUID `server_id`; prevent duplicate `bind_host:port` registrations and prune finished tasks. +- Gracefully stop individual servers (cancel + join) and register a one-time core shutdown hook that stops all servers on core exit. +- Expose start/stop/get/list as JSON-RPC / CLI controllers under the `http_host` namespace. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/http_host/mod.rs` | Module docstring + declarations; re-exports controller schema/registry pair; defines `LOG_PREFIX = "[http_host]"`. | +| `src/openhuman/http_host/types.rs` | Serde types: `StartHostedDirParams`, `HostedDirLookupParams`, `HostedDirServerInfo`, `HostedDirAuth`, and the `*Result` response shapes. | +| `src/openhuman/http_host/ops.rs` | In-process server manager: `HostedDirRegistry` (Mutex) + `OnceLock` singleton, `start/list/get/stop/stop_all` ops, shutdown-hook registration, finished-task pruning, collision checks. | +| `src/openhuman/http_host/handlers.rs` | `axum` router + request handlers (`HostedDirState`, `build_router`, root/path/file/directory serving, streamed file responses, generated directory listing HTML). | +| `src/openhuman/http_host/auth.rs` | Basic-auth verification (`ensure_authorized`), default username resolution from session/env, username sanitization, random password generation. | +| `src/openhuman/http_host/path_utils.rs` | Path safety + URL/HTML helpers: directory canonicalization, request-path traversal resolution, bind-host/label sanitization, href builders, `escape_html`, `content_type_for_path`, `redact_path_for_log`. | +| `src/openhuman/http_host/rpc.rs` | RPC adapters wrapping ops into `RpcOutcome` (`start`/`stop`/`get`/`list`). | +| `src/openhuman/http_host/schemas.rs` | `ControllerSchema`s + `handle_*` controller handlers; `all_controller_schemas` / `all_registered_controllers`. | +| `src/openhuman/http_host/tests.rs` | `#[cfg(test)]` test module. | + +## Public surface + +- `all_http_host_controller_schemas()` / `all_http_host_registered_controllers()` — re-exported from `schemas`; wired into the core controller registry. +- `pub mod ops` — `start_hosted_dir_server`, `list_hosted_dir_servers`, `get_hosted_dir_server`, `stop_hosted_dir_server`, `stop_all_hosted_dir_servers`. +- `pub mod rpc` — async `start`/`stop`/`get`/`list` returning `RpcOutcome<...>`. + +(`auth`, `handlers`, `path_utils`, `types` are private to the module.) + +## RPC / controllers + +Namespace `http_host` (invoked as `openhuman.http_host_`): + +| Method | Inputs | Outputs | +| --- | --- | --- | +| `http_host.start` | `directory` (req), `port` (req; `0` = OS-chosen), `bind_host` (default `127.0.0.1`), `server_name`, `disable_auth` (default false), `username` | `server` (JSON: `HostedDirServerInfo` incl. URLs + generated auth credentials) | +| `http_host.stop` | `server_id` (req) | `stopped` (bool), `server` (final snapshot) | +| `http_host.get` | `server_id` (req) | `server` (incl. current auth credentials) | +| `http_host.list` | none | `servers` (array of `HostedDirServerInfo`) | + +## Persistence + +None on disk. Running servers are held in a process-global `HostedDirRegistry` (`OnceLock` wrapping `Mutex>`). State is volatile and cleared on shutdown. + +## Dependencies + +- `crate::openhuman::config` — `load_config_with_timeout` to resolve the active config when deriving the default Basic-auth username (`auth.rs`). +- `crate::openhuman::credentials::session_support` — `build_session_state` to read the active user identity for the default auth username (`auth.rs`). +- `crate::core::shutdown` — `register` a one-time hook so all hosted servers stop when the core shuts down (`ops.rs`). +- `crate::core::all` — `ControllerFuture`, `RegisteredController` for controller registration (`schemas.rs`). +- `crate::core::{ControllerSchema, FieldSchema, TypeSchema}` — controller schema types (`schemas.rs`). +- `crate::rpc::RpcOutcome` — RPC response envelope (`rpc.rs`). +- External crates: `axum` (HTTP server/router), `tokio` (`TcpListener`, tasks), `tokio_util` (`CancellationToken`, `ReaderStream`), `uuid`, `base64`, `rand`, `urlencoding`, `serde`/`serde_json`. + +## Used by + +- `src/core/all.rs` — registers `all_http_host_registered_controllers()` (line ~145) and `all_http_host_controller_schemas()` (line ~309) into the core controller registry, exposing the RPC/CLI surface. +- `src/openhuman/mod.rs` — declares `pub mod http_host`. +- `src/core/observability.rs` references `http_host::path_utils` paths in error-classification docs/tests (`http_host` directory-not-found maps to a filesystem user-path-invalid class). + +## Notes / gotchas + +- **Credentials in responses**: `HostedDirServerInfo.auth` carries the generated password in `start`/`get`/`list` responses. Treat RPC output as sensitive. +- **Auth defaults**: when auth is enabled and no username resolves from session/env, the username falls back to `"openhuman"`. Passwords are 18 random bytes, URL-safe base64 (no padding). +- **Path safety**: `resolve_request_path` rejects URL-encoded traversal and verifies the canonicalized target stays under the hosted root; `canonicalize_hosted_directory` resolves and verifies the root is a real directory before binding. +- **Port `0`**: binding with port `0` lets the OS pick a free port; the actual assigned port (from `local_addr`) is what gets stored and reported. +- **No duplicate binds**: `start` rejects another server already registered on the same `bind_host:port`. +- **Logging redaction**: directory paths are logged via `redact_path_for_log` (only the leaf name, prefixed `/`) — full host paths are not emitted. +- **Lifetime**: servers do not persist across core restarts; the shutdown hook (`register_shutdown_hook_once`) is installed lazily on the first `start`. +- **IPv6**: `bind_host` containing `:` (and not already bracketed) is wrapped in `[...]` for both the bind target and the URL rendering. diff --git a/src/openhuman/inference/README.md b/src/openhuman/inference/README.md new file mode 100644 index 000000000..0806db380 --- /dev/null +++ b/src/openhuman/inference/README.md @@ -0,0 +1,124 @@ +# inference + +Unified inference domain: the canonical home for everything LLM/STT/TTS/embedding-related. It owns the local-runtime manager (Ollama / LM Studio / Whisper / Piper), the unified cloud + local provider abstraction (trait, factory, router, reliability/retry wrapper), voice transcription and TTS inference, OpenAI/Codex subscription OAuth, and an OpenAI-compatible `/v1/chat/completions` HTTP endpoint. It consolidates the previously separate `local_ai/`, `providers/`, and inference parts of `voice/` under one domain root. The RPC surface is still split across the `inference.*` and `local_ai.*` namespaces for backwards compatibility. + +## Responsibilities + +- Resolve workload names (`chat`, `reasoning`, `agentic`, `coding`, `memory`, `embeddings`, `heartbeat`, `learning`, `subconscious`, etc.) and provider strings (`openhuman`, `cloud`, `ollama:`, `lmstudio:`, `claude_agent_sdk:`, `:[@]`) to a concrete `Box` + model id. +- Manage the local AI runtime: detect/spawn/adopt `ollama serve` and LM Studio, install/run Whisper (STT) and Piper (TTS), track download progress, and enforce a minimum-context-window floor. +- Provide chat, vision (multimodal), summarization, embeddings, sentiment, and "should react" inference operations. +- Wrap providers with retry/backoff and config-rejection/billing-error classification (`reliable`, `config_rejection`, `billing_error`). +- Multi-model routing via a hint table (`RouterProvider`) keyed off abstract tier model names (`reasoning-v1`, `agentic-v1`, `coding-v1`, etc.). +- Run ChatGPT/Codex OAuth (PKCE) for the `openai` cloud slug and persist tokens in the encrypted auth-profile store. +- Expose an OpenAI-compatible `/v1/*` HTTP endpoint guarded by a stable user-managed external bearer. +- Detect device hardware profile and recommend/apply local model presets/tiers. + +## Key files + +| File / dir | Role | +| --- | --- | +| `mod.rs` | Domain root; module decls + re-exports; wires `inference.*` controller schemas/controllers. | +| `ops.rs` | Canonical handler file — `inference_*` business logic returning `RpcOutcome`; delegates to `local`, `provider`, `sentiment`, `device`, `presets`, `openai_oauth`. Includes Sentry-noise suppression for expected provider/user-config failures. | +| `schemas.rs` | `inference.*` controller schemas + `handle_*` fns + param DTOs. | +| `types.rs` | Serde DTOs: `LocalAiStatus`, `LocalAiAssetsStatus`, `LocalAiDownloadsProgress`, `LocalAiEmbeddingResult`, `LocalAiSpeechResult`, `LocalAiTtsResult`, etc. | +| `device.rs` | `DeviceProfile` hardware detection (RAM/CPU/GPU/OS), cached. | +| `model_ids.rs` | Effective chat/vision/embedding/STT/TTS/quantization model id resolution from config. | +| `model_context.rs` | Known model context-window sizes (`context_window_for_model`) for pre-dispatch budgeting. | +| `presets.rs` | `ModelPreset`, `ModelTier`, `VisionMode`; tier recommendation + apply-to-config; MVP preset gating. | +| `sentiment.rs` | `SentimentResult` + emotion/valence analysis via the local model. | +| `parse.rs` / `paths.rs` | Output parsing helpers / on-disk model artifact paths. | +| `local/` | Local runtime manager (was `local_ai/`). | +| `local/core.rs` | `LocalAiService` singleton (`global`/`try_global`), `model_artifact_path`. | +| `local/ops.rs` | Local RPC entrypoints (`local_ai_status/prompt/summarize/vision_prompt/embed/should_react`, `ReactionDecision`); re-exported as `local::rpc`. | +| `local/schemas.rs` | `local_ai.*` controller schemas + handlers. | +| `local/ollama.rs`, `local/lm_studio.rs` | Provider-specific runtime drivers; base-url resolution. | +| `local/install*.rs`, `local/voice_install_common.rs` | Whisper/Piper install + shared download logic. | +| `local/model_requirements.rs` | `MIN_CONTEXT_TOKENS`, `evaluate_context`, `ContextEligibility`. | +| `local/service/` | `LocalAiService` impl split: `bootstrap`, `ollama_admin`, `public_infer`, `speech`, `vision_embed`, `whisper_engine`, `assets`, `spawn_marker`. | +| `provider/` | Unified provider abstraction (was `providers/`). | +| `provider/traits.rs` | `Provider` trait + `ChatMessage`/`ChatRequest`/`ChatResponse`/`ToolCall`/`UsageInfo`/`ProviderDelta` etc. | +| `provider/factory.rs` | `create_chat_provider`, `provider_for_role`, provider-string grammar, local/cloud construction; `BYOK_INCOMPLETE_SENTINEL`. | +| `provider/router.rs` | `RouterProvider` hint-based multi-model routing. | +| `provider/reliable.rs` | Retry/backoff wrapper. | +| `provider/compatible*.rs` | OpenAI-compatible provider (request dump/parse/stream/types). | +| `provider/openhuman_backend.rs` | Managed OpenHuman backend provider (session JWT). | +| `provider/claude_agent_sdk/` | Claude Agent SDK subprocess provider (`protocol.rs`, `subprocess.rs`). | +| `provider/config_rejection.rs`, `provider/billing_error.rs` | Error classifiers (unknown-model / config rejection / budget exhausted). | +| `provider/temperature.rs`, `provider/thread_context.rs` | Per-workload temperature override; thread context plumbing. | +| `provider/ops.rs` | `list_configured_models`, SessionExpired publishing on auth failure. | +| `provider/schemas.rs` | Provider-layer schemas. | +| `voice/` | Inference implementations imported by `crate::openhuman::voice`. | +| `voice/cloud_transcribe.rs`, `voice/local_transcribe.rs`, `voice/local_speech.rs` | STT (cloud + local) and local TTS. | +| `voice/streaming.rs`, `voice/postprocess.rs`, `voice/hallucination.rs` | Streaming transcription, post-processing, hallucination filtering. | +| `openai_oauth/` | ChatGPT/Codex OAuth: `config.rs` (Codex OAuth config), `flow.rs` (start/complete/status/disconnect), `store.rs` (token persistence). | +| `http/` | OpenAI-compatible endpoint: `server.rs` (`router()`), `types.rs`; `EXTERNAL_OPENAI_COMPAT_PROVIDER` bearer id. | + +## Public surface + +From `mod.rs` re-exports: + +- `device::DeviceProfile` +- `model_context::context_window_for_model` +- `presets::{ModelPreset, ModelTier, VisionMode}` +- `sentiment::SentimentResult` +- `types::{LocalAiStatus, LocalAiAssetStatus, LocalAiAssetsStatus, LocalAiDownloadProgressItem, LocalAiDownloadsProgress, LocalAiEmbeddingResult, LocalAiSpeechResult, LocalAiTtsResult}` +- `local::all_local_ai_controller_schemas` / `local::all_local_ai_registered_controllers` +- `rpc` (alias for `ops`) and `all_inference_controller_schemas` / `all_inference_registered_controllers` + +Provider-layer (via `provider::`): `Provider`, `ChatMessage`, `ChatRequest`, `ChatResponse`, `create_chat_provider`, `provider_for_role`, `BYOK_INCOMPLETE_SENTINEL`, plus error classifiers. Local runtime: `local::{global, try_global}` → `Arc`. + +## RPC / controllers + +Two namespaces, both wired into the controller registry (`src/core/all.rs`). + +`inference.*` (`schemas.rs`): `status`, `get_client_config`, `update_model_settings`, `update_local_settings`, `list_models`, `device_profile`, `presets`, `apply_preset`, `diagnostics`, `openai_oauth_start`, `openai_oauth_complete`, `openai_oauth_status`, `openai_oauth_disconnect`, `summarize`, `prompt`, `vision_prompt`, `test_provider_model`, `should_react`, `analyze_sentiment`. + +`local_ai.*` (`local/schemas.rs`): `agent_chat`, `agent_chat_simple`, `transcribe`, `transcribe_bytes`, `tts`, `assets_status`, `downloads_progress`, `download_asset`, `install_whisper`, `install_piper`, `whisper_install_status`, `piper_install_status`, `test_connection`. + +Also exposes a non-RPC HTTP router (`http::router()`) nested at `/v1` by `src/core/jsonrpc.rs` (`/v1/chat/completions`, `/v1/models`), accepting either the core bearer or a stable external API key. + +## Events + +- Publishes `DomainEvent::SessionExpired` from `provider/ops.rs` when a provider chat attempt fails auth, so the credentials layer can clear/refresh the session. +- No `bus.rs` / `EventHandler` subscribers in this domain. + +## Persistence + +- `openai_oauth/store.rs` persists OAuth tokens via the credentials auth-profile store (`AuthProfilesStore`, `auth-profiles.json`, encrypted at rest) under profile key `provider:openai` / profile `oauth`. +- `LocalAiService` holds in-process runtime state (status, whisper engine handle, owned `ollama serve` child) via the `local::global` `OnceCell` singleton — process-lifetime, not durably persisted. +- Model artifacts live under `/models/local-ai/` (`local/core.rs::model_artifact_path`); installed Whisper/Piper assets via `local/install*`. +- Routing/provider/local settings persisted through `config` (no dedicated `store.rs`). + +## Dependencies + +- `crate::openhuman::config` — `Config`, `config::rpc` (load/save, `ModelSettingsPatch`, `LocalAiSettingsPatch`), cloud-provider schema (`AuthStyle`, slug reservation, id generation), abstract tier model constants. Heaviest dependency. +- `crate::openhuman::credentials` — `AuthService`, `AuthProfilesStore`/`AuthProfile`/`TokenSet`, state dir — for OAuth token storage and provider auth resolution. +- `crate::openhuman::tools` — `ToolSpec`/`ToolCall` types used in provider chat requests (agent tool plumbing). +- `crate::openhuman::agent` — agent harness types referenced by provider/thread-context paths. +- `crate::openhuman::voice` — voice RPC/audio layer that imports these inference STT/TTS implementations (also a consumer). +- `crate::openhuman::prompt_injection` — prompt-injection handling on the inference path. +- `crate::openhuman::util` — small shared helpers. +- `crate::core::all` — `ControllerFuture`, `RegisteredController` (controller registry). +- `crate::core::types` — `ControllerSchema`, `FieldSchema`, `TypeSchema`. +- `crate::core::event_bus` — `DomainEvent`, `publish_global` (SessionExpired). +- `crate::core::observability` — `expected_error_kind` for Sentry-noise classification. +- `crate::core::jsonrpc` — endpoint mounting reference for `/v1`. +- `crate::core::auth` — bearer auth for the OpenAI-compatible endpoint. +- External: `motosan_ai_oauth` (Codex OAuth), `sysinfo` (device profile), `reqwest`, `whisper`-cpp engine bindings. + +## Used by + +Widely depended on (74 internal `use` sites + many external). Top consumers (by file count) are the agent layer (`agent/harness`, `agent/harness/session`, `agent/tools`, `agent/triage`, `agent/harness/subagent_runner`), `context`, `voice`, `routing`, `memory_tree/tree_runtime`, `learning` (+ `learning/transcript_ingest`), `channels`, `embeddings`, `subconscious`, `screen_intelligence`, `threads`, and `migrations`/`config/schema`. + +## Notes / gotchas + +- `mod.rs` re-exports `super::{device, model_ids, parse, paths, presets, sentiment, types}` under `local::` so files migrated from the old `local_ai/` keep compiling without rewriting `super::` paths. +- Provider strings carry an optional `@` suffix that pins a per-workload temperature; the suffix is stripped before the model id is sent upstream. +- `update_model_settings` silently drops reserved cloud-provider slugs (`openhuman`/`cloud`/`pid` built-ins the frontend echoes back); `apply_model_settings` re-injects them from stored config so they aren't lost. +- `ops.rs` deliberately demotes known provider/user-config failures (unknown cloud provider, 401/429, model-not-found) to `warn!` to keep them out of Sentry; only unclassified failures escalate to `error!`. +- `apply_preset` is MVP-gated: only the 1B local preset (`ram_2_4gb`) and `disabled` are accepted; `custom` cannot be applied via this path. +- `diagnostics` returns its payload unwrapped (no `{result, logs}` envelope) to match the legacy `local_ai_diagnostics` shape that `json_rpc_e2e` asserts against. +- Adopted (externally started) `ollama serve` daemons are never killed on exit; only the child OpenHuman itself spawned (`owned_ollama`) is. +- `local::global` lazily initialises the `LocalAiService` singleton; use `try_global()` on shutdown paths to avoid creating it just to no-op. +- The `/v1/*` endpoint uses a stable external bearer (`EXTERNAL_OPENAI_COMPAT_PROVIDER`) separate from the core launch bearer, so external OpenAI-compatible harnesses can call it. +- Tests serialize through `inference_test_guard()` (a process-global mutex) since the runtime singleton and config are shared. diff --git a/src/openhuman/integrations/README.md b/src/openhuman/integrations/README.md new file mode 100644 index 000000000..3bde5daa5 --- /dev/null +++ b/src/openhuman/integrations/README.md @@ -0,0 +1,85 @@ +# integrations + +Agent integration tools — a family of third-party data/action providers (web search, place lookup, market data, web scraping/automation, phone calls) exposed to the agent as `Tool` implementations. Most integrations **proxy through OpenHuman backend endpoints** (`/agent-integrations/*`) authenticated with the user's app-session JWT, so API keys, billing, rate limiting, and provider markup stay server-side. A few (SearXNG, Brave, Querit, Seltz) call user-configured or provider endpoints **directly** with a user-supplied key/URL. The module owns no RPC controllers, no persisted state, and no event-bus subscribers — it is purely a shared HTTP client plus a set of agent tools. + +## Responsibilities + +- Provide `IntegrationClient`, a shared `reqwest`-based HTTP client for backend-proxied integrations: backend-URL sanitization, bearer auth, `{success,data,error}` envelope parsing, bounded error-detail extraction, and a lazily-fetched pricing cache. +- Build the client from root config (`build_client`): resolve the backend API base URL (falling back off local-AI endpoints) and the app-session JWT; return `None` when the user is not signed in. +- Fetch per-integration pricing from the backend (`/agent-integrations/pricing`), with a Composio-`direct`-mode short-circuit (`pricing_for_config`). +- Implement and export the concrete agent tools (Apify, Brave, Google Places, Parallel, Querit, SearXNG, Seltz, stock/market data, TinyFish, Twilio). +- Classify transport- and user-state failures through `core::observability::report_error_or_expected` so user-environment / user-input errors demote to breadcrumbs instead of firing Sentry events. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/integrations/mod.rs` | Export-only module root. Re-exports `build_client`, `pricing_for_config`, `IntegrationClient`, and the shared pricing/envelope/`ToolScope` types. Module docstring describes the proxy-vs-direct trust model. | +| `src/openhuman/integrations/client.rs` | `IntegrationClient` (backend URL, auth token, reusable HTTP client, `OnceCell` pricing cache) + `post`/`get`/`pricing`. Includes `sanitize_backend_url` (strips inference-style paths — issue #2075), `extract_error_detail`, `build_client`, `pricing_for_config`. | +| `src/openhuman/integrations/types.rs` | Shared serde types: `IntegrationPricing`, `PricingIntegrations`, `IntegrationPricingEntry`, `BackendResponse` envelope; re-exports `ToolScope` from `tools::traits`. | +| `src/openhuman/integrations/tools.rs` | Tool module aggregator — declares the `tools/` submodules and re-exports every concrete tool struct. | +| `src/openhuman/integrations/test_support.rs` | `spawn_fake_integration_backend` — an axum mock of the `/agent-integrations/*` backend routes that records requests; used by integration tool tests. | +| `src/openhuman/integrations/tools/apify.rs` | Apify actor run + status + dataset results (`apify_run_actor`, `apify_get_run_status`, `apify_get_run_results`). Backend-proxied. | +| `src/openhuman/integrations/tools/brave.rs` | Brave Search direct API (web/news/image/video search). Auth via `X-Subscription-Token`; not backend-proxied. | +| `src/openhuman/integrations/tools/google_places.rs` | Google Places search + details. Backend-proxied. | +| `src/openhuman/integrations/tools/parallel.rs` | Parallel search/extract/chat/research/enrich/dataset (FindAll). Backend-proxied. Exports `SearchResponse`/`SearchResultItem`. | +| `src/openhuman/integrations/tools/querit.rs` | Querit AI web search — direct `POST https://api.querit.ai/v1/search`, bearer auth. | +| `src/openhuman/integrations/tools/searxng.rs` | SearXNG self-hosted/private search — direct `GET /search?format=json` against a user-configured endpoint. Normalizes to `{title,url,snippet,source}`. Exports `normalize_categories`, args/response types, `MAX_RESULTS`. | +| `src/openhuman/integrations/tools/seltz.rs` | Seltz web search — direct `POST https://api.seltz.ai/v1/search`, `x-api-key` auth. | +| `src/openhuman/integrations/tools/stock_prices.rs` | Market data (Alpha Vantage via backend `/agent-integrations/financial-apis/*`): quote, options, exchange-rate, crypto-series, commodity. | +| `src/openhuman/integrations/tools/tinyfish.rs` | TinyFish search / page fetch / goal-based browser-automation agent run. Backend-proxied. | +| `src/openhuman/integrations/tools/twilio.rs` | Outbound phone calls via backend Twilio (`twilio_call`, `ToolScope::CliRpcOnly`, `PermissionLevel::Execute`). | +| `*_tests.rs` / inline `#[cfg(test)]` | Co-located tests for client, mod, test_support, and the apify/parallel/tinyfish tools (others use inline test modules). | + +## Public surface + +From `src/openhuman/integrations/mod.rs`: + +- `IntegrationClient` — shared HTTP client; `new(backend_url, auth_token)`, async `post(path, body)`, `get(path)`, `pricing()`. +- `build_client(&Config) -> Option>` — constructs the client from resolved backend URL + session JWT, or `None` if signed out. +- `pricing_for_config(&IntegrationClient, &Config) -> IntegrationPricing` — pricing fetch honouring Composio `direct` mode. +- Types: `BackendResponse`, `IntegrationPricing`, `IntegrationPricingEntry`, `PricingIntegrations`, `ToolScope` (re-exported from `tools::traits`). +- Tool structs (via `tools.rs`): `ApifyRunActorTool`, `ApifyGetRunStatusTool`, `ApifyGetRunResultsTool`; `BraveWebSearchTool`, `BraveNewsSearchTool`, `BraveImageSearchTool`, `BraveVideoSearchTool`; `GooglePlacesSearchTool`, `GooglePlacesDetailsTool`; `ParallelSearchTool`, `ParallelExtractTool`, `ParallelChatTool`, `ParallelResearchTool`, `ParallelEnrichTool`, `ParallelDatasetTool` (+ `SearchResponse`, `SearchResultItem`); `QueritSearchTool`; `SearxngSearchTool` (+ `SearxngSearchArgs`, `SearxngSearchResponse`, `normalize_categories`, `SEARXNG_MAX_RESULTS`); `SeltzSearchTool`; `StockQuoteTool`, `StockOptionsTool`, `StockExchangeRateTool`, `StockCryptoSeriesTool`, `StockCommodityTool`; `TinyFishSearchTool`, `TinyFishFetchTool`, `TinyFishAgentRunTool`; `TwilioCallTool`. + +## RPC / controllers + +None. This module exposes no RPC controllers, `schemas.rs`, or `handle_*` functions. Its capabilities reach callers only as agent `Tool`s registered by the `tools` domain. + +## Agent tools + +Owns the integration tool family listed above (each implements `crate::openhuman::tools::traits::Tool`). Tools are **not** self-registering here — they are constructed and registered by `src/openhuman/tools/ops.rs`, gated per-provider by `config.integrations..is_active()` (apify, google_places, tinyfish, stock_prices, twilio) and, for search providers, governed by `search.engine`. `parallel` is parsed but its tools are selected via the search engine setting, not its own toggle. `twilio_call` is `ToolScope::CliRpcOnly` (excluded from the autonomous agent loop). Backend-proxied tools require a signed-in user (`build_client` → `Some`); direct-API tools (searxng/brave/querit/seltz) require their respective user-configured endpoint/key. + +## Events + +None — no `bus.rs`, no `EventHandler` impls, no `DomainEvent` publishes or subscriptions. + +## Persistence + +None — no `store.rs`. The only in-memory cache is the per-`IntegrationClient` pricing `OnceCell`, which is process-lifetime, non-persisted, and resets on rebuild. + +## Dependencies + +- `crate::openhuman::tools::traits` — `Tool`, `ToolResult`, `ToolCallOptions`, `ToolCategory`, `ToolScope`, `PermissionLevel`; the trait surface every integration tool implements. +- `crate::core::observability` — `report_error_or_expected` / `expected_error_kind`; classify transport and user-state failures so user-environment errors skip Sentry. +- `crate::openhuman::tls` — `tls_client_builder()` for platform-appropriate TLS (schannel on Windows, rustls elsewhere). +- `crate::openhuman::util` — string truncation helpers (`truncate_at_byte_boundary`, `truncate_with_ellipsis`, `truncate_with_suffix`, `utf8_safe_prefix_at_byte_boundary`) for bounding error/log bodies. +- `crate::openhuman::config` — root `Config`; `composio.mode` / `COMPOSIO_MODE_DIRECT` for the pricing short-circuit, and (read by `tools/ops.rs`) the `integrations.*` toggles. +- `crate::api::config` — `api_url`, `effective_backend_api_url`, `effective_api_url`, `normalize_backend_api_base_url`, `redact_url_for_log`; backend URL resolution/sanitization/redaction. +- `crate::api::jwt::get_session_token` — the app-session JWT used as the bearer for backend-proxied calls. + +## Used by + +- `src/openhuman/tools/{ops,mod,schemas}.rs` — registers the integration tools into the agent tool registry, gated by config. +- `src/openhuman/tools/impl/network/web_search.rs` — web-search tool surface backed by integration providers. +- `src/openhuman/learning/linkedin_enrichment.rs` — uses an integration (Apify-style enrichment) for LinkedIn data. +- `src/openhuman/composio/*` — the Composio domain reuses `IntegrationClient` / `build_client` for its backend-proxied OAuth-integration calls. +- `src/core/observability.rs` — references integration error-classification paths. +- `src/openhuman/agent/harness/test_support.rs` — test wiring. + +## Notes / gotchas + +- **Two trust models.** Backend-proxied tools never see provider API keys (the backend holds them). Direct-API tools (searxng/brave/querit/seltz) send requests straight from the local core to a user-configured endpoint/key — callers must keep those base URLs trusted because traffic leaves the local process. The mod docstring calls this out. +- **`backend_url` invariant.** `IntegrationClient::new` re-runs `sanitize_backend_url` as defense-in-depth (issue #2075 / Sentry `OPENHUMAN-TAURI-H6`/`-HN`): an inference-style `BACKEND_URL` (e.g. `…/openai/v1/chat/completions`) would otherwise concatenate onto every domain path and 404. It `warn!`s once when it has to fix up the input. +- **Error classification.** Transport failures (DNS/TLS/connect/timeout) and user-state envelope errors (`success:false`, 4xx auth/input) are routed through `report_error_or_expected` and demoted to breadcrumbs; genuine backend bugs and 5xx still surface to Sentry. +- **Pricing is best-effort.** `IntegrationClient::pricing()` returns a default empty struct on network error so tool registration never fails; in Composio `direct` mode `pricing_for_config` short-circuits to empty (no backend session, logs `[composio-direct]`). +- **No sidecar / single client per build.** `build_client` returns a fresh `Arc` (and a fresh pricing cache) each call; there is no global singleton. diff --git a/src/openhuman/javascript/README.md b/src/openhuman/javascript/README.md new file mode 100644 index 000000000..1522bb3e0 --- /dev/null +++ b/src/openhuman/javascript/README.md @@ -0,0 +1,81 @@ +# javascript + +First-class **JavaScript language slot** for the core. This module is a thin re-export facade: it gives the rest of the codebase a stable `crate::openhuman::javascript` surface that talks to a *language* (`javascript`) rather than to a concrete backend. Today the implementation backend is the managed Node.js runtime in `crate::openhuman::runtime_node`; the facade exists so a future sibling slot (`python`, `ruby`, or an alternate JS backend) can swap in without churning callers. It owns no logic of its own — every symbol it exposes is a `pub use` of a `runtime_node` item. + +## Responsibilities + +- Provide the canonical `openhuman::javascript::*` import path for Node.js runtime resolution/bootstrap and the agent-tool bridge. +- Re-export the JS-namespaced RPC controller schemas/registry (`javascript.list_tools`, `javascript.execute_tool`) under stable `all_javascript_*` names so `src/core/all.rs` can wire them. +- Re-export `NodeBootstrap` and related resolve/download/extract types used by the system exec tools to locate a `node` binary. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/javascript/mod.rs` | Entire module. Module docstring + `pub use` re-exports from `runtime_node`. No types, no logic, no tests. | + +> All behavior lives in `src/openhuman/runtime_node/` (`bootstrap.rs`, `downloader.rs`, `extractor.rs`, `resolver.rs`, `ops.rs`, `schemas.rs`, `rpc.rs`, `types.rs`). This facade only renames/forwards their public surface. + +## Public surface + +Re-exported from `runtime_node`: + +- **Types**: `ExecuteToolOutcome`, `RuntimeToolSummary` (from `runtime_node::types`); `NodeBootstrap`, `NodeSource`, `ResolvedNode`, `NodeDistribution`, `SystemNode`. +- **Functions**: `execute_tool`, `list_tools` (tool bridge); `detect_system_node`, `parse_node_version` (system node detection); `download_distribution`, `fetch_shasums`, `atomic_install`, `extract_distribution` (managed-toolchain install path). +- **Controller wiring**: `all_javascript_controller_schemas` (← `all_runtime_node_controller_schemas`), `all_javascript_registered_controllers` (← `all_runtime_node_registered_controllers`). + +## RPC / controllers + +Namespace `javascript` (defined in `runtime_node/schemas.rs`, surfaced through this facade): + +| Method | Inputs | Outputs | +| --- | --- | --- | +| `javascript.list_tools` | — | `tools`: array of tool metadata (`name`, `description`, `category`, `permission_level`, `scope`, `supports_markdown`, `parameters`). | +| `javascript.execute_tool` | `tool_name` (required), `args` (optional Json, defaults to `{}`), `prefer_markdown` (optional bool) | `tool_name`, `elapsed_ms` (u64), `result` (MCP-style ToolResult: `{content, is_error, markdownFormatted?}`). | + +Handlers (`runtime_node/rpc.rs`) load config via `config::rpc::load_config_with_timeout`, build the full tool set via `tools::all_tools_with_runtime`, then list or dispatch a tool by exact name. Results are returned as `RpcOutcome` (CLI-compatible JSON). + +## Agent tools + +This module owns no `tools.rs`. Instead it acts as a **bridge** to the cross-cutting tool registry: `runtime_node::ops::build_runtime_tools` constructs the full tool set (`tools::all_tools_with_runtime`) and `list_tools` / `execute_tool` enumerate or invoke them by name. + +## Events + +`execute_tool` (in `runtime_node/ops.rs`) publishes via `publish_global`: + +- `DomainEvent::ToolExecutionStarted { tool_name, session_id: "javascript" }` +- `DomainEvent::ToolExecutionCompleted { tool_name, session_id: "javascript", success, elapsed_ms }` + +No event subscribers (`bus.rs`) are owned here. + +## Persistence + +No persisted domain state (`store.rs`). The managed-Node install path (`extractor::atomic_install`) writes a Node toolchain to disk, but that is filesystem installation, not domain state. + +## Dependencies + +(Direct dependency of this facade module: `runtime_node`. Transitive deps below come from the `runtime_node` implementation it forwards to.) + +- `crate::openhuman::runtime_node` — the entire backing implementation; all re-exports come from here. +- `crate::core::all` (`ControllerFuture`, `RegisteredController`) + `crate::core` (`ControllerSchema`, `FieldSchema`, `TypeSchema`) — RPC controller schema contract. +- `crate::core::event_bus` — publishes `ToolExecutionStarted` / `ToolExecutionCompleted`. +- `crate::openhuman::config` — loads `Config` to build the tool set; RPC handlers use `config::rpc::load_config_with_timeout`. +- `crate::openhuman::tools` — `all_tools_with_runtime`, `Tool`, `ToolCallOptions`, `ToolScope`; the actual tool registry being bridged. +- `crate::openhuman::agent::host_runtime` — `NativeRuntime` / `RuntimeAdapter` injected when building tools. +- `crate::openhuman::memory` / `memory_store` — `create_memory_with_local_ai` for memory-backed tools. +- `crate::openhuman::security` — `SecurityPolicy::from_config` + workspace audit logger. +- `crate::openhuman::skills::types::ToolResult` — tool result envelope type. + +## Used by + +- `src/core/all.rs` — wires `all_javascript_registered_controllers` / `all_javascript_controller_schemas` into the controller registry; the about-app catalog describes namespace `javascript`. +- `src/openhuman/tools/ops.rs`, `tools/impl/system/shell.rs`, `node_exec.rs`, `npm_exec.rs` — import `openhuman::javascript::NodeBootstrap` for Node binary resolution. +- `src/openhuman/runtime_node/rpc.rs` — calls `javascript::list_tools` through the facade. + +## Notes / gotchas + +- This is a **rename-only facade** (see the canonical "thin facade" exception in CLAUDE.md): no `types.rs`/`ops.rs`/`store.rs` here by design. Edit behavior in `runtime_node`, not here. +- `javascript.execute_tool` rebuilds the entire tool set on every call (via `build_runtime_tools`), then finds the named tool — there is no persistent tool cache. +- The `session_id` on emitted tool events is the literal string `"javascript"`, not a real chat/session id. +- Tool-list output is sorted by tool name (`list_tools` in `runtime_node/ops.rs`). +- Both RPC handlers are async and resolve config through `load_config_with_timeout`, so a slow/blocked config load surfaces as an RPC error. diff --git a/src/openhuman/keyring/README.md b/src/openhuman/keyring/README.md new file mode 100644 index 000000000..f714d43d9 --- /dev/null +++ b/src/openhuman/keyring/README.md @@ -0,0 +1,91 @@ +# keyring + +OS-keychain-backed secret storage with pluggable test/debug backends, plus a ChaCha20-Poly1305 encrypted secret store for config-field encryption. This is an **infrastructure domain** — it has no RPC controllers, no agent tools, and no event-bus subscribers. It provides a namespaced, user-scoped, `get`/`set`/`delete` interface over a backend selected once per process, and a separate `SecretStore` type that other domains use to encrypt/decrypt secret values stored in config. All keys are scoped under a `user_id` so multiple users coexist without collision; the backend entry key format is `"{user_id}:{logical_key}"`. + +## Responsibilities + +- Provide `get` / `set` / `delete` / `get_or_create_random` over a process-global secret-storage backend. +- Select the backend exactly once at first use (env override → `cfg(test)` → staging/prod vs dev) and freeze it in a `OnceLock`. +- Probe and cache backend availability (`is_available`) so callers (wallet guards, snapshot loops) can fall back to file storage without re-triggering OS keychain prompts. +- Migrate a secret from a plaintext file into the active backend (`migrate_from_file`), verifying the write before deleting the source. +- Encrypt/decrypt config-field secrets via `SecretStore` (ChaCha20-Poly1305, `enc2:` prefix), including migration of the legacy XOR `enc:` format. +- Load/cache a single app-scoped master encryption key from the OS keychain at startup (`init_master_key`) for the `encrypted_file` backend, reducing keychain access to one call per process. +- Migrate the legacy plaintext `dev-keychain.json` into the encrypted `secrets.enc` file on first use. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/keyring/mod.rs` | Module docstring + exports only. Documents backend-selection priority and the Linux-headless availability note. Re-exports the public surface. | +| `src/openhuman/keyring/ops.rs` | Core operations: `get`/`set`/`delete`/`is_available`/`get_or_create_random`/`migrate_from_file`, the `MigrationOutcome` enum, `namespaced_key` helper, cached availability probe, and `force_backend_for_test`. | +| `src/openhuman/keyring/store.rs` | Backend selection + global state. Owns `WORKSPACE_DIR` and `BACKEND` `OnceLock`s, `init_workspace`, `backend()`, `build_backend()`, and `workspace_dir_for_file_backend()` path derivation. | +| `src/openhuman/keyring/backend.rs` | `KeyringBackend` trait + `OsBackend` (native keychain via the `keyring` crate, service name `"openhuman"`), `FileBackend` (plaintext `dev-keychain.json`, test/debug only), and test-only `MockBackend`. | +| `src/openhuman/keyring/encrypted_file_backend.rs` | `EncryptedFileBackend` — all secrets in one ChaCha20-Poly1305 `secrets.enc` file keyed by an app master key; `init_master_key`/`is_master_key_available`; legacy `dev-keychain.json` migration; corrupt-file quarantine. | +| `src/openhuman/keyring/encrypted_store.rs` | `SecretStore` — config-field encryption (`enc2:` ChaCha20-Poly1305, legacy `enc:` XOR migration), keychain-backed master key with legacy `.secret_key` file migration, process-wide key cache, Windows ACL repair (`icacls`). | +| `src/openhuman/keyring/crypto.rs` | Shared ChaCha20-Poly1305 helpers (`chacha20_encrypt`/`chacha20_decrypt`), random-byte generation, hex encode/decode. Used by both `encrypted_store` and `encrypted_file_backend`. | +| `src/openhuman/keyring/error.rs` | `KeyringError` (thiserror) with variants `Os`/`InvalidUtf8`/`MigrationReadFailed`/`VerifyFailed`/`MigrationDeleteFailed`/`RandomGeneration`/`Crypto`/`Backend`, plus a log-safe `diagnostic()` that preserves the `keyring::Error` variant + `OSStatus`. | +| `src/openhuman/keyring/tests.rs` | Module tests (backend isolation via `force_backend_for_test`). | +| `src/openhuman/keyring/encrypted_store_tests.rs` | `SecretStore` tests (wired via `#[path]` from `encrypted_store.rs`). | + +## Public surface + +Re-exported from `mod.rs`: + +- `KeyringBackend` — backend trait (`get`/`set`/`delete`/`name`). +- `SecretStore` — config-field encrypt/decrypt; `encrypt`/`decrypt`/`decrypt_and_migrate`/`needs_migration`/`is_encrypted`/`is_secure_encrypted`/`new`. +- `KeyringError` — error enum with `diagnostic()`. +- `init_master_key` — load the app master key from the OS keychain at startup (staging/prod only). +- `init_workspace` — register the workspace dir for file/encrypted-file backends. +- `get`, `set`, `delete`, `get_or_create_random`, `is_available`, `migrate_from_file`, `MigrationOutcome`. +- `force_backend_for_test` — `pub(crate)`, test-only. + +## RPC / controllers + +None. The keyring domain exposes no `schemas.rs`, no `all_*_controller_schemas`, and no `openhuman.keyring_*` methods. It is consumed in-process by other domains. + +## Agent tools + +None. + +## Events + +None — no `bus.rs`, publishes/subscribes to no `DomainEvent`. + +## Persistence + +Secret storage backend, selected once and frozen in a `OnceLock`: + +- **`os`** (production default outside staging/prod special-casing): native OS credential store — macOS Keychain, Windows Credential Manager, Linux Secret Service — under service name `"openhuman"`. +- **`encrypted_file`** (staging/production, and via `OPENHUMAN_KEYRING_BACKEND=encrypted_file`): single ChaCha20-Poly1305 file `{workspace}/secrets.enc`, encrypted with a master key loaded once from the OS keychain (`openhuman` / `app:master_key`). Files written `0600` on Unix via temp-file + atomic rename. +- **`file`** (dev default, `cfg(test)`, or `OPENHUMAN_KEYRING_BACKEND=file`): plaintext JSON `{workspace}/dev-keychain.json`. **Not encrypted — test/debug only.** +- **`mock`** (test-only): in-memory `HashMap`. + +`SecretStore` additionally manages a master encryption key: keychain-backed (slot `secretstore.master_key`) in normal builds with one-time migration from the legacy `{data_dir}/openhuman/.secret_key` file; the file path is retained only for unit tests. Decoded keys are cached process-wide keyed by normalized path. + +Workspace dir resolves from `init_workspace`, else `OPENHUMAN_WORKSPACE`, else `~/.openhuman` (or `~/.openhuman-staging` under `OPENHUMAN_APP_ENV=staging`). + +## Dependencies + +Internal openhuman/core modules: **none** — the keyring module's own files only `use crate::openhuman::keyring::*` (self-internal). It is a leaf infrastructure module. External crates: `keyring`, `chacha20poly1305`, `serde_json`, `parking_lot`, `thiserror`, `anyhow`, `chrono`, `dirs`. + +## Used by + +Discovered consumers (`crate::openhuman::keyring::*`): + +- `src/lib.rs` and `src/core/jsonrpc.rs` — call `init_master_key()` at startup. +- `src/openhuman/security/secrets.rs`, `src/openhuman/security/mod.rs` — secret handling. +- `src/openhuman/config/schema/load.rs` — `SecretStore::new` / `is_encrypted` to encrypt/decrypt config fields on load. +- `src/openhuman/credentials/profiles.rs`, `credentials/ops.rs` — `SecretStore`, `is_available`, `get`/`set`/`delete` for per-profile credential storage. +- `src/openhuman/wallet/ops.rs` — `is_available`/`get`/`set` for the wallet mnemonic. +- `src/openhuman/devices/rpc.rs` — device secret handling. + +## Notes / gotchas + +- **Backend is frozen on first use.** Selection order: `OPENHUMAN_KEYRING_BACKEND` (`os`/`file`/`encrypted_file`) → `cfg(test)` → `file` → staging/prod → `encrypted_file`, dev → `file`. Once `BACKEND` is set it cannot change for the process lifetime. +- **`is_available()` is cached after the first probe.** The probe performs delete/set/get/delete round-trips on the `os` backend; running it per-call triggered repeated macOS permission dialogs and starved frequent pollers. Non-os backends short-circuit to `true`. A failed probe is logged at `warn` because it silently flips `use_keychain` off. +- **`force_backend_for_test` panics if `BACKEND` is already initialized** — it must run before any keyring call in the same process (dedicated test binary or very top of a test). +- **`migrate_from_file` never deletes the source unless the verified write succeeds**, so failures are retryable. +- **`encrypted_file` corrupt/undecryptable files are quarantined** (renamed `secrets.enc.corrupt.`) and treated as empty rather than crashing. +- **`SecretStore` master-key file is write-once**; on Windows it survives transient AV-scanner sharing violations via retry/backoff and attempts `icacls` ACL self-repair on permission errors. Decoded keys are cached so repeated decrypts (e.g. snapshot polls) hit memory. +- **Legacy formats:** `SecretStore` migrates `enc:` (XOR) → `enc2:` (ChaCha20-Poly1305) on decrypt; `EncryptedFileBackend` migrates plaintext `dev-keychain.json` → `secrets.enc` (renaming the legacy file `.json.migrated`). +- **Errors never carry secret values** — only namespaced keys; `diagnostic()` is safe to log and preserves the underlying `keyring::Error` variant/`OSStatus`. diff --git a/src/openhuman/learning/README.md b/src/openhuman/learning/README.md new file mode 100644 index 000000000..1b18244b5 --- /dev/null +++ b/src/openhuman/learning/README.md @@ -0,0 +1,126 @@ +# learning + +The agent **self-learning subsystem**. It observes completed turns, the memory substrate, and ingested transcripts; distils them into durable signals about the user (preferences, identity, vetoes, goals, tooling, communication style); and feeds those signals back into the agent's system prompt and `PROFILE.md`. The core abstraction is the **ambient personalization cache** (`user_profile_facets` table): a bounded, decaying set of scored `(class, key) → value` facets built by a stability detector from a stream of `LearningCandidate` evidence. The module also owns the one-shot LinkedIn enrichment pipeline and the transcript-to-memory ingestion pipeline. + +The subsystem is organised in phases (issue #566): **Phase 1** the candidate taxonomy + buffer, **Phase 2** producers that emit candidates, **Phase 3** the stability detector + cache + scheduler, **Phase 4** prompt injection and `PROFILE.md` rendering. + +## Responsibilities + +- Collect evidence (`LearningCandidate`) from many producers into a thread-safe global ring-buffer. +- Periodically (and on relevant events) drain the buffer, score every `(class, key)` pair by a recency-decayed stability formula, resolve value conflicts, enforce per-class budgets, assign lifecycle states, and persist the result to `user_profile_facets`. +- Run post-turn hooks: reflect on turns (`ReflectionHook`), track tool effectiveness (`ToolTrackerHook`), and extract explicit user preferences (`UserProfileHook`). +- Inject learned context, the user profile, and a memory-access instruction into the system prompt; render cache-derived managed blocks into `PROFILE.md`. +- Expose RPC controllers for inspecting / managing the cache and facets, plus running LinkedIn enrichment and saving a profile. +- Mine Gmail (via Composio) for a LinkedIn URL, scrape it via Apify, summarise it, and persist `PROFILE.md` + memory. +- Ingest completed session transcripts into durable conversational memory + reflections. + +## Key files + +| File | Role | +| --- | --- | +| `mod.rs` | Export-focused module root; phase docstrings + `pub use` re-exports. | +| `candidate.rs` | Phase 1 taxonomy: `FacetClass`, `CueFamily` (+ `weight()`), `EvidenceRef`, `LearningCandidate`, and the bounded FIFO `Buffer` with a `global()` singleton (cap 1024). | +| `cache.rs` | `FacetCache` — typed wrapper over `user_profile_facets`; class↔key helpers (`class_from_key`, `key_with_class`, `class_prefix`). Delegates to `memory_store::profile`. | +| `stability_detector.rs` | Phase 3 `StabilityDetector::rebuild` — the scoring/budget/state-assignment cycle; thresholds, half-lives, budgets, and the `stability()` formula. Publishes `CacheRebuilt`. | +| `scheduler.rs` | Periodic rebuild loop (`spawn_rebuild_loop`, default 30 min) + event-driven debounced trigger (`register_event_trigger`) subscribing to memory/tree-summarizer events. | +| `schemas.rs` | All RPC controller schemas + `handle_*` async handlers for the `learning.*` namespace. | +| `reflection.rs` | `ReflectionHook` post-turn hook: heuristic reflection-cue capture + LLM reflection, stores observations/patterns/preferences/reflections, emits Goal/Style candidates. | +| `tool_tracker.rs` | `ToolTrackerHook` post-turn hook + `ToolStats`; per-tool running success/failure/duration tallies in the `tool_effectiveness` memory category. | +| `user_profile.rs` | `UserProfileHook` post-turn hook; Aho-Corasick DFA over curated preference phrases, stores matches in the `user_profile` category. | +| `prompt_sections.rs` | `LearnedContextSection`, `UserProfileSection`, `MemoryAccessSection` (+ `MEMORY_ACCESS_INSTRUCTION`), and `load_learned_from_cache` (synchronous cache→prompt loader, cap 25). | +| `profile_md_renderer.rs` | `ProfileMdRenderer` — subscribes to `CacheRebuilt`, re-renders 5 managed `PROFILE.md` blocks (style/identity/tooling/vetoes/goals) from Active facets. | +| `linkedin_enrichment.rs` | Gmail→LinkedIn→Apify enrichment pipeline; `run_linkedin_enrichment`, `summarise_profile_with_llm`, `render_profile_markdown`, `scrape_linkedin_profile`. | +| `extract/` | Phase 2 producers (`mod.rs` + `signature.rs`, `heuristics.rs`, `summary_facets.rs`). | +| `extract/signature.rs` | Email-signature parser → Identity candidates; subscribes to `DocumentCanonicalized` (email). | +| `extract/heuristics.rs` | `LengthRatioDetector` / `EditWindowDetector` / `CorrectionRepeatDetector` → Style + Veto candidates via `record_turn`. | +| `extract/summary_facets.rs` | Parses the LLM summariser's structured JSON block; `route_facets_to_buffer` pushes validated candidates (requires `evidence_chunks`). | +| `transcript_ingest/` | Transcript→memory pipeline (`mod.rs`, `extract.rs`, `dedupe.rs`, `persist.rs`, `types.rs`). | +| `transcript_ingest/mod.rs` | `ingest_transcript_path` / `ingest_session_transcript`; extract→dedupe→persist into `conversation_memory` + `conversation_reflections` namespaces. | +| `*_tests.rs` | Sibling test suites (`cache`, `reflection`, `prompt_sections`, `linkedin_enrichment`). | + +## Public surface + +Re-exported from `mod.rs`: + +- **Candidate types**: `Buffer`, `CueFamily`, `EvidenceRef`, `FacetClass`, `LearningCandidate`. +- **Cache / detector**: `FacetCache`, `StabilityDetector`. +- **Hooks**: `ReflectionHook`, `ToolTrackerHook`, `UserProfileHook` (all impl `PostTurnHook`). +- **Prompt**: `LearnedContextSection`, `UserProfileSection`, `MemoryAccessSection`, `MEMORY_ACCESS_INSTRUCTION`, `load_learned_from_cache`. +- **Profile**: `ProfileMdRenderer`. +- **Schemas**: `all_learning_controller_schemas`, `all_learning_registered_controllers`, `learning_schemas`. + +Not re-exported but public within the module: `candidate::global()`, the `linkedin_enrichment::*` pipeline fns, `transcript_ingest::ingest_*`, `scheduler::{spawn_rebuild_loop, register_event_trigger, DEFAULT_REBUILD_INTERVAL}`, and `stability_detector` constants (`TAU_*`, `HALF_LIFE_*`, `BUDGET_*`, `stability`, `half_life`, `class_budget`). + +## RPC / controllers + +Namespace `learning` (wired into `src/core/all.rs`; 11 controllers). Methods: + +| Method | Purpose | +| --- | --- | +| `learning.linkedin_enrichment` | Run the Gmail→LinkedIn→Apify pipeline (optional `profile_url` to skip Gmail search). | +| `learning.save_profile` | Write markdown to `{workspace_dir}/PROFILE.md`; optional `summarize` runs it through the LLM compressor first. | +| `learning.rebuild_cache` | Manually trigger a `StabilityDetector` rebuild; returns added/evicted/kept/total_size. | +| `learning.cache_stats` | Cache totals + per-state and per-class breakdown. | +| `learning.list_facets` | List Active + Provisional facets, optional `class` filter. | +| `learning.get_facet` | Fetch one facet by `class` + `key` suffix. | +| `learning.update_facet` | Set a facet value and pin it (`user_state = Pinned`). | +| `learning.pin_facet` / `learning.unpin_facet` | Toggle `user_state` Pinned ↔ Auto. | +| `learning.forget_facet` | Mark `Dropped` + `user_state = Forgotten` (blocks re-promotion). | +| `learning.reset_cache` | Delete all `Auto` rows, preserve `Pinned`. | + +All handlers go through the memory client's `profile_conn()` and a `FacetCache`; `linkedin_enrichment` / `save_profile` load config via `config::rpc::load_config_with_timeout`. + +## Agent tools + +The module defines no `tools.rs`. However the `tool_effectiveness` stats it writes are surfaced by the cross-cutting `tool_stats` tool in `src/openhuman/tools/impl/system/tool_stats.rs`, and `memory_tools` references learning namespaces. + +## Events + +Uses the typed event bus (`src/core/event_bus/`): + +- **Publishes**: `DomainEvent::CacheRebuilt { added, evicted, kept, total_size, rebuilt_at }` after each rebuild (`stability_detector.rs`). +- **Subscribes**: + - `profile_md_renderer.rs` (`ProfileMdRenderer::subscribe`) → `CacheRebuilt` → re-render `PROFILE.md` blocks. + - `scheduler.rs` (`RebuildTriggerHandler`, domains `memory` / `tree_summarizer`) → `DocumentCanonicalized` (email/document) and `TreeSummarizerPropagated` → debounced (60 s) rebuild. + - `extract/signature.rs` → `DocumentCanonicalized` (email) → emit Identity candidates. + +These are subscriber registrations rather than a single `bus.rs`; subscriptions must keep their `SubscriptionHandle` alive (callers store them in statics). + +## Persistence + +- **`user_profile_facets`** (the ambient cache) — accessed via `FacetCache`, backed by `memory_store::profile` helpers (`profile_select_active/all`, `profile_get_by_key`, `profile_upsert_full`, `profile_set_user_state`, `profile_delete_*`). Stores `ProfileFacet { key, value, state, user_state, stability, confidence, evidence_count, evidence_refs, class, cue_families, first/last_seen_at, … }`. +- **KV memory namespaces** (via the `Memory` trait): `learning_observations`, `learning_patterns`, `learning_reflections`, `user_profile`, `tool_effectiveness`, plus transcript-ingest `conversation_memory` / `conversation_reflections`. LinkedIn enrichment also persists via `MemoryClient::store_skill_sync` and writes `{workspace_dir}/PROFILE.md`. +- **In-memory**: the global `candidate::Buffer` (transient evidence, not persisted) and per-session state in `extract/heuristics.rs`. + +## Dependencies + +- `crate::openhuman::memory_store::profile` — the `ProfileFacet` / `FacetState` / `UserState` types and the SQL helpers backing `FacetCache` (heaviest dependency). +- `crate::openhuman::memory` / `memory_store` — the `Memory` trait, `MemoryClient`, categories; all KV persistence and the global memory client used by RPC handlers. +- `crate::openhuman::agent::hooks` — `PostTurnHook` / `TurnContext` / `ToolCallRecord` implemented by the three hooks. +- `crate::openhuman::agent::harness::session::transcript` — `SessionTranscript` parsing for transcript ingestion. +- `crate::openhuman::inference::provider` / `inference::local` — LLM calls for reflection and profile summarisation. +- `crate::openhuman::config` — `Config` / `LearningConfig` / `ReflectionSource` feature flags and `config::rpc` loader. +- `crate::openhuman::context::prompt` — `PromptContext` / `PromptSection` / `LearnedContextData` for prompt injection. +- `crate::openhuman::composio` — `composio::client` (Gmail fetch for enrichment) and `composio::providers::profile_md` (`replace_managed_block` for `PROFILE.md`). +- `crate::openhuman::integrations` — `build_client` / `IntegrationClient` for the Apify scrape call. +- `crate::core::event_bus` — publish/subscribe + `EventHandler` for `CacheRebuilt` and trigger handlers. +- `crate::core::all` / `crate::rpc` — controller registry types (`RegisteredController`, `ControllerFuture`) and `RpcOutcome`. + +## Used by + +- `src/core/all.rs` — registers the `learning.*` controllers + schemas. +- `src/openhuman/agent/harness/session/{builder,turn}.rs` and `agent/memory_loader.rs` — wire the post-turn hooks, prompt sections, and learned-context loading into the agent loop. +- `src/openhuman/channels/runtime/startup.rs` — likely registers schedulers/subscribers at startup. +- `src/openhuman/memory_store/unified/profile.rs`, `memory_sync/composio/providers/profile.rs`, `memory_tools/{capture,mod}.rs`, `tools/impl/system/tool_stats.rs`, `tools/schemas.rs` — consume facet/learning types. + +## Notes / gotchas + +- **Pinned ⇒ stability ∞, Forgotten ⇒ stability 0.** User overrides hard-win over scoring in both `stability()` and `state_from_stability()`. `update_facet` implicitly pins. +- **Class is encoded in the key prefix** (`style/verbosity`, `goal/learn_rust`). `candidate.key` carries no prefix; the detector prepends `class_prefix`. Legacy rows without a recognised prefix are skipped by rebuild. +- **`emit_candidates_*` uses the global buffer length as a synthetic `episodic_id`** (a Phase-2 placeholder noted to be replaced by a real `episodic_log` row id). +- **Goal facets render value-only** (full sentence, no key prefix) in both prompt injection and `PROFILE.md`; other classes render `**key**: value`. +- **Prompt loaders are synchronous SQLite reads** — `load_learned_from_cache` runs in the sync prompt-build path and degrades to empty on error. Both the cache path and the legacy KV-namespace path are still active (KV slated for later removal). +- **Reflection has a local/cloud gate.** Local route requires `local_ai.usage.learning` flag; if off it falls back to a cloud provider or no-ops (empty string), and an empty response is a clean-skip sentinel that rolls back the throttle counter. Per-session reflections are throttled by `max_reflections_per_session`. +- **LinkedIn enrichment short-circuits** if `PROFILE.md` already exists, and the Composio-only Gmail-search stage is documented as currently erroring (Gmail-via-Composio removed) — callers should pass `preset_profile_url` obtained via the frontend's webview Gmail helper. +- **Transcript ingestion is heuristic-only by design** (no hard LLM dependency) so it can run as a background task without provider credentials. +- The `profile_md_renderer` deliberately does **not** touch the `connected-accounts` block — that's owned by the Composio provider path. diff --git a/src/openhuman/mcp_audit/README.md b/src/openhuman/mcp_audit/README.md new file mode 100644 index 000000000..9dc38ba03 --- /dev/null +++ b/src/openhuman/mcp_audit/README.md @@ -0,0 +1,62 @@ +# mcp_audit + +Persistent audit log for MCP write-tool calls. When the MCP server dispatches a write tool (e.g. `memory.store`, `tree.tag`) on behalf of an external MCP client (Claude Desktop, Cursor, …), this domain records the attempt — successful or failed — into the local workspace SQLite database and exposes a read-only query surface over those records. The audit table lives inside the existing memory-tree chunk database, reusing the same per-workspace persistence rather than opening a new file. + +## Responsibilities + +- Record every MCP write-tool attempt: timestamp, originating client, tool name, an arg summary, the resulting chunk id (if any), success flag, and error message. +- Truncate over-long error messages at a UTF-8 char boundary (cap 1024 bytes) before persisting. +- Query the audit log newest-first with optional filters: `since_ms`, exact `client_filter`, exact `tool_filter`, `success_only`, plus `limit`/`offset` paging (default limit 50, hard cap 500). +- Expose the query as an internal-only RPC controller (`mcp_audit.list`). + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/mcp_audit/mod.rs` | Export-only module surface; re-exports store fns, types, and the controller-schema pair. | +| `src/openhuman/mcp_audit/types.rs` | Serde types: `NewMcpWriteRecord` (insert), `McpWriteRecord` (row), `McpWriteListQuery` (filters/paging). | +| `src/openhuman/mcp_audit/store.rs` | Persistence: `record_write` (INSERT) and `list_writes` (filtered SELECT), arg/error serialization, limit/offset normalization, row mapping. Includes the test suite. | +| `src/openhuman/mcp_audit/schemas.rs` | Controller schema for `mcp_audit.list` + `handle_list` handler delegating to `store::list_writes`. Internal-controller registration. | + +## Public surface + +From `mod.rs`: +- Types: `McpWriteRecord`, `NewMcpWriteRecord`, `McpWriteListQuery`. +- Store fns: `record_write(&Config, NewMcpWriteRecord) -> Result`, `list_writes(&Config, &McpWriteListQuery) -> Result>`. +- Schema/controller exports: `all_mcp_audit_controller_schemas`, `all_mcp_audit_registered_controllers`, `all_mcp_audit_internal_controllers`, `mcp_audit_schemas`. + +## RPC / controllers + +- **`openhuman.mcp_audit_list`** (namespace `mcp_audit`, function `list`) — lists write-attempt audit records (successes and rejected/failed attempts) ordered by `timestamp_ms` descending. + - Inputs (all optional): `limit` (u64, default 50, max 500), `offset` (u64), `since_ms` (u64), `client_filter` (string, exact), `tool_filter` (string, exact), `success_only` (bool). + - Output: `records` — array of `McpWriteRecord`. +- **Internal-only.** Registered via `all_internal_controllers` and wired into `src/core/all.rs` through `all_mcp_audit_internal_controllers()`. Per `src/core/all_tests.rs`, the method is routable internally (`schema_for_rpc_method`) but **not** exposed publicly via `rpc_method_from_parts`. + +## Persistence + +- Table `mcp_writes` in the memory-tree chunk SQLite DB; schema and indexes (`idx_mcp_writes_timestamp`, `idx_mcp_writes_client`, `idx_mcp_writes_tool`) are created in `src/openhuman/memory_store/chunks/store.rs`, not here. +- Columns: `id`, `timestamp_ms`, `client_info`, `tool_name`, `args_summary` (JSON text), `resulting_chunk_id`, `success` (0/1), `error_message`. +- Connections are obtained via `chunk_store::with_connection(config, …)`; this module does not own a DB handle. + +## Dependencies + +- `crate::openhuman::config::Config` — workspace location used to resolve the DB. +- `crate::openhuman::config::rpc` (`load_config_with_timeout`) — loads config in the RPC handler. +- `crate::openhuman::memory_store::chunks::store` — provides `with_connection`; the audit table is co-located in the chunk DB. +- `crate::core::all` (`ControllerFuture`, `RegisteredController`) and `crate::core` (`ControllerSchema`, `FieldSchema`, `TypeSchema`) — controller/schema plumbing. +- External crates: `rusqlite`, `serde`/`serde_json`, `anyhow`. + +## Used by + +- `src/openhuman/mcp_server/write_dispatch.rs` — calls `mcp_audit::record_write` after each write attempt and `mcp_audit::list_writes` for its own surfaces/tests. +- `src/openhuman/mcp_server/tools_tests.rs` — reads back audit rows in tests. +- `src/core/all.rs` — registers the internal controller. +- `src/openhuman/tool_registry/ops.rs` — independently `COUNT`s `mcp_writes` rows (queries the shared table directly, not via this module's API). + +## Notes / gotchas + +- Filters `client_filter`/`tool_filter` are trimmed and dropped when empty (`normalized_filter`); matching is exact, not prefix/substring. +- `limit`/`offset` are `u64` in the query but converted to `i64` for SQLite (`u64_to_i64` errors on overflow); `limit` is clamped to `MAX_LIST_LIMIT` (500), default 50. +- Error messages longer than 1024 bytes are truncated at the nearest preceding UTF-8 char boundary, so multibyte error text stays valid. +- The table itself is created by the memory_store chunk DB migration — this module assumes it already exists and will fail if that migration hasn't run. +- Logging uses the grep-friendly `[mcp_audit]` prefix at `debug`/`trace`/`warn`. diff --git a/src/openhuman/mcp_client/README.md b/src/openhuman/mcp_client/README.md new file mode 100644 index 000000000..b6e6399ca --- /dev/null +++ b/src/openhuman/mcp_client/README.md @@ -0,0 +1,86 @@ +# mcp_client + +Reusable **MCP client transport library** plus a **read-only static server set** declared in the user's TOML config. This module knows how to *talk to* a remote MCP server over two transports — Streamable HTTP (with OAuth discovery + SSE per the MCP spec) and subprocess JSON-RPC over stdin/stdout — and exposes the servers the user pinned under `[[mcp_client.servers]]` as a queryable registry. It owns no RPC surface, no persistence, and no event-bus subscribers; it is a building block consumed by other domains. Its sibling `mcp_registry` reuses this module's `McpStdioClient` for all stdio transport of dynamically-installed Smithery servers. + +## Responsibilities + +- Implement the MCP **Streamable HTTP** client (`McpHttpClient`): `initialize` handshake + protocol-version negotiation, `tools/list`, `tools/call`, SSE event draining, session lifecycle (`Mcp-Session-Id`), `notifications/initialized`, and graceful `close_session` via HTTP DELETE. +- Implement the MCP **stdio** client (`McpStdioClient`): spawn a subprocess, speak newline-delimited JSON-RPC over stdin/stdout, and cache a single long-lived session. +- Handle **OAuth / authorization discovery** on HTTP transport: parse `WWW-Authenticate` challenges on 401, fetch protected-resource metadata and authorization-server metadata (OIDC `.well-known` + OAuth `.well-known`). +- Apply per-server **auth** (bearer / basic / custom header / query param) to outbound requests. +- **Reinitialize-and-retry once** on a 404 indicating an expired HTTP session. +- Mirror tool-schema parameters tagged `x-mcp-header` into `Mcp-Param-*` request headers. +- Render raw MCP `tools/call` results into the skills `ToolResult` shape. +- Build a static `McpServerRegistry` from `Config` (`[[mcp_client.servers]]` + a legacy `gitbooks` server), enforce per-server allow/deny tool lists, and pick HTTP vs stdio transport per entry. +- **Redact** endpoints in logs/errors (scheme + authority only; credentials → ``). + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/mcp_client/mod.rs` | Export-focused: module docstring + `mod`/`pub use` re-exports of the transport types and registry. No logic. | +| `src/openhuman/mcp_client/client.rs` | `McpHttpClient` (Streamable HTTP transport), shared MCP protocol types, SSE/`WWW-Authenticate` parsing, `render_tool_result`, `redact_endpoint`, `x-mcp-header` mirroring. Largest file; carries the inline test suite. | +| `src/openhuman/mcp_client/stdio.rs` | `McpStdioClient` — subprocess spawn + newline-delimited JSON-RPC over stdin/stdout, single cached `StdioSession`. | +| `src/openhuman/mcp_client/registry.rs` | `McpServerRegistry`, `McpServerDefinition`, `McpTransportClient` (Http/Stdio dispatch enum), `McpRegistrySource`; builds the static set from `Config`, applies allow/deny tool filtering. | + +## Public surface + +Re-exported from `mod.rs`: + +- **HTTP transport / protocol types** (`client.rs`): `McpHttpClient`, `McpInitializeResult`, `McpRemoteTool`, `McpServerToolResult`, `McpSseEvent`, `McpAuthChallenge`, `McpAuthorizationContext`, `ProtectedResourceMetadata`, `AuthorizationServerMetadata`, and the helper `redact_endpoint`. +- **Stdio transport** (`stdio.rs`): `McpStdioClient`. +- **Registry** (`registry.rs`): `McpServerRegistry`, `McpServerDefinition`, `McpTransportClient`, `McpRegistrySource`. + +Key entry points: + +- `McpHttpClient::new(endpoint, timeout_secs)` / `with_options(endpoint, timeout_secs, auth, identity)` → `initialize`, `list_tools`, `call_tool`, `discover_authorization`, `drain_events`, `close_session`, `initialize_snapshot`. +- `McpStdioClient::new(command, args, env, cwd, identity)` → `initialize`, `list_tools`, `call_tool`, `close_session`. +- `McpServerRegistry::from_config(&Config)` → `list`, `get`, `list_tools`, `call_tool`, `initialize`, `discover_authorization`, `is_empty`. +- `McpTransportClient` — enum unifying `Http`/`Stdio`, forwarding `initialize`/`list_tools`/`call_tool`/`discover_authorization` (stdio always returns `None` for authorization discovery). + +## Configuration + +Reads `config.mcp_client` (`McpClientConfig`): `enabled`, `servers` (`Vec`), and `client_identity` (`McpClientIdentityConfig` → client name/title/version sent in the `initialize` handshake). Each `McpServerConfig` supplies `name`, `endpoint`, `command`/`args`/`env`/`cwd` (stdio), `description`, `enabled`, `allowed_tools`, `disallowed_tools`, `timeout_secs`, and `auth` (`McpAuthConfig`). Also reads `config.gitbooks` (`enabled`, `endpoint`, `timeout_secs`) to seed a legacy `gitbooks` HTTP server when no explicit server of that name exists. HTTP clients apply the runtime proxy via `config::apply_runtime_proxy_to_builder(builder, "tool.mcp_client")`. + +Transport selection in `build_transport_client`: a non-empty `command` ⇒ stdio; otherwise HTTP against `endpoint`. + +## RPC / controllers + +None. This module exposes no JSON-RPC controllers or schemas of its own. RPC-facing MCP work lives in the sibling `mcp_registry` and in the generic bridge tools. + +## Agent tools + +None owned here. The generic bridge tools that drive this registry (`mcp_list_servers`, `mcp_list_tools`, `mcp_call_tool`) live in `src/openhuman/tools/impl/network/mcp.rs`; the bespoke `gitbooks` tool (`tools/impl/network/gitbooks.rs`) consumes `McpHttpClient` directly. + +## Events + +None. No `bus.rs`; publishes/subscribes to no `DomainEvent`s. + +## Persistence + +None. No `store.rs`. The static registry is rebuilt from `Config` in memory; HTTP/stdio session state (session id, negotiated protocol version, cached tool list, child process) lives only in the in-memory client instances. + +## Dependencies + +- `crate::openhuman::config` — `Config`, `McpClientConfig`, `McpServerConfig`, `McpAuthConfig`, `McpClientIdentityConfig`, and `apply_runtime_proxy_to_builder` (proxy-aware reqwest builder). Source of the static server set and per-server auth/identity. +- `crate::openhuman::skills::types::ToolResult` — the rendered result shape returned from `tools/call` (via `render_tool_result`). + +External crates: `reqwest` (HTTP), `tokio` (process + async IO + `sync::Mutex`), `parking_lot::Mutex` (HTTP session state), `serde`/`serde_json`, `base64`, `anyhow`, `tracing`. + +## Used by + +- `src/openhuman/mcp_registry/` (`mod.rs`, `connections.rs`, `setup_ops.rs`) — reuses `McpStdioClient` (and HTTP client) for all transport of dynamically-installed servers; carries no transport code of its own. +- `src/openhuman/tools/impl/network/mcp.rs` and `tools/ops.rs` — generic `mcp_*` bridge tools driving the static registry. +- `src/openhuman/tools/impl/network/gitbooks.rs` — uses `McpHttpClient` directly for the GitBook docs server. +- `src/openhuman/mcp_server/http.rs` — references this module. + +## Notes / gotchas + +- **Latest protocol version is `2025-11-25`**, sent on every `initialize`; the server's negotiated version must be one of the four `SUPPORTED_PROTOCOL_VERSIONS` or `initialize` fails. The constant is duplicated in both `client.rs` and `stdio.rs`. +- HTTP transport uses `redirect::Policy::none()` and a 10s connect timeout; the per-request timeout comes from the server's `timeout_secs`. +- A 404 on a request while a session id is held triggers exactly one reinitialize + retry (`allow_reinitialize` guards against loops); after that the error propagates. +- `Accept` is always `application/json, text/event-stream`; the client transparently parses an SSE-framed single response (`parse_sse_message`) or a plain JSON body based on `Content-Type`. +- Tool allow/deny enforcement is **fail-closed and pre-transport**: `is_tool_allowed` rejects empty names, anything in `disallowed_tools`, and (when `allowed_tools` is non-empty) anything not allow-listed; `registry.call_tool` blocks before any network/subprocess I/O. +- The legacy `gitbooks` server is only auto-seeded when `config.gitbooks.enabled` and no explicit server named `gitbooks` exists (explicit config wins, flipping `source` from `LegacyGitbooks` to `Config`). +- `redact_endpoint` returns `` for any URL containing userinfo (`@`) or a non-`http(s)` scheme — used in all log/error output so endpoints never leak credentials. +- `McpStdioClient` discards child stderr (`Stdio::null()`) and skips non-JSON stdout lines (logged at debug); the session is a single cached `StdioSession` guarded by a tokio `Mutex`. diff --git a/src/openhuman/mcp_registry/README.md b/src/openhuman/mcp_registry/README.md new file mode 100644 index 000000000..74f57e225 --- /dev/null +++ b/src/openhuman/mcp_registry/README.md @@ -0,0 +1,123 @@ +# mcp_registry + +The dynamic, user-facing side of MCP-client support. It browses upstream MCP server directories (Smithery.ai + the official modelcontextprotocol/registry), persists the user's chosen installs to SQLite, supervises their per-server connection lifecycle (stdio subprocess **or** HTTP-remote dial), and surfaces the connected servers' tools to agents via the unified tool registry. It also hosts a separate "setup agent" RPC surface (`mcp_setup`) that lets an LLM walk a non-technical user through search → secret collection → dry-run test → install-and-connect, with raw secret values kept out of the agent's context via opaque `secret://` refs. + +The transport primitives themselves (stdio + HTTP MCP clients) live in the sibling `mcp_client` module — this module carries no transport code of its own, only lifecycle, dispatch, persistence, and the registry HTTP adapters. (Naming note: the RPC namespace and SQLite db filename are still `mcp_clients` for backwards-compat; the Rust module path is `mcp_registry`.) + +## Responsibilities + +- Search multiple upstream MCP registries in parallel and merge results; route detail fetches back to the originating registry. +- Persist installed-server records (no env values) + per-server env values + a TTL'd registry response cache in SQLite. +- Establish, track, and tear down live MCP connections keyed by `server_id`, dispatching on each install's `Transport` (stdio subprocess vs HTTP-remote). +- Spawn all installed local servers at boot without blocking core startup. +- Expose the install/connect/status/tool-call lifecycle over JSON-RPC (`mcp_clients_*`). +- Run a setup-agent surface (`mcp_setup_*`) with out-of-band secret collection so credentials never enter the LLM context. +- Provide an AI `config_assist` flow that guides users through filling required env vars. +- Publish lifecycle `DomainEvent`s for observability. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/mcp_registry/mod.rs` | Module docstring + exports; re-exports controller schema/registry pair and key types. | +| `src/openhuman/mcp_registry/types.rs` | Domain types: `CommandKind`, `Transport` (Stdio / HttpRemote), `InstalledServer`, `McpTool`, `ServerStatus`, `ConnStatus`, Smithery DTOs, `ChatTurn`. | +| `src/openhuman/mcp_registry/store.rs` | SQLite persistence (`mcp_clients/mcp_clients.db`): server CRUD, env values, registry cache; additive `transport`/`deployment_url` column migration. | +| `src/openhuman/mcp_registry/registry.rs` | Multi-registry dispatch: `registry_search` (parallel fan-out + merge), `registry_get` (source-prefix routing or first-hit). | +| `src/openhuman/mcp_registry/registries/mod.rs` | `Registry` trait + `enabled_registries` / `registry_for_source`; `SOURCE_*` constants. | +| `src/openhuman/mcp_registry/registries/smithery.rs` | Smithery.ai adapter (`registry.smithery.ai`), 10-min SQLite cache, optional `SMITHERY_API_KEY` auth. | +| `src/openhuman/mcp_registry/registries/mcp_official.rs` | Official modelcontextprotocol/registry adapter; cursor→page mapping with a bounded sequential cursor walk; optional `MCP_OFFICIAL_REGISTRY_*` env overrides. | +| `src/openhuman/mcp_registry/connections.rs` | Global in-process connection registry; `connect`/`disconnect`/`call_tool`/`all_status`/`all_connected_tools`; dispatches stdio vs HTTP via `ActiveClient`. | +| `src/openhuman/mcp_registry/boot.rs` | `spawn_installed_servers` — boot-time connect of all installs; per-server failures logged, never fatal. | +| `src/openhuman/mcp_registry/ops.rs` | `mcp_clients_*` RPC handler implementations + `resolve_command` + `config_assist` inference call. | +| `src/openhuman/mcp_registry/setup.rs` | Opaque secret-ref machinery (`SecretRef`, mint/fulfill/await/resolve/consume/gc) for the setup agent. | +| `src/openhuman/mcp_registry/setup_ops.rs` | `mcp_setup_*` RPC handlers (search/get/request_secret/submit_secret/test_connection/install_and_connect) + connection `pick_connection`. | +| `src/openhuman/mcp_registry/schemas.rs` | Controller schemas + `handle_*` dispatch for both `mcp_clients` and `mcp_setup` namespaces. | +| `src/openhuman/mcp_registry/bus.rs` | `McpClientEventSubscriber` — logs lifecycle events; `init()` registers it. | + +## Public surface + +From `mod.rs`: +- `all_mcp_registry_controller_schemas` / `all_mcp_registry_registered_controllers` (`schemas::all_controller_schemas` / `all_registered_controllers`). +- `mcp_registry_schemas` (`schemas::schemas`). +- Types `ConnStatus`, `InstalledServer`, `McpTool`. + +`pub mod boot`, `bus`, `connections`, `setup`, `setup_ops`, `store`, `types` are public; `ops`, `registries`, `registry`, `schemas` are private (reached via the schema handlers). Notably `connections::all_connected_tools()` is consumed by `tool_registry`, and `boot::spawn_installed_servers` is called from core startup. + +## RPC / controllers + +Two namespaces. `all_controller_schemas()` returns 16 controllers (10 `mcp_clients` + 6 `mcp_setup`). + +**`mcp_clients`** (`ops.rs`): +- `registry_search` — search the registries. +- `registry_get` — full server detail (augmented with `required_env_keys`). +- `installed_list` — list installed servers (env values omitted). +- `install` — install from registry (stdio-only legacy path); stores env values, publishes `McpServerInstalled`. +- `uninstall` — disconnect + delete. +- `connect` / `disconnect` — bring a server's connection up/down. +- `status` — per-server connection summaries. +- `tool_call` — invoke a tool on a connected server. +- `config_assist` — AI helper for filling required env vars (calls inference at `{api_url}/openai/v1/chat/completions`, falls back to a stub when unconfigured). + +**`mcp_setup`** (`setup_ops.rs`): +- `search` / `get` — thin wrappers over `registry`. +- `request_secret` — mint a `secret://` ref, publish `McpSetupSecretRequested`, block up to 5 min for the UI to submit. +- `submit_secret` — UI-side fulfillment of a pending ref. +- `test_connection` — dry-run: dial a candidate (stdio scratch subprocess or HTTP-remote), list tools, tear down; nothing persisted. +- `install_and_connect` — commit: persist install + consume secret refs into `mcp_client_env`, then connect; returns `connected` or `installed_disconnected`. + +## Agent tools + +This module does not define `Tool` impls directly. The setup-agent tools live in `src/openhuman/tools/impl/network/mcp_setup.rs` as thin wrappers calling `mcp_registry::setup_ops`. The MCP browse/list/call tools (`McpListServersTool`, `McpListToolsTool`, etc.) are wired in `src/openhuman/tools/ops.rs` against the connected-tools surface, and `tool_registry` pulls live tools via `connections::all_connected_tools()`. + +## Events + +Publishes (via `publish_global`): +- `McpServerInstalled` (ops::install, setup_ops::install_and_connect) +- `McpServerConnected` (ops::connect) +- `McpServerDisconnected` (ops::disconnect) +- `McpClientToolExecuted` (ops::tool_call) +- `McpSetupSecretRequested` (setup_ops::request_secret) + +Subscribes: `bus::McpClientEventSubscriber` (domain `"mcp_client"`) — logs the four `McpServer*` / `McpClientToolExecuted` events for observability; no side effects. + +## Persistence + +SQLite at `{workspace_dir}/mcp_clients/mcp_clients.db` (`store.rs`), three tables: +- `mcp_servers` — installed-server metadata (no env values); `transport`/`deployment_url` added via idempotent additive migration (pre-migration rows default to `stdio`). +- `mcp_client_env` — per-server env key/value pairs; values never serialized into any response and never logged. `ON DELETE CASCADE` from `mcp_servers`. +- `mcp_registry_cache` — registry HTTP response bodies with a 10-minute TTL. + +Setup-agent secrets (`setup.rs`) are held in a **process-local in-memory map** (not SQLite) with a 5-min request timeout and 15-min idle GC; they only persist if committed via `consume_refs` → `mcp_client_env` during `install_and_connect`. + +The in-process connection registry (`connections.rs`) is a `OnceLock>>`, ephemeral per process. + +## Dependencies + +- `crate::openhuman::config::Config` — workspace dir, `mcp_client.client_identity`, inference api_url/api_key/default_model. +- `crate::openhuman::config::rpc` (`load_config_with_timeout`) — config load inside RPC handlers. +- `crate::openhuman::mcp_client` (`McpStdioClient`, `McpHttpClient`, `McpRemoteTool`, `McpServerToolResult`) — actual transport clients used by `connections.rs` and `setup_ops.rs`. +- `crate::core::event_bus` (`publish_global`, `DomainEvent`, `EventHandler`, `subscribe_global`) — lifecycle events + subscriber. +- `crate::core::all` (`ControllerFuture`, `RegisteredController`) and `crate::core::{ControllerSchema, FieldSchema, TypeSchema}` — controller wiring. +- `crate::rpc::RpcOutcome` — handler return contract. +- External crates: `rusqlite` (store), `reqwest` + `futures` (registry HTTP / fan-out), `tokio::sync` (RwLock/Mutex/oneshot), `uuid`, `async_trait`, `parking_lot` (official-registry cursor cache). + +## Used by + +- `src/openhuman/mod.rs` — declares `pub mod mcp_registry`. +- `src/core/all.rs` — registers `all_mcp_registry_registered_controllers()` + `all_mcp_registry_controller_schemas()`. +- `src/core/jsonrpc.rs` — calls `boot::spawn_installed_servers` during startup. +- `src/openhuman/tool_registry/ops.rs` — uses `connections` to surface MCP tools to agents. +- `src/openhuman/tools/ops.rs` + `src/openhuman/tools/impl/network/mcp_setup.rs` — agent tool wrappers. +- `src/openhuman/about_app/catalog.rs` — capability entry `channels.mcp_registry_browse`. +- `src/bin/test_mcp_stub.rs` + `tests/mcp_registry_e2e.rs` — E2E stub server. + +## Notes / gotchas + +- **`mcp_clients` vs `mcp_registry`**: the RPC namespace and db filename keep the old `mcp_clients` name for backwards-compat; only the Rust module path is `mcp_registry`. +- **Two install paths**: legacy `mcp_clients_install` is **stdio-only** (filters for `stdio` connections). HTTP-remote installs go exclusively through the newer `mcp_setup_install_and_connect`, which routes via `pick_connection` (preference: published stdio → any stdio → published http_remote → any http_remote). +- **Smithery DTO naming**: the canonical result shapes are named `Smithery*` for wire compat; non-Smithery registries adapt into the same shapes and tag the `source` field. +- **HTTP-remote env**: env vars for HTTP-remote installs (typically OAuth tokens) are picked up by `McpHttpClient`'s own auth config, not injected at dial time by this module. +- **Secret safety**: raw secret values flow only through `submit_secret` and the just-in-time resolve in `test_connection`/`install_and_connect`; never echoed in responses or logged. `consume_refs` only removes refs after values are persisted. +- **Boot is best-effort**: a misbehaving server logs and is skipped; it never blocks core startup. +- **`all_connected_tools` caveat**: it currently returns `server_id` in the `qualified_name` slot — callers needing the real qualified name must re-join against `store::list_servers`. +- **Official registry cursor walk**: deep-page cache misses walk pages sequentially up to `MAX_CURSOR_WALK_PAGES` (50) before bailing to avoid request amplification. diff --git a/src/openhuman/mcp_server/README.md b/src/openhuman/mcp_server/README.md new file mode 100644 index 000000000..345f999a6 --- /dev/null +++ b/src/openhuman/mcp_server/README.md @@ -0,0 +1,99 @@ +# mcp_server + +Opt-in **Model Context Protocol (MCP) server** that exposes a curated, security-gated slice of OpenHuman's tool surface (memory-tree reads/writes, core/agent introspection, subagent execution, SearXNG search) and bundled prompt assets to external MCP clients (Claude Desktop, Cursor, Windsurf, …). Started via `openhuman-core mcp` — stdio transport by default, or `--transport http` for Streamable HTTP + SSE on a local bind address. It is a JSON-RPC dispatcher, not a registered RPC domain: it has no `schemas.rs`/controllers and is wired only through `src/core/cli.rs`, translating each MCP `tools/call` into an existing registered core RPC method. + +## Responsibilities + +- Implement the MCP JSON-RPC server lifecycle: `initialize`, `ping`, `tools/list`, `tools/call`, `resources/list`, `resources/templates/list`, `resources/read`, plus notifications (`notifications/initialized`, `notifications/cancelled`). +- Advertise a fixed catalog of MCP tools (`tool_specs`) with input JSON-schemas and MCP `ToolAnnotations` (`readOnlyHint`/`destructiveHint`/`idempotentHint`/`openWorldHint`). +- Validate/normalize tool arguments at the MCP layer (explicit rejection over silent clamping), map them to registered core RPC params, and dispatch via `all::try_invoke_registered_rpc`. +- Enforce `SecurityPolicy` per call: read tools require `ToolOperation::Read`; `agent.run_subagent` and the three write tools require `ToolOperation::Act`. +- Run write tools (`memory.store`, `memory.note`, `tree.tag`) through a dedicated write-dispatch + audit pipeline that records every attempt (success and rejection) to the MCP write-audit log. +- Serve bundled prompt assets (`IDENTITY.md`, `SOUL.md`, `USER.md`, and each built-in subagent's `prompt.md`) as static MCP resources under the `openhuman://prompts/...` URI scheme. +- Provide two transports — newline-delimited JSON-RPC over stdio, and Axum-based Streamable HTTP + SSE with session-id + protocol-version handshakes and optional bearer auth. +- Capture client provenance from `initialize` `clientInfo.name` into a per-session `source_type` (e.g. `mcp:claude-desktop`) used for audit attribution. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/mcp_server/mod.rs` | Module docstring + private submodule decls; re-exports `run_http`/`HttpServerConfig`, `run_stdio_from_cli`, `tool_specs`/`McpToolSpec`. | +| `src/openhuman/mcp_server/protocol.rs` | JSON-RPC 2.0 dispatch core: parses lines/values (single + batch), routes `initialize`/`ping`/`tools/*`/`resources/*`, builds success/error envelopes, negotiates protocol version. | +| `src/openhuman/mcp_server/tools.rs` | Tool catalog (`tool_specs`, `base_tool_specs`, `searxng_tool_spec`), input schemas, argument validation, `call_tool` dispatch, read/act policy enforcement, subagent execution, slug/key helpers. | +| `src/openhuman/mcp_server/write_dispatch.rs` | Write/audit pipeline for `memory.store`/`memory.note`/`tree.tag`: config load, act-policy enforcement, RPC dispatch to `openhuman.memory_doc_put`, audit-record write (success/rejection), PII-redacting arg summaries. | +| `src/openhuman/mcp_server/resources.rs` | Static `RESOURCE_CATALOG` of compile-time-embedded (`include_str!`) prompt markdown; `resources/list`, `resources/templates/list` (always empty), `resources/read`. Test cross-checks catalog vs `agent::agents::BUILTINS`. | +| `src/openhuman/mcp_server/session.rs` | `McpSession` — captures + normalizes client name from `initialize` into a `source_type`; first observation locks the value. | +| `src/openhuman/mcp_server/http.rs` | Axum Streamable HTTP + SSE transport (`run_http`, `HttpServerConfig`): POST/GET/DELETE on `/`, session map, protocol-version checks, optional `Authorization: Bearer`, SSE keep-alive, session-id redaction. | +| `src/openhuman/mcp_server/stdio.rs` | CLI entry `run_stdio_from_cli` (arg parse: `--transport`/`--host`/`--port`/`--auth-token`/`-v`/`--help`), logging init, stdio read/write loop (`run_stdio`). | +| `src/openhuman/mcp_server/tools_tests.rs` | Sibling `#[cfg(test)]` suite for `tools.rs` (via `#[path]`). | + +## Public surface + +Re-exported from `mod.rs`: + +- `run_stdio_from_cli(args: &[String]) -> Result<()>` — CLI entry point; builds its own tokio runtime and selects stdio vs HTTP transport. +- `run_http(config: HttpServerConfig) -> Result<()>` and `HttpServerConfig { bind_addr, auth_token }` — HTTP/SSE server. +- `tool_specs() -> Vec` and `McpToolSpec { name, title, description, rpc_method, input_schema, annotations }` — the advertised tool catalog. + +## RPC / controllers + +This module exposes **no** registered core RPC methods (no `schemas.rs`, no controllers, no `openhuman.mcp_server_*` namespace). It is the **client/server-of-MCP**, not an RPC domain. Instead it *consumes* existing registered RPC methods, mapping each MCP tool to one via `all::try_invoke_registered_rpc` after validating against `all::schema_for_rpc_method`: + +| MCP tool | Mapped core RPC method | +| --- | --- | +| `memory.search` | `openhuman.memory_tree_search` | +| `memory.recall` | `openhuman.memory_tree_recall` | +| `tree.read_chunk` | `openhuman.memory_tree_get_chunk` | +| `tree.browse` | `openhuman.memory_tree_list_chunks` | +| `tree.top_entities` | `openhuman.memory_tree_top_entities` | +| `tree.list_sources` | `openhuman.memory_tree_list_sources` | +| `memory.store` / `memory.note` / `tree.tag` | `openhuman.memory_doc_put` | +| `searxng_search` | `openhuman.tools_searxng_search` | +| `core.list_tools` / `core.tool_instructions` / `agent.list_subagents` / `agent.run_subagent` | (no RPC mapping — handled in-process via `Agent` / `AgentDefinitionRegistry`) | + +## Agent tools + +It does **not** own any agent tools in the `tools.rs`/`src/openhuman/tools` sense. The "tools" here are **MCP-protocol tools** advertised to external clients: + +- Read-only (`ToolOperation::Read`): `core.list_tools`, `core.tool_instructions`, `agent.list_subagents`, `memory.search`, `memory.recall`, `tree.read_chunk`, `tree.browse`, `tree.top_entities`, `tree.list_sources`, `searxng_search` (config-gated on `searxng.enabled`). +- Act-policy (`ToolOperation::Act`): `agent.run_subagent` (annotated destructive/open-world; rejects `integrations_agent`), and the write tools `memory.store`, `memory.note`, `tree.tag` (annotated destructive/idempotent, local-only). + +Argument bounds enforced in-layer: `k`/limits capped at `MAX_LIMIT` (50), default 10; `tree.tag` capped at 50 tags / 128 bytes per tag; SearXNG `max_results` capped at `SEARXNG_MAX_RESULTS`. Write tools derive deterministic upsert keys (`mcp-store-`, `mcp-note-`, `mcp-tag-`). + +## Events + +None. This module publishes/subscribes no `DomainEvent`s and has no `bus.rs`. + +## Persistence + +No `store.rs`. The only durable side effect is the **MCP write-audit log**, written via `crate::openhuman::mcp_audit::record_write` (separate `mcp_audit` domain) for every write-tool attempt — including pre-dispatch rejections. Audit rows store a PII-redacted `args_summary` (titles truncated to 128 chars; `content`/`note_text` bodies omitted, only lengths/counts kept), `client_info`, success flag, and resulting `document_id`. Audit inserts run off the hot path via `spawn_blocking` (or a thread fallback). HTTP session records (`Mcp-Session-Id` → negotiated protocol version) are kept in an in-memory map only. + +## Dependencies + +- `crate::core::all` — `try_invoke_registered_rpc`, `schema_for_rpc_method`, `validate_params`: dispatch MCP tool calls into the registered core RPC layer and validate params against controller schemas. +- `crate::core::logging` (`CliLogDefault`, `init_for_cli_run`) — install the stderr tracing subscriber for the MCP subprocess. +- `crate::openhuman::config` (`Config`, `rpc::load_config_with_timeout`, `McpAuthConfig`/`McpClientIdentityConfig` in tests) — load config for policy/searxng gating and per-call config. +- `crate::openhuman::security` (`SecurityPolicy`, `ToolOperation`) — enforce read/act autonomy policy per tool call. +- `crate::openhuman::agent` (`Agent`, `agents::BUILTINS`, `harness::AgentDefinitionRegistry`) — build the orchestrator agent for `core.list_tools`/`core.tool_instructions`, list/run subagents, and cross-check the resource catalog. +- `crate::openhuman::inference::provider::traits::build_tool_instructions_text` — render the markdown tool-use instructions block for `core.tool_instructions`. +- `crate::openhuman::tools` (`SEARXNG_MAX_RESULTS`, `normalize_categories`) — SearXNG bounds + category normalization for `searxng_search`. +- `crate::openhuman::mcp_audit` (`record_write`, `NewMcpWriteRecord`, list/query helpers in tests) — durable write-audit log. +- `crate::openhuman::mcp_client::McpHttpClient` — round-trip test harness for the HTTP transport (test-only). +- External crates: `axum`/`tokio`/`tokio-stream` (HTTP+SSE), `serde_json`, `uuid`, `sha2`/`hex` (session-id redaction, slug fallback hash), `chrono` (audit timestamps). + +## Used by + +- `src/core/cli.rs` — dispatches `mcp` / `mcp-server` subcommands to `run_stdio_from_cli`; the only production entry point. (`src/core/legacy_aliases.rs` references the command surface; `about_app/catalog.rs` lists it in the capability catalog.) +- Other `mcp_*` modules (`mcp_registry`, `mcp_client`, `mcp_audit`, `tool_registry`) are **siblings** in the broader MCP feature set, not consumers of this server's code paths (except the test-only `McpHttpClient` round-trip). + +## Notes / gotchas + +- **Not a controller-registry domain.** Don't look for `schemas.rs`/`all_controller_schemas` — exposure is via the CLI only, and tool calls re-enter the registry through `core::all`. +- **`ToolCallError` variant → JSON-RPC code is deliberate:** `InvalidParams` → `-32602` (client-actionable, including policy denials so the reason text surfaces), `Internal` → `-32603` (config load / platform failures). Policy denials are intentionally `InvalidParams`, not `Internal`. +- **Explicit rejection over silent clamping** throughout: over-cap `k`, oversize/over-count tags, blank required strings, and unexpected arguments all error rather than being trimmed/dropped. +- **Write audit is fire-and-forget but mandatory:** rejections are audited even before config is loaded (`audit_write_rejection_without_config`), and `dispatch_write_tool` returns `Ok(tool_error(...))` (not `Err`) on RPC-handler failure so the client gets an MCP `isError` result while the failure is still recorded. +- **Protocol negotiation:** supports `2024-11-05`, `2025-03-26`, `2025-06-18`, and `2025-11-25` (`LATEST_PROTOCOL_VERSION`); unknown requested versions fall back to latest. HTTP enforces an exact session protocol-version match on subsequent requests. +- **HTTP security:** bearer auth is optional (`--auth-token`); session ids are SHA-256-redacted in logs; default bind is `127.0.0.1:9300`. +- **Resource catalog parity is CI-enforced:** `resources.rs` content is `include_str!`-embedded at compile time and the `catalog_mirrors_builtins` test fails if a built-in subagent lacks a matching `openhuman://prompts/agents/` entry. +- **`agent.run_subagent` limits:** rejects `integrations_agent` (toolkit binding not yet supported over first-level MCP) and runs a fresh single-turn agent session tagged with an `mcp::` event context. +- **stdio logging defaults to `warn`** on stderr (so failures surface in client UIs); `--verbose` → `debug`; a user-set `RUST_LOG` always wins. Stdout is reserved for protocol messages only. diff --git a/src/openhuman/meet/README.md b/src/openhuman/meet/README.md new file mode 100644 index 000000000..ea52fc768 --- /dev/null +++ b/src/openhuman/meet/README.md @@ -0,0 +1,59 @@ +# meet + +Google Meet integration domain. Lets a user ask the agent to join a Google Meet call as an **anonymous guest**. The core's responsibility is deliberately narrow: validate that the supplied URL is a Google Meet meeting URL, validate/trim the guest display name, and mint a `request_id` that the desktop (Tauri) shell uses to label the per-call CEF webview window and its data directory. All platform-specific work — opening the webview, driving Meet's join page over CDP, surfacing a virtual camera — lives in the Tauri shell (`app/src-tauri/src/...`), keeping this domain platform-agnostic. + +## Responsibilities + +- Validate that `meet_url` is `https://meet.google.com/` (three lowercase-letter groups, each ≥3 chars, separated by `-`) or `https://meet.google.com/lookup/` (single non-empty segment). Reject any other scheme, host, or path. +- Trim and validate `display_name`: non-empty, ≤64 chars, no control characters. +- Mint a stable UUID `request_id` for each accepted join attempt. +- Return a normalized echo (`ok`, `request_id`, normalized `meet_url`, `display_name`) for the shell to act on. +- Keep the meeting code (a credential) out of logs — only host + display-name length are logged. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/meet/mod.rs` | Export-focused. Module docstring + re-exports `all_meet_controller_schemas` / `all_meet_registered_controllers` and `types::*`. | +| `src/openhuman/meet/types.rs` | Serde request/response types: `MeetJoinCallRequest`, `MeetJoinCallResponse`. | +| `src/openhuman/meet/ops.rs` | Pure validation helpers: `validate_meet_url`, `validate_display_name` (plus private `is_meet_code` / `is_lookup_path`). Inline `tests` module. | +| `src/openhuman/meet/rpc.rs` | Async JSON-RPC handler `handle_join_call` — parses params, validates, mints `request_id`, returns CLI-compatible JSON via `RpcOutcome`. | +| `src/openhuman/meet/schemas.rs` | Controller schema definitions, `MEET_CONTROLLER_DEFS` table, `all_controller_schemas` / `all_registered_controllers` / `schemas`, and the `handle_join_call_wrap` handler wrapper. | +| `src/openhuman/meet/ops_tests.rs` | Sibling test file wired into `ops.rs` via `#[path = "ops_tests.rs"]`. | + +## Public surface + +- `types::MeetJoinCallRequest` — `{ meet_url: String, display_name: String }`. +- `types::MeetJoinCallResponse` — `{ ok: bool, request_id: String, meet_url: String, display_name: String }`. +- `all_meet_controller_schemas()` / `all_meet_registered_controllers()` (re-exported from `schemas`). +- `ops::validate_meet_url`, `ops::validate_display_name` (pure helpers, reachable for tests/CLI without the shell). + +## RPC / controllers + +| Method | Inputs | Outputs | +| --- | --- | --- | +| `openhuman.meet_join_call` (namespace `meet`, function `join_call`) | `meet_url` (string, required), `display_name` (string, required) | `ok` (bool), `request_id` (string UUID), `meet_url` (normalized string), `display_name` (trimmed string) | + +Validates the URL and mints `request_id`; returns immediately. The actual webview lifecycle is the Tauri shell's job, keyed by `request_id`. Registered into the global controller registry via `src/core/all.rs` — no branches in `cli.rs` / `jsonrpc.rs` / `dispatch.rs`. + +## Dependencies + +- `crate::rpc::RpcOutcome` (`rpc.rs`) — wraps the handler result into CLI-compatible JSON. +- `crate::core::all::{ControllerFuture, RegisteredController}` (`schemas.rs`) — controller registry types for the boxed-future handler. +- `crate::core::{ControllerSchema, FieldSchema, TypeSchema}` (`schemas.rs`) — shared controller schema contract. +- External crates: `serde`/`serde_json` (types + handler), `url` (URL parsing/validation), `uuid` (request_id), `tracing` (diagnostics). + +No dependencies on other `openhuman::*` domains. + +## Used by + +- `src/core/all.rs` — registers the controllers (`all_meet_registered_controllers`, line 255) and schemas (`all_meet_controller_schemas`, line 370) into the global registry. +- The desktop shell (`app/src-tauri/src/...`) consumes the minted `request_id` to open/label the per-call CEF webview (out of this crate). + +> Note: `config::MeetConfig` (`src/openhuman/config/schema/`) is a separate config submodule named `meet`, unrelated to this domain. + +## Notes / gotchas + +- **Not a generic "open any URL in CEF" entrypoint** — host is hard-pinned to `meet.google.com`, scheme to `https`, and the path must match a Meet code or a single-segment `lookup/`. Nested `lookup/` paths and look-alike hosts (e.g. `meet.google.evil.com`) are rejected. +- The meeting code is treated as a **credential**; it is intentionally kept out of logs (only host + display-name char count are emitted). +- Core does no webview/CDP work and persists no state — there is no `store.rs`, no `bus.rs`, and no agent tools (`tools.rs`). The domain is stateless validation + an ID mint. diff --git a/src/openhuman/meet_agent/README.md b/src/openhuman/meet_agent/README.md new file mode 100644 index 000000000..fe7ac554c --- /dev/null +++ b/src/openhuman/meet_agent/README.md @@ -0,0 +1,92 @@ +# meet_agent + +The **listening + speaking loop for a live Google Meet call**. Where `meet/` is single-shot pure validation (mint a `request_id`, hand it to the shell to open a window), `meet_agent/` is the long-lived per-call session that actually makes the bot listen and talk. While a call is open, the Tauri shell streams captured audio (PCM frames) and scraped caption lines into the core; the core runs VAD-segmented STT (or wake-word matching on captions), routes the utterance through the user's full orchestrator agent, synthesizes TTS, and streams PCM back out for the shell's virtual-mic pump. All state is keyed by the same `request_id` `meet/` mints. + +## Responsibilities + +- Maintain a process-wide registry of per-call sessions keyed by `request_id` (open / poll / close, idempotent restart). +- Accept inbound PCM16LE @ 16 kHz mono frames and run a crude RMS energy VAD with hangover to detect end-of-utterance. +- Accept live caption lines from Meet's captions DOM and run a wake-word state machine (`"hey openhuman"` and brand-mangle variants). +- Enforce a **privacy gate**: only the configured call owner (or owner-granted allowlist members) may wake the bot for tool-backed turns. Non-owners get a friendly greeting or a polite refusal; the bot never wakes on its own caption echo. +- Orchestrate a turn: STT (audio path) or caption text → full orchestrator Agent (with tools/memory/integrations) → TTS → enqueue outbound PCM. Falls back to a bare toolless chat-completions path / deterministic stubs when the backend token is missing. +- Handle barge-in (cancel/flush outbound on a new turn), pre-roll filler acks for slow integration paths, dedup + cooldown + min-turn-gap rate limiting against Meet's caption re-emits. +- Persist a recent-calls record (JSONL) on `stop_session` and expose a `list_calls` history read. +- Strip markdown / chain-of-thought / reasoning preamble from LLM output so TTS reads a clean spoken sentence; hard-cap reply length. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/meet_agent/mod.rs` | Module docstring + exports; re-exports controller schema pair and session registry types. Export-focused only. | +| `src/openhuman/meet_agent/types.rs` | Serde request/response types for all RPC endpoints + `SessionEvent` / `SessionEventKind` transcript record. Audio crosses the boundary as base64 PCM16LE. | +| `src/openhuman/meet_agent/ops.rs` | Pure, tokio-free helpers: sample-rate validation, `request_id` sanitization, `frame_rms`, and the stateful `Vad` (energy + hangover). | +| `src/openhuman/meet_agent/session.rs` | `MeetAgentSession` (ring buffers, VAD state, transcript log, counters, wake-word state machine, owner/allowlist privacy gate) + `MeetAgentSessionRegistry` + the `SESSION_REGISTRY` `OnceLock` singleton. | +| `src/openhuman/meet_agent/brain.rs` | Turn orchestration: STT → LLM → TTS. Owns the per-meet cached orchestrator `Agent`, the audio `run_turn` and caption `run_caption_turn`, soft-deny / grant turns, system prompts, and speech-cleanup helpers. | +| `src/openhuman/meet_agent/rpc.rs` | JSON-RPC handlers (deserialize-validate-dispatch only); spawns brain turns, persists the call record on stop. | +| `src/openhuman/meet_agent/schemas.rs` | Controller schema definitions + `handle_*` futures delegating to `rpc.rs`; the registry `all_controller_schemas` / `all_registered_controllers`. | +| `src/openhuman/meet_agent/store.rs` | `MeetCallRecord` type + append-only JSONL persistence (`append_record`, `read_recent`) under the workspace data dir. | +| `src/openhuman/meet_agent/wav.rs` | `pack_pcm16le_mono_wav` — minimal RIFF/WAVE header wrapper so raw PCM batches can be posted to the cloud Whisper endpoint as `audio/wav`. | +| `src/openhuman/meet_agent/ops_tests.rs` | Sibling test suite for `ops.rs` (`#[path]`-included). | + +## Public surface + +- `MeetAgentSession`, `MeetAgentSessionRegistry`, `SESSION_REGISTRY` (re-exported from `session`). +- `all_meet_agent_controller_schemas` / `all_meet_agent_registered_controllers` (re-exported from `schemas`). +- All `types::*` (request/response structs, `SessionEvent`, `SessionEventKind`). +- Module-internal-but-`pub`: `session::registry()`, `session::CaptionOutcome`; `brain::run_turn`, `run_caption_turn`, `run_soft_deny_turn`, `run_grant_turn`, `forget_session_agent`; `store::{MeetCallRecord, append_record, read_recent, MAX_RECENT_CALLS, meet_calls_jsonl_path}`; `ops::{Vad, VadEvent, validate_sample_rate, sanitize_request_id, frame_rms, REQUIRED_SAMPLE_RATE}`; `wav::pack_pcm16le_mono_wav`. + +## RPC / controllers + +Namespace `meet_agent` (dispatched as `openhuman.meet_agent_`), wired into `src/core/all.rs`: + +| Function | Purpose | +| --- | --- | +| `start_session` | Open a session for a `request_id`; installs owner/bot identities + meet URL. Inputs: `request_id` (required), `sample_rate_hz`, `owner_display_name`, `bot_display_name`, `meet_url`. | +| `push_listen_pcm` | Shell pushes captured PCM frames; spawns a brain turn on VAD end-of-utterance. Returns `turn_started`. | +| `push_caption` | Shell pushes a scraped caption line; runs the wake-word/privacy gate and spawns a normal, soft-deny, or no-op turn. Returns `turn_started`. | +| `poll_speech` | Pull synthesized outbound PCM (`pcm_base64`, `utterance_done`, `flush_pending` for barge-in). | +| `stop_session` | Close session, drop the cached Agent, persist a `MeetCallRecord`; returns `listened_seconds` / `spoken_seconds` / `turn_count`. | +| `list_calls` | Return recent completed calls (newest first) from the JSONL log. Optional `limit` (default 50, capped at `MAX_RECENT_CALLS` = 200). | + +## Agent tools + +None. This domain does not own any agent tools (no `tools.rs`). It is a *consumer* of the orchestrator agent — it builds and drives an `agent::harness::session::Agent` per meet, inheriting that agent's tool surface. + +## Events + +None via the event bus. There is no `bus.rs` and no `DomainEvent` publish/subscribe. The orchestrator turn sets a per-meet event context (`agent.set_event_context("meet_{request_id}", "meet_agent")`) for harness observability scoping, but `meet_agent` itself neither publishes nor subscribes to `DomainEvent`s. + +## Persistence + +- **Recent calls log** (`store.rs`): append-only JSONL at `{workspace_dir}/meet_agent/calls.jsonl`, one `MeetCallRecord` per closed call. `read_recent` reads, drops malformed lines, sorts newest-first by `started_at_ms`, and caps at `MAX_RECENT_CALLS`. Chosen over sqlite for low-cardinality, write-rarely data. +- **In-memory session state** (`session.rs`): the `SESSION_REGISTRY` `OnceLock>` holds live `MeetAgentSession`s (ring buffers, VAD, transcript events, wake state, allowlist) for the duration of a call. Dropped on `stop_session`; not persisted. +- **Per-meet Agent cache** (`brain.rs`): `AGENT_CACHE` `OnceLock>>>>` reuses one orchestrator Agent across a call's turns (accumulating history); dropped by `forget_session_agent` on stop. + +## Dependencies + +- `crate::openhuman::agent::harness::session::Agent` — builds/drives the full orchestrator (tools, memory tree, MCP, skills) per meet turn (`brain.rs`). +- `crate::openhuman::voice::cloud_transcribe` — cloud STT (`transcribe_cloud`) for the audio listen path. +- `crate::openhuman::voice::reply_speech` — cloud TTS (`synthesize_reply`, `pcm_16000` output) for spoken replies. +- `crate::openhuman::config` — `Config` / `config::ops::load_config_with_timeout` for backend URL, token, and workspace dir resolution. +- `crate::api::{BackendOAuthClient, config::effective_backend_api_url, jwt::get_session_token}` — the bare chat-completions fallback path in `brain.rs`. +- `crate::core::all::{ControllerFuture, RegisteredController}` and `crate::core::{ControllerSchema, FieldSchema, TypeSchema}` — controller registry contract (`schemas.rs`). +- `crate::rpc::RpcOutcome` — RPC response envelope (`rpc.rs`). +- External crates: `base64`, `serde`/`serde_json`, `tokio`, `chrono`, `reqwest`. + +## Used by + +- `src/core/all.rs` — registers the controllers/schemas and routes the `"meet_agent"` namespace. +- `src/openhuman/mod.rs` — declares the domain module. +- `src/openhuman/about_app/catalog.rs` — capability catalog entry. +- `src/openhuman/desktop_companion/pipeline.rs` — reuses `meet_agent::wav::pack_pcm16le_mono_wav` and is modelled on `meet_agent::brain`'s STT/LLM/TTS pipeline. + +## Notes / gotchas + +- **Single sample rate.** Only 16 kHz is accepted (`REQUIRED_SAMPLE_RATE`); `validate_sample_rate` hard-rejects anything else. The shell must resample CEF's native rate down before pushing. WAV packing and duration math assume this constant. +- **Privacy gate fails closed.** With no `owner_display_name` configured, *no* wake fires — a misconfigured launch can never expose the user's tool surface to a remote participant. Owner match is light-normalised (lowercase, trim, single trailing parenthetical strip like `(host)`/`(you)`). Bot-self captions are dropped first so the bot never re-wakes on its own TTS echo. +- **Two listen paths.** Audio (`push_listen_pcm` → VAD → `run_turn` → STT → LLM → TTS) and captions (`push_caption` → wake-word gate → `run_caption_turn`, which skips STT since captions are already text). The caption path is the primary in-product flow. +- **No toolless fallback on agentic failure.** When the orchestrator path fails/times out (`AGENTIC_TURN_TIMEOUT_SECS` = 90s), the bot speaks a canned "Let me get back to you" rather than the bare LLM — a toolless model confidently hallucinates "I don't have access" to calendar/Slack/Gmail, which is worse than a deferral. +- **Heavy rate-limiting against Meet's caption churn.** Meet re-emits the same caption row every ~250 ms with punctuation/case jitter; the session layers per-speaker dedup, a `turn_in_progress` gate, a 60s min-turn-gap, a caption-ts wake cooldown, and a 20s unauthorized soft-deny cooldown to prevent "sorry, sorry, sorry" / multi-fire loops. +- **Owner grant flow.** A non-owner wake records a pending grantee for `PENDING_GRANT_WINDOW_MS` (2 min); the owner saying "allow"/"go ahead"/"let them in" (`looks_like_grant_intent`) adds the speaker to the per-call allowlist via `run_grant_turn`. The allowlist resets when the session is dropped on stop. +- **Reasoning-model output is scrubbed.** `strip_for_speech` removes `` blocks, markdown, and untagged reasoning preamble, then `cap_for_speech` truncates to `MAX_TTS_CHARS` (400) at a sentence boundary so TTS stays interruptible. +- **Persistence is best-effort.** A failed `append_record` on stop logs a warning but never blocks the `stop_session` response. diff --git a/src/openhuman/memory_sources/README.md b/src/openhuman/memory_sources/README.md new file mode 100644 index 000000000..eb7030d48 --- /dev/null +++ b/src/openhuman/memory_sources/README.md @@ -0,0 +1,107 @@ +# memory_sources + +Registry of data connectors that feed memory. This domain owns the **"what feeds my memory"** question: a typed registry of sources (Composio OAuth connections, local folders, GitHub repos, RSS feeds, Twitter queries, web pages) persisted in `config.toml` under `[[memory_sources]]`. It provides CRUD for source entries, a `SourceReader` trait with per-kind reader implementations that list items and read individual item content, manual sync orchestration that ingests reader output into the memory pipeline, per-source sync status, and the `openhuman.memory_sources_*` RPC surface. It does **not** own sync scheduling or the ingestion engine itself — `memory_sync` / `memory` do that; this module only defines connectors, reads from them, and dispatches sync work to the right backend. + +## Responsibilities + +- CRUD for `MemorySourceEntry` records (add/get/list/update/remove) persisted in `Config.memory_sources`. +- Validate kind-specific required fields at add/update time. +- Provide a uniform `SourceReader` trait with one implementation per `SourceKind`, plus a `reader_for(kind)` dispatcher. +- List readable items and read individual item content from each source. +- Trigger a manual sync per source: Composio sources delegate to `memory_sync::composio`; reader-backed kinds (folder/github/rss/web) walk items and ingest each via `memory::ingest_pipeline::ingest_document`; Twitter is a placeholder. +- Emit sync progress as `MemorySyncStageChanged` events tagged with `connection_id = Some(source.id)`. +- Compute per-source sync status (chunks synced/pending, last-chunk timestamp, freshness label) by querying `mem_tree_chunks`. +- Reconcile active Composio connections into the registry at boot / on list, so freshly-connected integrations appear as sources without a restart. +- Auto-upsert a Composio source on OAuth connection creation (called from `memory_sync::composio::bus`). + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/memory_sources/mod.rs` | Module docstring + `pub mod` decls + re-exports (registry CRUD, schemas, core types). Export-focused. | +| `src/openhuman/memory_sources/types.rs` | Core types: `SourceKind`, `MemorySourceEntry` (flattened kind-specific `Option` fields) with `validate()`, `SourceItem`, `ContentType`, `SourceContent`. | +| `src/openhuman/memory_sources/registry.rs` | CRUD over `Config.memory_sources` via the config load/save cycle; `MemorySourcePatch` partial-update payload; `upsert_composio_source` for auto-registration. | +| `src/openhuman/memory_sources/rpc.rs` | RPC handler impls returning `RpcOutcome` (request/response structs for list/get/add/update/remove/list_items/read_item/sync/status_list). `list_rpc` lazily reconciles Composio sources. | +| `src/openhuman/memory_sources/schemas.rs` | Controller-registry schemas + `handle_*` fns delegating to `rpc.rs`; `all_controller_schemas` / `all_registered_controllers`. | +| `src/openhuman/memory_sources/sync.rs` | Per-source sync orchestration. Spawns background task, dispatches by kind, ingests reader output, emits stage events. | +| `src/openhuman/memory_sources/status.rs` | `SourceStatus`, `FreshnessLabel`, `source_status` / `status_list` — queries `mem_tree_chunks` by source-id prefix. | +| `src/openhuman/memory_sources/reconcile.rs` | `ensure_composio_sources` — scans active Composio sync targets and upserts them as sources. | +| `src/openhuman/memory_sources/readers/mod.rs` | `SourceReader` async trait + `reader_for(kind)` dispatcher. | +| `src/openhuman/memory_sources/readers/composio.rs` | `ComposioReader` — returns the connection as a single item; `read_item` is a non-op placeholder (sync is provider-driven). | +| `src/openhuman/memory_sources/readers/folder.rs` | `FolderReader` — glob over a local dir (default `**/*.md`, 10 MB cap), reads file content with path-traversal guard. | +| `src/openhuman/memory_sources/readers/github.rs` | `GithubReader` — pulls project activity (commits/issues/PRs) via `gh` CLI or public REST fallback. | +| `src/openhuman/memory_sources/readers/rss.rs` | `RssReader` — RSS/Atom feed items. | +| `src/openhuman/memory_sources/readers/twitter.rs` | `TwitterReader` — Twitter query reader (sync placeholder pending credentials). | +| `src/openhuman/memory_sources/readers/web_page.rs` | `WebPageReader` — fetches a web page, optional CSS selector. | + +## Public surface + +Re-exported from `mod.rs`: + +- **registry**: `add_source`, `get_source`, `list_sources`, `list_enabled_by_kind`, `remove_source`, `update_source`, `upsert_composio_source`, `MemorySourcePatch`. +- **schemas**: `all_memory_sources_controller_schemas`, `all_memory_sources_registered_controllers`. +- **types**: `ContentType`, `MemorySourceEntry`, `SourceContent`, `SourceItem`, `SourceKind`. + +Reader trait `SourceReader` and `reader_for` are public under `readers`; sync/status/reconcile entry points (`sync::sync_source`, `status::status_list` / `source_status`, `reconcile::ensure_composio_sources`) are public within the module path. + +## RPC / controllers + +Namespace `memory_sources` (`openhuman.memory_sources_*`). Nine controllers, each schema/handler pair defined in `schemas.rs` and delegating to `rpc.rs`: + +| Function | Description | +| --- | --- | +| `list` | List all configured sources (lazily reconciles Composio first). | +| `get` | Get one source by `id`. | +| `add` | Add a source; kind-specific fields are flat on the request. | +| `update` | Partial update of a source. | +| `remove` | Remove a source by `id`. | +| `list_items` | List readable items from a source via its reader. | +| `read_item` | Read one item's content. | +| `sync` | Queue a manual sync (returns immediately; progress via events). | +| `status_list` | Per-source sync status (chunks, freshness, last-chunk ts). | + +Wired into the registry via `core/all.rs` (`all_memory_sources_registered_controllers` / `all_memory_sources_controller_schemas`). + +## Agent tools + +None. This module exposes no agent tools (`tools.rs` does not exist). + +## Events + +No `bus.rs` / `EventHandler` of its own. It **publishes** `DomainEvent::MemorySyncStageChanged` indirectly via `memory::sync::emit_sync_stage` during `sync_source` (stages: Requested, Fetching, Stored, Ingesting, Completed, Failed), tagged `connection_id = Some(source.id)`. The reverse direction — auto-registering a Composio source on connection-created — is driven by `memory_sync::composio::bus`, which calls this module's `upsert_composio_source`. + +## Persistence + +- **Source registry**: persisted in `Config.memory_sources` (`config/schema/types.rs`), serialized as `[[memory_sources]]` in `config.toml`. All mutations reload the live config, apply, and `config.save()` atomically. +- **No dedicated `store.rs`.** Sync status is *read* (not written) from the memory store: `status.rs` queries `mem_tree_chunks` (via `memory_store::chunks::store::with_connection`) using a `source_id LIKE` prefix — `mem_src:{source.id}:%` for reader kinds, `{toolkit}:%` for Composio. Chunks themselves are written by the `memory` ingest pipeline, not here. + +## Dependencies + +- `openhuman::config` (`Config`, `config::rpc::load_config_with_timeout`) — the registry's backing store; readers/sync receive `&Config`. +- `openhuman::memory::ingest_pipeline::ingest_document` — ingests reader-backed source items into memory. +- `openhuman::memory::sync` (`emit_sync_stage`, `MemorySyncStage`, `MemorySyncTrigger`) — sync-progress event emission. +- `openhuman::memory_sync::composio` — Composio sync delegate (`run_connection_sync`, `scan_active_sync_targets`, `SyncReason`). +- `openhuman::memory_sync::canonicalize::document::DocumentInput` — document shape for ingestion. +- `openhuman::memory_store::chunks::store::with_connection` — SQLite access for status queries against `mem_tree_chunks`. +- `core::all` (`ControllerFuture`, `RegisteredController`) and `core` schema types (`ControllerSchema`, `FieldSchema`, `TypeSchema`) — controller registry wiring. +- `rpc::RpcOutcome` — RPC return contract. +- External: `glob`, `async_trait`, `uuid`, `chrono`, `serde`/`serde_json`, `schemars`, `toml`; `gh` CLI (optional) for the GitHub reader. + +## Used by + +- `core/all.rs`, `core/jsonrpc.rs` — registers the `memory_sources` controllers/schemas. +- `openhuman/mod.rs` — declares the domain module. +- `config/schema/types.rs` — `Config.memory_sources: Vec` field (the persisted store). +- `memory_sync::composio::bus` / `memory_sync::composio::mod` — calls `upsert_composio_source` to auto-register a source on connection creation. +- `composio::ops` — references gmail memory-source cleanup targets (`gmail_memory_sources_for_connection`). + +## Notes / gotchas + +- `MemorySourceEntry` is a single flat struct: all kind-specific fields are `Option`/`Vec` and the `kind` discriminator decides which are required, enforced only by `validate()` (not the type system). RPC `add`/`update` mirror this flat shape. +- `list_rpc` performs a lazy Composio reconciliation on every list call so newly-connected integrations show up immediately (the connection-created hook only fires on OAuth handoff, not on first launch after a prior connect). +- `sync_source` returns `Ok(())` as soon as work is queued; it spawns a nested `tokio::spawn` so a panic in the sync task surfaces as a `tracing::error!` rather than a dropped join handle. Actual completion/failure arrives only via `MemorySyncStageChanged` events. +- Composio sync does not ingest item-by-item — it delegates wholesale to `memory_sync::composio::run_connection_sync`. The `ComposioReader::read_item` body is an explanatory placeholder, never a real fetch. +- Twitter sync is intentionally unimplemented: `sync_source` returns an error for `TwitterQuery` ("Twitter sync not yet configured"). +- Status freshness thresholds: ≤30 s → `Active`, ≤5 min → `Recent`, else / no chunk → `Idle`. `status.rs` surfaces real SQL errors (so a broken DB isn't reported as a healthy zero-row state), but `status_list` degrades a per-source failure to an `Idle` zero-row entry rather than failing the whole call. +- Composio chunk-count matching is by `toolkit` prefix only (`{toolkit}:%`), so distinct connections sharing a toolkit (e.g. two Gmail accounts) are not disambiguated in status counts. +- `FolderReader` caps files at 10 MB on both list and read, and canonicalizes paths to deny traversal outside the configured base. diff --git a/src/openhuman/migration/README.md b/src/openhuman/migration/README.md new file mode 100644 index 000000000..c7ca5cd32 --- /dev/null +++ b/src/openhuman/migration/README.md @@ -0,0 +1,80 @@ +# migration + +Data-migration helpers that import memory from **other AI assistants' workspaces** (OpenClaw, Hermes Agent) into the current OpenHuman workspace's memory backend. It scans a source workspace for SQLite (`brain.db`) and Markdown memory artifacts, normalizes them into `Memory` entries, backs up the target's existing memory, and writes the imported entries — supporting a `dry_run` plan-only mode and idempotent re-runs (unchanged entries are skipped, conflicts are renamed). Exposes two RPC controllers under the `migrate` namespace. + +> Not to be confused with `crate::openhuman::migrations` (plural), which handles internal config **schema** version upgrades. This module migrates **user memory data** from foreign vendors. + +## Responsibilities + +- Resolve a source workspace path (explicit override, else vendor default: `~/.openclaw/workspace`, or `~/.hermes` / `%LOCALAPPDATA%\hermes` on Windows). +- Refuse self-migration when source resolves to the current OpenHuman workspace. +- **OpenClaw**: read memory entries from `memory/brain.db` (SQLite `memories` table, schema-tolerant column detection) plus `MEMORY.md` and `memory/*.md`. +- **Hermes**: read a fixed file mapping — `MEMORY.md` → core, `USER.md` → `Custom("user_profile")`, `SOUL.md` → `Custom("persona")`. +- Normalize keys (non-alphanumeric → `_`), parse/map categories, de-dup exact duplicates for deterministic re-runs. +- Back up the target workspace's existing memory (`MEMORY.md`, `brain.db`, `memory/*.md` → `memory_backup/`) before applying. +- Import into the target memory backend: skip entries whose content is unchanged, rename key on content conflict (`key_1`, `key_2`, …). +- Produce a `MigrationReport` (source/target paths, dry-run flag, `MigrationStats`, warnings). + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/migration/mod.rs` | Export-only: declares `core`/`ops`/`schemas`, re-exports `core::*` and `ops::*`, aliases `ops as rpc`, and exports the `all_migration_controller_schemas` / `all_migration_registered_controllers` pair. | +| `src/openhuman/migration/core.rs` | Core logic. `MigrationStats`, `MigrationReport`, `SourceEntry`; `migrate_openclaw_memory` / `migrate_hermes_memory`; source readers (SQLite + Markdown), workspace resolution, key/category normalization, backup, conflict-rename helpers. Inline `#[cfg(test)]` unit tests. | +| `src/openhuman/migration/ops.rs` | JSON-RPC/CLI adapter (the canonical handler file, re-exported as `rpc`). `migrate_openclaw` / `migrate_hermes` wrap the core fns, map `anyhow::Error` → `String`, and return `RpcOutcome` with a `"migration completed"` log. Tests cover dry-run, apply, missing-source, and self-migration. | +| `src/openhuman/migration/schemas.rs` | Controller schemas + handlers. Defines `MigrateOpenClawParams` / `MigrateHermesParams`, `all_controller_schemas`, `all_registered_controllers`, `schemas(function)`, and `handle_migrate_openclaw` / `handle_migrate_hermes` which load config and delegate to `migration::rpc::*`. | + +## Public surface + +From `mod.rs` re-exports (`core::*` + `ops::*`): + +- Types: `MigrationReport`, `MigrationStats`. +- Core fns: `migrate_openclaw_memory(config, source_workspace, dry_run) -> Result`, `migrate_hermes_memory(...)`. +- RPC fns (via `ops` / alias `rpc`): `migrate_openclaw(...) -> Result, String>`, `migrate_hermes(...)`. +- Controller registry exports: `all_migration_controller_schemas`, `all_migration_registered_controllers`. + +(`SourceEntry` and the internal helpers in `core.rs` are private.) + +## RPC / controllers + +Two controllers in the `migrate` namespace (registered via `all_registered_controllers`): + +| Method | Description | Inputs | Output | +| --- | --- | --- | --- | +| `migrate.openclaw` | Migrate OpenClaw memory into current workspace. | `source_workspace?: String`, `dry_run?: bool` | `report: MigrationReport` | +| `migrate.hermes` | Migrate Hermes Agent memory into current workspace. | `source_workspace?: String`, `dry_run?: bool` | `report: MigrationReport` | + +Both inputs are optional; `dry_run` **defaults to `true`** when omitted (`handle_*` use `unwrap_or(true)`). Handlers load config via `config_rpc::load_config_with_timeout()`. Unknown function names yield an `"unknown"` placeholder schema with an `error` output. + +## Agent tools + +None. This module owns no `tools.rs`. + +## Events + +None. No `bus.rs` / event-bus subscribers or publishers. + +## Persistence + +No own store. It writes imported entries through the **target memory backend** obtained from `memory_store::create_memory_for_migration(&config.memory, &config.workspace_dir)`, using `Memory::store` / `Memory::get`. Before applying (non-dry-run), it copies the target's existing memory artifacts into `/memory_backup/`. + +## Dependencies + +- `crate::openhuman::config::Config` — source of `workspace_dir` and `memory` config; `config::rpc::load_config_with_timeout` in handlers. +- `crate::openhuman::memory` (`Memory`, `MemoryCategory`) — target backend trait + category enum entries are mapped into. +- `crate::openhuman::memory_store` — `create_memory_for_migration` constructs the target memory backend. +- `crate::core::all` (`ControllerFuture`, `RegisteredController`) and `crate::core::{ControllerSchema, FieldSchema, TypeSchema}` — controller registry/schema types. +- `crate::rpc::RpcOutcome` — RPC response envelope. +- External crates: `rusqlite` (read OpenClaw `brain.db`), `directories::UserDirs` (home dir), `anyhow`, `serde`/`serde_json`. + +## Used by + +- `src/core/all.rs` — registers `all_migration_registered_controllers()` (line ~165) and `all_migration_controller_schemas()` (line ~319) into the global controller/schema registry, exposing both methods over CLI and JSON-RPC. + +## Notes / gotchas + +- `dry_run` default is `true` at the RPC boundary — callers must explicitly pass `dry_run: false` to actually apply a migration. +- OpenClaw SQLite reading is **schema-tolerant**: it inspects `PRAGMA table_info(memories)` and picks key/content/category columns from candidate name lists (`key`/`id`/`name`, `content`/`value`/`text`/`memory`, `category`/`kind`/`type`), bailing only if no content-like column exists. DB is opened read-only. +- Idempotency: exact-duplicate source entries are de-duped; unchanged target entries are skipped (`skipped_unchanged`); content conflicts get a renamed key (`renamed_conflicts`). +- `migrate_openclaw_apply_imports_markdown_entries` in `ops.rs` documents a regression (#1440): the apply path previously bailed in `create_memory_for_migration` under the unified-namespace memory core; that hard-disable was removed. +- Memory entries are stored under the empty namespace (`""`). diff --git a/src/openhuman/migrations/README.md b/src/openhuman/migrations/README.md new file mode 100644 index 000000000..bd14da6ee --- /dev/null +++ b/src/openhuman/migrations/README.md @@ -0,0 +1,75 @@ +# migrations + +Startup data-migration runner gated by `Config::schema_version`. Each migration is a one-shot, idempotent transformation of on-disk data (the persisted `config.toml` and session transcripts). The runner — `run_pending` — is invoked from `Config::load_or_init` and is a fast no-op for workspaces whose `schema_version` already matches `CURRENT_SCHEMA_VERSION`. Failures are logged but never block startup; the next launch retries from the same starting version. + +> Not to be confused with the sibling `src/openhuman/migration/` (singular), which is a **user-triggered RPC** that imports memory from a legacy OpenClaw workspace. This module (`migrations`, plural) is the **automatic schema-version runner** that fires once per workspace on the first launch of a new build. + +## Responsibilities + +- Maintain `CURRENT_SCHEMA_VERSION` (currently `6`) as the target schema version, bumped alongside every new migration. +- Run pending migrations in order, each guarded by an exact `schema_version ==` check so a failed earlier step is never skipped (the 0→1 step uses `< 1`). +- Persist each version bump via `Config::save()` **only after** the migration reports success; on a save failure, roll the in-memory `schema_version` back so the next launch retries the same gate. +- Offload the blocking fs walk (`phase_out_profile_md`, 0→1) onto `tokio::task::spawn_blocking` to keep the executor responsive; pure in-memory config mutations run inline. +- Coalesce two independent, idempotent migrations behind the single 5→6 transition (`repair_http_request_limits` + `reconcile_orphaned_providers`); bump + save only when both succeed. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/migrations/mod.rs` | Module docstring + the `run_pending` runner and `CURRENT_SCHEMA_VERSION` constant; declares each migration as a private `mod`. | +| `src/openhuman/migrations/phase_out_profile_md.rs` | **0→1.** Deletes `{workspace}/PROFILE.md` and strips `### PROFILE.md` blocks from persisted JSONL session transcripts (`session_raw/**`) and `.md` companions (`sessions/**`). Blocking fs I/O. | +| `src/openhuman/migrations/unify_ai_provider_settings.rs` | **1→2.** Consolidates scattered AI-provider settings into per-workload provider strings; seeds `cloud_providers` (always an `Openhuman` entry) and migrates a non-backend `inference_url` into a `Custom` cloud entry. Pure in-memory. | +| `src/openhuman/migrations/retire_chat_v1_model.rs` | **2→3.** Remaps a persisted `default_model = "chat-v1"` to `"reasoning-quick-v1"` (backend dropped `chat-v1` from its strict registry; sub-agent spawns 400'd). Pure in-memory. | +| `src/openhuman/migrations/expand_autonomy_defaults.rs` | **3→4.** Additively merges expanded `autonomy.allowed_commands` / `auto_approve` defaults (PR #2500) and bumps `max_actions_per_hour` from the old `20` to `u32::MAX` only when still exactly `20`. Pure in-memory. | +| `src/openhuman/migrations/remove_write_auto_approve.rs` | **4→5.** Removes `file_write` / `edit_file` from `autonomy.auto_approve` so Supervised mode resumes its ask-before-edit prompt. Pure in-memory. | +| `src/openhuman/migrations/repair_http_request_limits.rs` | **5→6 (a).** Coerces stale-zero `[http_request]` `timeout_secs` / `max_response_size` to schema defaults (30s / 1 MB); a persisted `0` is an instant timeout / empty-body cap that serde defaults don't repair. Pure in-memory. | +| `src/openhuman/migrations/reconcile_orphaned_providers.rs` | **5→6 (b).** Resets per-workload `*_provider` strings (and a dangling `primary_cloud`) that point at a cloud provider no longer in `cloud_providers`, which the inference factory hard-errors on; mirrors the factory's exact grammar. Pure in-memory. | +| `src/openhuman/migrations/mod_tests.rs` | Tests for `run_pending` ordering, gating, rollback-on-save-failure. | +| `src/openhuman/migrations/*_tests.rs` | Per-migration unit tests (`phase_out_profile_md_tests.rs`, `reconcile_orphaned_providers_tests.rs`, `unify_ai_provider_settings_tests.rs`); other migrations keep inline `#[cfg(test)]` tests. | + +## Public surface + +- `migrations::run_pending(config: &mut Config) -> impl Future` — the only public entry point; called by `Config::load_or_init`. +- `migrations::CURRENT_SCHEMA_VERSION: u32` — target schema version (`6`). + +All individual migration modules (`phase_out_profile_md`, `unify_ai_provider_settings`, …) are **private** to the module; each exposes a `run(...)` fn and a `*Stats` struct used internally by the runner for diagnostics logging. + +## RPC / controllers + +None. This module exposes no controllers or RPC methods (the RPC-facing migration surface lives in the sibling `src/openhuman/migration/`, singular). + +## Agent tools + +None. + +## Events + +None — no event-bus publishers or subscribers. + +## Persistence + +Does not own a `store.rs`. Its effects are written through other layers: + +- `Config` mutations (incl. `schema_version`) persisted via `Config::save()` to `config.toml`, driven by the runner in `mod.rs`. +- `phase_out_profile_md` rewrites session transcript files in place under `{workspace}/session_raw/**` and `{workspace}/sessions/**`, and deletes `{workspace}/PROFILE.md`. + +## Dependencies + +- `crate::openhuman::config` (`Config`, `HttpRequestConfig`, `schema::cloud_providers::{CloudProviderCreds, AuthStyle, CloudProviderType, generate_provider_id}`, `schema::{MODEL_CHAT_V1, MODEL_REASONING_QUICK_V1}`) — the migrations read and mutate the config struct; `schema_version` is the gating field. +- `crate::openhuman::agent::harness::session::transcript` (`transcript`, `SessionTranscript`, `write_transcript`) — `phase_out_profile_md` parses and rewrites persisted session transcripts byte-compatibly. +- `crate::openhuman::inference::provider::factory` — `reconcile_orphaned_providers` mirrors the factory's exact, case-sensitive provider-string grammar so "resolvable here" matches "resolvable at inference time"; `unify_ai_provider_settings` references the factory's provider-string format. (`inference::provider::ChatMessage` is imported only by tests.) + +## Used by + +- `src/openhuman/config/schema/load.rs` — calls `migrations::run_pending(&mut config)` from `Config::load_or_init` (three call sites) and seeds new configs with `schema_version: CURRENT_SCHEMA_VERSION`. +- `src/openhuman/mod.rs` — declares `pub mod migrations;`. + +## Notes / gotchas + +- **Best-effort, never blocks startup.** A migration error or a `spawn_blocking` join failure is logged at WARN and `run_pending` returns; the gate stays at the failing version and retries next launch. +- **Save-failure rollback.** Each step bumps `config.schema_version` in memory, then `Config::save().await`; if the save fails the version is rolled back to `previous_version` so disk and memory stay consistent. +- **Exact `==` gating (except 0→1).** Steps 1→2 through 4→5 guard on `schema_version == N`, not `< N+1`, so a failed earlier migration cannot be silently skipped. The first step uses `< 1`. +- **Idempotency is doubly guaranteed:** externally by the `schema_version` gate, and internally — every migration is a no-op when re-run on already-migrated data (e.g. additive merges guarded by `contains`, `is_none()` field checks, absence of a `### PROFILE.md` block). +- **5→6 shares one version bump for two modules.** Both `repair_http_request_limits` and `reconcile_orphaned_providers` run; the gate advances to 6 only if both succeed. Re-running the one that already succeeded on the next launch is a no-op. +- **Adding a migration:** add a `mod`, bump `CURRENT_SCHEMA_VERSION`, and add a `if config.schema_version == N` branch in `run_pending` that calls the module and bumps + saves on success (see the mod.rs docstring). +- Migration `run(...)` fns return `anyhow::Result<…>` to match the runner's dispatch signature even when the transform is currently infallible, so a future I/O-backed step slots in without churning the runner. diff --git a/src/openhuman/notifications/README.md b/src/openhuman/notifications/README.md new file mode 100644 index 000000000..9ce853f33 --- /dev/null +++ b/src/openhuman/notifications/README.md @@ -0,0 +1,98 @@ +# notifications + +The notifications domain owns two complementary sub-systems. The **core-bridge** translates selected `DomainEvent`s (cron completions, webhook failures, sub-agent results, triaged integration notifications) into compact `CoreNotificationEvent` payloads pushed over a broadcast channel into the Socket.IO bridge, surfacing as in-app notification-center items. The **integration-notification pipeline** ingests notifications captured from embedded webview integrations (Gmail, Slack, WhatsApp, …), persists them to a per-workspace SQLite store, runs each through the triage LLM pipeline in the background to back-fill an importance score/action, and exposes a unified RPC surface for listing, marking, settings, and stats. + +## Responsibilities + +- Subscribe to cross-domain `DomainEvent`s and republish a curated subset as user-facing `CoreNotificationEvent`s (with title/body/category/deep-link) onto an in-process broadcast bus consumed by the Socket.IO bridge. +- Filter noise at the bridge: only failed webhooks surface; only `routed` triage actions (`escalate`/`react`) surface; `drop`/`acknowledge`/unrouted are silent. +- Ingest integration notifications, dedup against identical content received in the last 60s, and persist them immediately. +- Spawn background triage per ingest; map the triage action to a 0.0–1.0 importance score and back-fill `importance_score`/`triage_action`/`triage_reason`/`scored_at` in place. +- Auto-route high-importance (`escalate`/`react`) notifications to the orchestrator when the provider's settings allow (re-reading settings just before routing). +- Persist and expose per-provider settings (enabled, importance threshold, route-to-orchestrator). +- Track lifecycle state (`unread`/`read`/`acted`/`dismissed`) and aggregate pipeline stats. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/notifications/mod.rs` | Export-focused module root; docstring + re-exports of bus helpers, schema registries, and types. | +| `src/openhuman/notifications/types.rs` | Serde domain types: `CoreNotificationEvent`/`CoreNotificationCategory` (bridge), `IntegrationNotification`, `NotificationStatus`, `NotificationSettings`, `NotificationStats`, and RPC request types. | +| `src/openhuman/notifications/bus.rs` | `NotificationBridgeSubscriber` (`EventHandler`), the `NOTIFICATION_BUS` broadcast static, `publish_core_notification`/`subscribe_core_notifications`, the pure `event_to_notification` translator, and `register_notification_bridge_subscriber`. | +| `src/openhuman/notifications/rpc.rs` | Async RPC handler fns (`handle_ingest`, `handle_list`, `handle_mark_read`, `handle_dismiss`, `handle_mark_acted`, `handle_settings_get`/`_set`, `handle_stats`) + `triage_action_to_score` heuristic. | +| `src/openhuman/notifications/schemas.rs` | Controller schema defs, the `NOTIFICATION_CONTROLLER_DEFS` table, `all_controller_schemas`/`all_registered_controllers`, and handler wrappers delegating to `rpc.rs`. | +| `src/openhuman/notifications/store.rs` | SQLite persistence (`integration_notifications` + `notification_settings` tables) via a per-call `with_connection` helper; insert/list/dedup/triage-update/status/settings/stats queries. | +| `src/openhuman/notifications/bus_tests.rs` | Sibling test suite for the bridge. | +| `src/openhuman/notifications/store_tests.rs` | Sibling test suite for the store. | + +## Public surface + +Re-exported from `mod.rs`: + +- From `bus`: `publish_core_notification`, `subscribe_core_notifications`, `register_notification_bridge_subscriber`, `NotificationBridgeSubscriber`. +- From `schemas`: `all_notifications_controller_schemas` (alias of `all_controller_schemas`), `all_notifications_registered_controllers` (alias of `all_registered_controllers`). +- From `types` (`pub use types::*`): `CoreNotificationEvent`, `CoreNotificationCategory`, `IntegrationNotification`, `NotificationStatus`, `NotificationSettings`, `NotificationStats`, `NotificationIngestRequest`, `NotificationSettingsUpsertRequest`. + +## RPC / controllers + +Namespace `notification` (8 controllers, registered via `all_notifications_registered_controllers`): + +| Function | Inputs | Output | +| --- | --- | --- | +| `ingest` | `provider`, `title`, `body`, `raw_payload` (req); `account_id` (opt) | `{ id?, skipped, reason? }` — persists then spawns background triage; skips when provider disabled or duplicate. | +| `list` | `provider?`, `limit?` (50), `offset?` (0), `min_score?` | `{ items, unread_count }` — ordered `received_at` DESC; unscored items pass the score filter. | +| `mark_read` | `id` | `{ ok }` | +| `dismiss` | `id` | `{ ok }` (true when a row matched) | +| `mark_acted` | `id` | `{ ok }` (true when a row matched) | +| `settings_get` | `provider` | `{ settings }` (defaulted if absent) | +| `settings_set` | `provider`, `enabled`, `importance_threshold`, `route_to_orchestrator` | `{ ok, settings }` — threshold clamped to 0.0–1.0. | +| `stats` | — | `{ total, unread, unscored, by_provider, by_action }` | + +Schemas + handlers are wired into the controller registry in `src/core/all.rs`. + +## Agent tools + +None. This domain owns no `tools.rs`. + +## Events + +**Subscribes** (via `NotificationBridgeSubscriber`, no `domains()` filter — matches on variant): `DomainEvent::CronJobCompleted` (→ Agents), `WebhookProcessed` (failures only → System), `SubagentCompleted`/`SubagentFailed` (→ Agents), and `NotificationTriaged` (only when `routed` and action is `escalate`/`react` → Agents). Each is translated to a `CoreNotificationEvent` and published on the broadcast bus. + +**Publishes**: `DomainEvent::NotificationTriaged` from `rpc::handle_ingest`'s background triage task (carries `id`, `provider`, `action`, `importance_score`, `latency_ms`, `routed`). + +The bridge bus is a separate `tokio::sync::broadcast` channel (not the global event bus); `core::socketio` subscribes to it and forwards each event as the `core_notification` / `core:notification` Socket.IO message. + +## Persistence + +SQLite DB at `{workspace_dir}/notifications/notifications.db`, opened per-call via `with_connection` (idempotent schema migration on each open). Two tables: + +- `integration_notifications` — one row per ingested notification (id, provider, account_id, title, body, raw_payload JSON, importance_score, triage_action, triage_reason, status, received_at, scored_at). Indexed on provider, status, and a dedup tuple (provider, account_id, title, body, received_at). +- `notification_settings` — per-provider routing prefs (enabled, importance_threshold, route_to_orchestrator), upserted on `provider` PK. + +`insert_if_not_recent` runs a `BEGIN IMMEDIATE` transaction so concurrent duplicate ingests collapse to a single insert. + +## Dependencies + +- `crate::core::event_bus` — `DomainEvent`, `EventHandler`, `publish_global`, `subscribe_global` for bridge subscription and triage-result publishing. +- `crate::core::all` (`ControllerFuture`, `RegisteredController`) and `crate::core` (`ControllerSchema`, `FieldSchema`, `TypeSchema`) — controller registry contract. +- `crate::openhuman::config` — `Config` (workspace dir for the DB path) and `config::rpc::load_config_with_timeout` in handlers. +- `crate::openhuman::agent::triage` — `run_triage`, `apply_decision`, `TriageOutcome`, `TriggerEnvelope`, `TriggerSource`, `TriageAction` for the background scoring/routing pipeline. +- `crate::rpc::RpcOutcome` — RPC response shaping. +- External crates: `rusqlite` (store), `chrono`, `uuid`, `serde_json`, `tokio`, `once_cell`, `async_trait`. + +## Used by + +- `src/core/all.rs` — registers the controllers/schemas into the RPC registry. +- `src/core/jsonrpc.rs` — calls `register_notification_bridge_subscriber()` at startup. +- `src/core/socketio.rs` — calls `subscribe_core_notifications()` to forward events to web clients. +- `src/openhuman/cron/scheduler.rs` and `src/openhuman/heartbeat/planner/*` reference the notification surface (e.g. triggering/observing notifications). + +## Notes / gotchas + +- `CoreNotificationEvent` ids embed a publish timestamp, so each cron run / webhook failure / subagent event produces a distinct notification-center entry rather than coalescing. +- `CoreNotificationCategory` must stay in sync with `NotificationCategory` in `app/src/store/notificationSlice.ts`. +- The ingest RPC returns immediately; triage runs in a spawned task and back-fills the score later — list/stats may show `importance_score: null` (unscored) until triage completes. +- Triage→score mapping is a fixed heuristic in `rpc::triage_action_to_score`: Drop 0.1, Acknowledge 0.35, React 0.65, Escalate 0.9. +- Routing re-reads provider settings just before escalation so a mid-flight settings toggle takes effect; routing requires `score >= importance_threshold` AND `route_to_orchestrator`. +- Dedup window is a hard-coded 60 seconds (`exists_recent` / `insert_if_not_recent`). +- The bridge bus is fire-and-forget: with no subscribers, events are dropped (`publish_core_notification` returns the receiver count). diff --git a/src/openhuman/overlay/README.md b/src/openhuman/overlay/README.md new file mode 100644 index 000000000..9c13d0720 --- /dev/null +++ b/src/openhuman/overlay/README.md @@ -0,0 +1,50 @@ +# overlay + +Signals pushed from the core to the desktop **overlay window** — a separate Tauri WebView (`OverlayApp.tsx`, configured in `app/src-tauri/tauri.conf.json`) that renders short, non-focus-stealing messages over the desktop. Because the overlay runs in its own JS runtime it cannot share Redux state with the main window; instead it subscribes to a dedicated Socket.IO connection and reacts to events the core broadcasts. This module owns a single fire-and-forget broadcast bus for **attention** events; the Socket.IO transport bridge (`src/core/socketio.rs`) subscribes and forwards them to the overlay window. It is deliberately light: export-focused, one broadcast channel, no persistence and no RPC. + +## Responsibilities + +- Define the `OverlayAttentionEvent` payload and its `OverlayAttentionTone` visual hint. +- Provide a process-global broadcast channel so any core caller (subconscious loop, heartbeat, screen intelligence, …) can surface a transient overlay message without threading a sender around. +- Expose `publish_attention` (fire-and-forget producer) and `subscribe_attention_events` (consumed by the Socket.IO bridge). +- (STT/dictation overlay activation is driven separately by `voice::dictation_listener`'s `dictation:toggle` / `dictation:transcription` events — not by this module.) + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/overlay/mod.rs` | Module docstring + re-exports only (`publish_attention`, `subscribe_attention_events`, `OverlayAttentionEvent`, `OverlayAttentionTone`). | +| `src/openhuman/overlay/types.rs` | `OverlayAttentionEvent` (message + optional `id` / `tone` / `ttl_ms` / `source`) with builder helpers, and the `OverlayAttentionTone` enum (`neutral` / `accent` / `success`). | +| `src/openhuman/overlay/bus.rs` | `Lazy` `tokio::sync::broadcast` channel (capacity 64) plus `publish_attention` / `subscribe_attention_events`; includes inline `#[cfg(test)]` tests. | + +## Public surface + +- `OverlayAttentionEvent` — `{ id: Option, message: String, tone: OverlayAttentionTone, ttl_ms: Option, source: Option }`. Builders: `new(message)`, `with_source(..)`, `with_tone(..)`, `with_ttl_ms(..)`. Only `message` is required. +- `OverlayAttentionTone` — `Neutral` (default), `Accent`, `Success`; serialized lowercase. Frontend maps to bubble colours. +- `publish_attention(event) -> usize` — broadcasts; returns the number of active subscribers that received it (`0` if none, event then dropped). Logs under `[overlay]`. +- `subscribe_attention_events() -> broadcast::Receiver` — receiver for the bus. + +## Events + +Not a `DomainEvent` / event-bus (`src/core/event_bus/`) participant. It runs its own standalone `tokio::sync::broadcast` channel. The Socket.IO bridge in `src/core/socketio.rs` (`spawn_web_channel_bridge`, task #3) subscribes via `subscribe_attention_events()` and emits each event to the overlay socket as both `overlay:attention` and `overlay_attention`; it logs and continues on `Lagged` and breaks on `Closed`. + +## Persistence + +None. State is purely an in-memory broadcast channel; events not consumed when published are dropped. + +## Dependencies + +- External crates only: `serde` (types), `once_cell::sync::Lazy` + `tokio::sync::broadcast` (bus). No `crate::openhuman::*` or `crate::core::*` imports in the module's own source. + +## Used by + +- `src/core/socketio.rs` — subscribes to the bus and forwards events to the overlay WebView over Socket.IO. +- `src/openhuman/notifications/bus.rs` — references this module's bus only as a documented pattern to mirror (no code dependency). +- Per the module docstring, intended publishers include the subconscious loop, heartbeat, and screen-intelligence domains via `publish_attention`. + +## Notes / gotchas + +- **Fire-and-forget:** if the Socket.IO bridge hasn't started or the overlay socket is disconnected, `publish_attention` drops the event and returns `0`. There is no buffering/replay for late subscribers. +- **Channel capacity is 64** — a slow/absent overlay consumer causes `Lagged` drops (logged as a warning by the bridge), not back-pressure. +- Keep `message` short: the overlay types it out character-by-character and auto-dismisses after `ttl_ms` (frontend default when `None`). +- The bridge emits two event names (`overlay:attention` and `overlay_attention`) for client compatibility. diff --git a/src/openhuman/people/README.md b/src/openhuman/people/README.md new file mode 100644 index 000000000..f5e087e95 --- /dev/null +++ b/src/openhuman/people/README.md @@ -0,0 +1,85 @@ +# people + +Contact resolution + relationship scoring (the "A5" module). Maps any of three handle kinds — iMessage handle, email, or display name — to a single stable `PersonId`, and ranks known people by a deterministic composite score (recency × frequency × reciprocity × depth) derived from observed interaction rows. Backed by its own SQLite database (people / handle aliases / interactions). Can seed itself from the macOS system Address Book (`CNContactStore`). Intentionally self-contained — per its module docstring it has no dependency on `life_capture`, `chronicle`, `nudges`, or UI; downstream integration is left to later slices. + +## Responsibilities + +- Canonicalize handles (lowercase/trim emails + email-style iMessage handles; whitespace-collapse display names) so the same person resolves consistently across case and spacing. +- Deterministically resolve a `Handle` to an existing `PersonId`, or mint a new `Person` skeleton on first sight (`create_if_missing`). +- Link handles together (`link`) so an email + phone + display name can be attached to one person — without ever *auto*-merging distinct identities that share only a display name or an unverified handle. +- Record interactions and aggregate them into a per-person composite score plus an explainable component breakdown. +- Rank all known people by score for `people.list`. +- Seed the store from the macOS Address Book, distinguishing "permission denied" from "no contacts". +- Persist people, handle aliases, and interactions in a dedicated SQLite DB with idempotent migrations. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/people/mod.rs` | Export-focused. Declares submodules and re-exports `all_people_controller_schemas` / `all_people_registered_controllers`. | +| `src/openhuman/people/types.rs` | Domain types: `PersonId`, `Handle` (with `canonicalize` / `as_key`), `Person`, `Interaction`, `ScoreComponents`, `AddressBookContact`. | +| `src/openhuman/people/resolver.rs` | `HandleResolver` — `resolve`, `resolve_or_create(_with_status)`, `link`, `seed_from_address_book`. The deterministic handle→PersonId logic + cross-source merge-safety contract. | +| `src/openhuman/people/scorer.rs` | Pure `score(interactions, now) -> ScoreComponents`. Recency half-life, frequency window/cap, reciprocity balance, depth cap as module constants. | +| `src/openhuman/people/store.rs` | SQLite-backed `PeopleStore` (`Arc>`) + process-global `OnceCell` accessor (`init` / `get`). CRUD, lookup, interaction read/write, batched interaction fetch. | +| `src/openhuman/people/address_book.rs` | `ContactsSource` trait + `SystemContactsSource` (macOS `CNContactStore` FFI via objc2) and non-mac stub; `MockContactsSource` for tests; `AddressBookError`. | +| `src/openhuman/people/rpc.rs` | Domain RPC handlers (`handle_list`, `handle_resolve`, `handle_score`, `handle_refresh_address_book`) returning `RpcOutcome`; callable directly in tests with a constructed `PeopleStore`. | +| `src/openhuman/people/schemas.rs` | Controller schemas + param-parsing adapter handlers that fetch the global store and delegate to `rpc.rs`. | +| `src/openhuman/people/migrations.rs` | Idempotent migration runner (bookkeeping table `_people_migrations`, per-migration transaction). | +| `src/openhuman/people/migrations/0001_init.sql` | Schema: `people`, `handle_aliases`, `interactions` + indexes. | +| `src/openhuman/people/tests.rs` | Cross-file integration tests for the domain. | + +## Public surface + +- Types: `PersonId`, `Handle` (`IMessage` / `Email` / `DisplayName`), `Person`, `Interaction`, `ScoreComponents`, `AddressBookContact`. +- `HandleResolver::{resolve, resolve_or_create, resolve_or_create_with_status, link, seed_from_address_book}`. +- `scorer::score` + tunable constants `RECENCY_HALF_LIFE_DAYS`, `FREQUENCY_WINDOW_DAYS`, `FREQUENCY_CAP`, `DEPTH_CAP_CHARS`. +- `store::{PeopleStore, init, get}` and `ConnHandle`. +- `address_book::{ContactsSource, SystemContactsSource, read, read_with, AddressBookError}`. +- `mod.rs` re-exports `all_people_controller_schemas` / `all_people_registered_controllers` for the controller registry. + +## RPC / controllers + +Registered via the controller registry (wired in `src/core/all.rs`). Four controllers in the `people` namespace: + +| Method | Inputs | Output | +| --- | --- | --- | +| `people.list` | `limit?` (default 100, capped at 500) | `people[]` ranked by score desc — each with `person_id`, `display_name?`, `primary_email?`, `primary_phone?`, `handles[]`, `score`, `components`, `interaction_count`. | +| `people.resolve` | `kind` (`imessage`/`email`/`display_name`), `value`, `create_if_missing?` | `person_id?` (null when unknown and not creating), `created`. | +| `people.score` | `person_id` (UUID) | `person_id`, `score`, `components`, `interaction_count`. Errors if person not found. | +| `people.refresh_address_book` | — | `seeded`, `skipped`, `permission_denied`. | + +`score` / composite is `recency * frequency * reciprocity * depth`, each clamped to `[0,1]`. + +## Persistence + +Dedicated SQLite DB managed by `PeopleStore` (open via `open_at(path)` or `open_in_memory()`; migrations run on open). Three tables (see `0001_init.sql`): + +- `people` — one row per resolved person (uuid id, display name, primary email/phone, timestamps). +- `handle_aliases` — `(kind, value)` primary key → `person_id` (FK, `ON DELETE CASCADE`); `value` is the canonicalized form. This table *is* the resolver index. +- `interactions` — `(person_id, ts, is_outbound, length)` rows the scorer aggregates; indexed by `(person_id, ts DESC)` and `ts DESC`. + +Migrations are tracked in `_people_migrations` and applied idempotently in a transaction. The store is exposed process-globally through a `tokio::sync::OnceCell` (`init` once at startup, `get` from controller handlers). Note: no `people::store::init` call was found outside the module — controller handlers will return "people store not initialised" until something seeds it. + +## Dependencies + +- `crate::core::all::{ControllerFuture, RegisteredController}` — controller registry types for RPC exposure. +- `crate::core::{ControllerSchema, FieldSchema, TypeSchema}` — controller schema definitions. +- `crate::rpc::RpcOutcome` — standard RPC result envelope (`RpcOutcome`). +- External crates: `rusqlite` (storage), `tokio` (async + `spawn_blocking` for sync SQL, `OnceCell`, `Mutex`), `chrono` (timestamps/scoring), `uuid` (`PersonId`), `serde`; on macOS, `block2` / `objc2` / `objc2-contacts` / `objc2-foundation` for the `CNContactStore` FFI in `address_book.rs`. + +Notably it depends on **no other `openhuman` domain** — consistent with its "self-contained" docstring. + +## Used by + +- `src/core/all.rs` — registers the people controllers and schemas, and routes the `"people"` namespace. +- `src/openhuman/memory_store/` — reuses `people::types::{Person, PersonId, Handle}` (e.g. `Person` aliased as `Contact` in `kinds.rs`, and in `traits.rs`). + +## Notes / gotchas + +- **Cross-source merge safety (issue #1538):** two identities that share only a display name or only an unverified handle from different sources are **never** auto-merged. Merging only happens via explicit `link()`. Resolver tests lock this contract in. +- **Idempotent seeding:** `seed_from_address_book` re-runs as a no-op for already-known handles; on `PermissionDenied` it writes nothing (no partial state). The "primary" link target is first email, else first phone, else display name. +- **macOS Address Book FFI must not run on the main thread** — `CNContactStore` access requests deadlock there; `request_access` blocks on a completion-handler channel. Non-mac builds return an empty contact list. +- **Scoring constants are module-level**, not config-driven yet — kept fixed so tests stay stable; the docstring notes they can move to config later without breaking the API. +- **Composite is a product:** any zero component (e.g. a one-sided conversation → reciprocity 0) zeroes the whole score. +- **SQL runs on `spawn_blocking`:** the connection is sync `rusqlite` behind `Arc>`; `JoinError`s from blocking tasks are mapped into a synthetic `rusqlite` IO error. +- **Tests bypass the global store:** they construct `PeopleStore::open_in_memory()` and call `rpc::*` / `HandleResolver` directly rather than going through the schema adapters (which require the `OnceCell`-initialized global). diff --git a/src/openhuman/prompt_injection/README.md b/src/openhuman/prompt_injection/README.md new file mode 100644 index 000000000..b7bc23de3 --- /dev/null +++ b/src/openhuman/prompt_injection/README.md @@ -0,0 +1,70 @@ +# prompt_injection + +Centralized, deterministic prompt-injection screening. Given a user-provided prompt and a small enforcement context, it normalizes the text, scores it against a set of regex rules plus a couple of heuristics, and returns a verdict (`Allow` / `Review` / `Block`) and a corresponding enforcement action. Callers run this **before** any model or tool execution path so adversarial prompts (instruction overrides, role hijacks, system-prompt / credential exfiltration, tool-abuse coercion) are caught up front. There is no persisted state, no RPC surface, and no event subscriber — it is a pure synchronous analysis library exposed via a single function. + +## Responsibilities + +- Normalize incoming prompts to defeat common obfuscation: lowercasing, leet-speak (`0→o`, `1→i`, `@→a`, …), Cyrillic homoglyph folding, fullwidth-ASCII folding, zero-width / bidi / formatting-character stripping, and whitespace collapse (plus a whitespace-stripped `compact` variant). +- Score the prompt against deterministic `DETECTION_RULES` (compiled once into a single `RegexSet` DFA, matched across the `lowered` / `collapsed` / `compact` variants) and two inline heuristics (`has_instruction_override`, `has_exfiltration_intent`). +- Optionally apply an env-gated `HeuristicClassifier` that adds a small bounded score for suspicious trait combinations. +- Map the summed score to a verdict via fixed thresholds: `Block ≥ 0.70`, `Review ≥ 0.55`, else `Allow`. +- Produce a `PromptEnforcementDecision` carrying verdict, score, reason codes/messages, action, a SHA-256 prompt hash, and prompt char count, and emit a structured `tracing::info!` audit line (PII-safe: logs the hash, not the prompt text). + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/prompt_injection/mod.rs` | Module docstring + re-exports of the public surface. No logic. | +| `src/openhuman/prompt_injection/detector.rs` | All logic: types, normalization, detection rules + `RegexSet`, heuristics, optional classifier, scoring/thresholds, and the `enforce_prompt_input` entry point. | +| `src/openhuman/prompt_injection/tests.rs` | `#[cfg(test)]` suite (~40 cases) covering allow/review/block verdicts, obfuscation handling, and known false-positive regressions (TAURI-140, issue #1940). | + +## Public surface + +Re-exported from `mod.rs` (all defined in `detector.rs`): + +- `enforce_prompt_input(input: &str, context: PromptEnforcementContext) -> PromptEnforcementDecision` — the single entry point. +- `PromptEnforcementContext<'a>` — borrowed `source` plus optional `request_id` / `user_id` / `session_id` for the audit log. +- `PromptEnforcementDecision` — `{ verdict, score: f32, reasons, action, prompt_hash: String, prompt_chars: usize }`. +- `PromptEnforcementAction` — `Allow` / `Blocked` / `ReviewBlocked`. +- `PromptInjectionVerdict` — `Allow` / `Block` / `Review` (serde `lowercase`). +- `PromptInjectionReason` — `{ code: String, message: String }`. + +Internal-only (not exported): `DetectionRule`, `NormalizedPrompt`, `HeuristicClassifier`, `OptionalClassifier`, `analyze_prompt`, `normalize_prompt`, `prompt_hash`. + +## Configuration + +- `OPENHUMAN_PROMPT_INJECTION_CLASSIFIER` (env) — resolved once via `Lazy`. `"heuristic"` enables `HeuristicClassifier`; anything else (default `"off"`) disables the optional classifier. The active choice is logged at `debug`. + +## Persistence + +None. The module is stateless aside from `Lazy` statics (compiled regexes, the classifier selection). + +## Dependencies + +No `crate::openhuman::*` or `crate::core::*` dependencies — the module is self-contained. External crates only: + +- `regex` (`Regex`, `RegexSet`) — pattern matching / batched DFA. +- `once_cell::sync::Lazy` — compile-once statics. +- `serde` — derive on the verdict/reason types. +- `sha2` + `hex` — SHA-256 prompt hashing for audit logs. +- `tracing` — structured audit and debug logging. +- `std::env` — classifier selection. + +## Used by + +Consumers call `enforce_prompt_input` and treat any non-`Allow` action as a rejection (returning a user-facing guard message): + +- `src/openhuman/agent/harness/session/runtime.rs` — gates agent session turns; emits `prompt_injection_blocked`. +- `src/openhuman/agent/bus.rs` — screens inbound prompts on the agent event path. +- `src/openhuman/channels/providers/web.rs` — screens chat payloads at the web channel ingress (`start_chat`). +- `src/openhuman/inference/local/ops.rs` — rejects injected prompts before local-AI runtime execution. +- `src/openhuman/about_app/catalog.rs` — surfaces the `conversation.prompt_injection_guard` capability entry. +- `src/core/observability.rs` — classifies `prompt_injection_blocked` error messages for telemetry (string-based, not a code dependency). + +## Notes / gotchas + +- **Three-variant matching is load-bearing**: rules are matched against `lowered`, `collapsed`, and the whitespace-stripped `compact` strings, so spacing-obfuscated attacks (`j a i l b r e a k`, `j w t`) still contribute to score/reasons. +- **Threshold history is encoded in comments**: `Review` was tuned 0.45 → 0.50 → 0.55 and the obfuscated-instruction signal bumped to 0.56 to eliminate a false-positive band while keeping spaced-out overrides at Review level (TAURI-140). +- **Deliberately conservative verb list**: `exfiltrate.credentials_with_intent` excludes high-false-positive verbs (`show`, `give`, `tell`, `fetch`, `return`, `output`) and requires a determiner within a bounded window, so benign technical questions ("show me the password reset flow", "reveal how to set my api key") do not trip it (issue #1940). +- `is_obfuscation_char` is the single source of truth shared between the `had_zwsp` flag and the stripping step to prevent drift. +- `Review` and `Block` both yield non-`Allow` actions; every current caller treats them identically (rejection), so the distinction is informational/audit-only at the call sites. diff --git a/src/openhuman/provider_surfaces/README.md b/src/openhuman/provider_surfaces/README.md new file mode 100644 index 000000000..46cfee0e5 --- /dev/null +++ b/src/openhuman/provider_surfaces/README.md @@ -0,0 +1,75 @@ +# provider_surfaces + +Local assistive surfaces for third-party provider apps. This domain owns a normalized provider-event model and an in-memory **respond queue** that sits above embedded webviews (and future API-first integrations), so the assistant can surface actionable items (messages, mentions, etc.) from providers like LinkedIn/Gmail in a single local-first list. The current cut is an intentionally minimal scaffold: it exposes two RPC controllers, keeps state in process memory, and is wired into the controller registry ahead of behavioral/SQLite work. + +## Responsibilities + +- Define a normalized inbound `ProviderEvent` contract (provider slug, account id, event kind, entity id, optional thread/title/snippet/sender/deep-link, timestamp, `requires_attention`, optional raw payload). +- Ingest provider events and upsert them into a local respond queue keyed by `provider:account_id:event_kind:entity_id`. +- List the respond queue (newest-first) for assistive UI surfaces. +- Bound queue growth with a soft cap (500 items, oldest-from-tail dropped) under provider firehose volume. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/provider_surfaces/mod.rs` | Export-only: declares submodules; re-exports `all_provider_surfaces_controller_schemas` / `all_provider_surfaces_registered_controllers`. | +| `src/openhuman/provider_surfaces/types.rs` | Serde domain types: `ProviderEvent`, `RespondQueueItem`, `RespondQueueListResponse`. Snake_case contract shared by request and response. | +| `src/openhuman/provider_surfaces/ops.rs` | Business logic / entry points: `ingest_event`, `list_queue`. Wrap results in `ApiEnvelope` + `RpcOutcome`. Contains the inline test suite. | +| `src/openhuman/provider_surfaces/store.rs` | In-memory persistence: process-global `RESPOND_QUEUE` (`OnceLock>>`), `upsert_queue_item`, `list_queue_items`, `clear_queue` (test-only). | +| `src/openhuman/provider_surfaces/schemas.rs` | Controller registry: `ControllerSchema`s + `handle_*` fns delegating to `ops.rs`. Inline schema tests. | +| `src/openhuman/provider_surfaces/rpc.rs` | Docstring-only placeholder; no code. The handler delegation lives in `schemas.rs`, not here. | + +## Public surface + +- `types::ProviderEvent` — inbound normalized provider event (`#[serde(deny_unknown_fields)]`). +- `types::RespondQueueItem` — queue entry (adds `id` and `status`, default `"pending"`). +- `types::RespondQueueListResponse` — `{ items, count }`. +- `ops::ingest_event(ProviderEvent)` / `ops::list_queue(EmptyRequest)` — async handlers returning `RpcOutcome>`. +- `store::{upsert_queue_item, list_queue_items}` — used directly by `desktop_companion` (see Used by). +- Re-exported from `mod.rs`: `all_provider_surfaces_controller_schemas`, `all_provider_surfaces_registered_controllers`. + +## RPC / controllers + +Namespace `provider_surfaces` (two controllers, registered via `src/core/all.rs`): + +| Method | Inputs | Output | +| --- | --- | --- | +| `provider_surfaces.ingest_event` | `provider`, `account_id`, `event_kind`, `entity_id`, `timestamp` (required); `thread_id`, `title`, `snippet`, `sender_name`, `sender_handle`, `deep_link` (optional); `requires_attention` (bool, defaults false), `raw_payload` (optional JSON) | Envelope containing the upserted `RespondQueueItem`. | +| `provider_surfaces.list_queue` | none | Envelope containing `{ items, count }`. | + +`requires_attention` is `required: false` in the schema to match `ProviderEvent`'s `#[serde(default)]` so registry `validate_params` agrees with the struct. + +## Agent tools + +None — no `tools.rs`; this domain owns no agent tools. + +## Events + +None — no `bus.rs`; no `DomainEvent`s published or subscribed. + +## Persistence + +In-memory only. State lives in a process-global `RESPOND_QUEUE` (`static OnceLock>>`) in `store.rs`, prepend-ordered (newest-first), soft-capped at `MAX_QUEUE_ITEMS = 500` (oldest dropped from the tail). Upsert dedupes by composite id `provider:account_id:event_kind:entity_id`. Module docstrings flag SQLite-backed persistence for normalized events, queue state, and local drafts as follow-up work — not yet present. + +## Dependencies + +- `crate::openhuman::memory` — `ApiEnvelope`, `ApiMeta`, `EmptyRequest` (response envelope shape + empty-request type). +- `crate::rpc::RpcOutcome` — RPC return contract. +- `crate::core::all` — `RegisteredController`, `ControllerFuture` (controller registry wiring). +- `crate::core::{ControllerSchema, FieldSchema, TypeSchema}` — controller schema types. +- External crates: `serde` / `serde_json`, `uuid` (request ids), `tracing` (debug logging with `[provider-surfaces]` prefix). + +## Used by + +- `src/core/all.rs` — registers the controllers/schemas into the global registry (`all_provider_surfaces_registered_controllers`, `all_provider_surfaces_controller_schemas`, and a `"provider_surfaces"` dispatch arm). +- `src/openhuman/desktop_companion/handoff.rs` — reads `store::list_queue_items()` and matches `RespondQueueItem`s to correlate desktop companion handoff actions against the queue (light-touch, read-only against the store). +- `src/openhuman/task_sources/pipeline_tests.rs` — references the queue in tests. + +## Notes / gotchas + +- **Scaffold, not finished domain.** Docstrings across `mod.rs`/`ops.rs`/`store.rs` explicitly call this an initial cut: state is in-memory, SQLite store + drafts + provider-specific assistive actions are deferred. +- **`rpc.rs` is empty (docstring only).** Despite the canonical module shape suggesting `rpc.rs` holds the pure-domain API, here handlers live in `schemas.rs` delegating to `ops.rs`; `rpc.rs` is a placeholder. +- **Queue id is deterministic** (`provider:account_id:event_kind:entity_id`), so re-ingesting the same entity upserts (removes + re-prepends) rather than duplicating. +- **Process-global mutable state** means tests must serialize around `RESPOND_QUEUE`; `ops.rs` tests use a `TEST_MUTEX` + `store::clear_queue()` to avoid interleaving under cargo's parallel runner. Mutex poisoning is recovered via `into_inner()`. +- **Snake_case contract is intentional and shared** between request (`ProviderEvent`) and response (`RespondQueueItem`) so callers see one consistent shape. diff --git a/src/openhuman/redirect_links/README.md b/src/openhuman/redirect_links/README.md new file mode 100644 index 000000000..6c210c346 --- /dev/null +++ b/src/openhuman/redirect_links/README.md @@ -0,0 +1,82 @@ +# redirect_links + +Redirect-link shortener for token-heavy URLs. Long tracking URLs (e.g. `trip.com/forward/...?bizData=...`) burn model tokens every time they pass through a prompt. This domain encodes them to a short `openhuman://link/` placeholder on inbound text, keeps the full URL in a local SQLite store, and expands the placeholder back to the original URL on outbound messages so the user never sees the placeholder. It also has a separate helper for tagging public `openhm.xyz` short links with a `?u=` attribution param on the way out. + +## Responsibilities + +- **Shorten** a long URL to a content-addressed `openhuman://link/` form and persist it (idempotent / deterministic by URL). +- **Expand** a short id back to its full URL, bumping a hit counter + `last_used_at`. +- **Inbound rewrite**: replace every long URL (≥ `min_len`, default 80) in a text blob with its placeholder, preserving surrounding prose and trailing sentence punctuation. +- **Outbound rewrite**: replace every `openhuman://link/` placeholder in text back to the stored URL; unknown ids are left untouched (nothing silently disappears). +- **Public-link attribution**: append `?u=` (URL-encoded, idempotent, fragment-safe) to `openhm.xyz/` URLs, guarding against lookalike domains. +- List and remove stored links; expose all of the above over JSON-RPC. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/redirect_links/mod.rs` | Export-focused: module docstring, `mod` decls, `pub use` re-exports of ops + schemas + types; aliases `ops as rpc`. | +| `src/openhuman/redirect_links/types.rs` | Serde types: `RedirectLink`, `RewriteReplacement`, `RewriteResult`. | +| `src/openhuman/redirect_links/ops.rs` | Business logic: URL/short-URL/public-URL regexes, inbound/outbound rewrite, `append_user_id_to_public_links`, and the `rl_*` RPC handlers returning `RpcOutcome`. Holds `DEFAULT_MIN_URL_LEN = 80`. | +| `src/openhuman/redirect_links/store.rs` | SQLite persistence: `shorten`/`expand`/`peek`/`list`/`remove`, content-addressed id allocation (SHA-256 hex prefix), schema bootstrap, id<->short-URL helpers (`short_url_for`, `id_from_short`, `SHORT_URL_PREFIX`). | +| `src/openhuman/redirect_links/schemas.rs` | Controller schemas (`all_controller_schemas`, `all_registered_controllers`, `schemas`) + `handle_*` fns delegating to `ops.rs`. | + +## Public surface + +Re-exported from `mod.rs`: + +- Functions (via `pub use ops::…`): `shorten_url`, `expand_link`, `rewrite_inbound`, `rewrite_outbound`, `rewrite_outbound_for_user`, `append_user_id_to_public_links`. +- `pub use ops as rpc` — exposes the `rl_*` handlers under `redirect_links::rpc`. +- Schemas: `all_redirect_links_controller_schemas`, `all_redirect_links_registered_controllers`, `redirect_links_schemas`. +- Types: `RedirectLink`, `RewriteReplacement`, `RewriteResult`. + +Also defined (not re-exported through `mod.rs`): `ops::rewrite_inbound_with_threshold`, `ops::DEFAULT_MIN_URL_LEN`; `store::{shorten, expand, peek, list, remove, short_url_for, id_from_short, SHORT_URL_PREFIX}`. + +## RPC / controllers + +Namespace `redirect_links` (RPC methods `openhuman.redirect_links_`): + +| Function | Inputs | Output | +| --- | --- | --- | +| `shorten` | `url: String` | `link: RedirectLink` | +| `expand` | `id: String` | `link: RedirectLink` (errors if not found) | +| `list` | `limit?: u64` (default 50, max 1000) | `links: RedirectLink[]` (newest first) | +| `remove` | `id: String` | `{ id, removed: bool }` | +| `rewrite_inbound` | `text: String`, `min_len?: u64` (default 80) | `result: RewriteResult` | +| `rewrite_outbound` | `text: String` | `result: RewriteResult` | + +Handlers load config via `config::rpc::load_config_with_timeout()` and return `RpcOutcome` serialized with `into_cli_compatible_json()`. + +## Persistence + +SQLite DB at `{config.workspace_dir}/redirect_links/links.db`, table `redirect_links`: + +| Column | Notes | +| --- | --- | +| `id` | TEXT PRIMARY KEY — SHA-256(url) hex prefix, 8 chars default, grown by 2 up to 32 on prefix collision with a different URL. | +| `url` | TEXT NOT NULL UNIQUE (indexed). | +| `created_at` | RFC3339 TEXT. | +| `last_used_at` | RFC3339 TEXT, nullable; set on each expand. | +| `hit_count` | INTEGER, bumped on each expand. | + +Insert is atomic (`ON CONFLICT DO NOTHING`) so concurrent shortens of the same URL converge on one id with no PRIMARY KEY / UNIQUE error (regression-tested). The connection is opened per call and the schema is created if missing. + +## Dependencies + +- `crate::openhuman::config::Config` — supplies `workspace_dir` for the DB path; handlers call `config::rpc::load_config_with_timeout`. +- `crate::core::all` (`ControllerFuture`, `RegisteredController`) and `crate::core::{ControllerSchema, FieldSchema, TypeSchema}` — controller registry wiring. +- `crate::rpc::RpcOutcome` — RPC return envelope. +- External crates: `rusqlite` (SQLite), `sha2` + `hex` (content-addressed ids), `regex` (URL matching), `chrono` (timestamps), `urlencoding` (user-id encoding), `serde`/`serde_json`, `anyhow`. + +## Used by + +- `src/core/all.rs` registers the controllers + schemas and maps the `"redirect_links"` namespace description; `src/openhuman/mod.rs` declares the module. No other in-tree Rust callers of the rewrite/shorten functions were found — the rewrite pipeline is currently reachable via RPC rather than wired into an inbound/outbound message path inside the core. + +## Notes / gotchas + +- **Ids are content-addressed, not random**: same URL → same id (deterministic, deduped). Removing a link and re-shortening the same URL yields the same id again. +- **Length threshold guards token waste**: the placeholder is ~24 bytes, so URLs below `DEFAULT_MIN_URL_LEN` (80) are left untouched by inbound rewrite. +- **Trailing punctuation handling**: inbound rewrite and public-link tagging strip trailing `. , ; : !` so prose like "see https://…/path." doesn't capture the period into the stored/tagged URL. +- **`append_user_id_to_public_links` is anchored to `openhm.xyz`** specifically and rejects lookalikes (`evil-openhm.xyz`, `openhm.xyz.evil.com`); it splits off `#fragment` so `?u=` always lands in the query, and is idempotent against existing `?u=`/`&u=`. +- **`id_from_short`** accepts both `openhuman://link/` and a bare hex ``, lowercasing the result; non-hex input returns `None`. +- **No agent tools, no event-bus subscribers, no `bus.rs`/`tools.rs`** — this domain is store + ops + RPC only. diff --git a/src/openhuman/referral/README.md b/src/openhuman/referral/README.md new file mode 100644 index 000000000..82f5514c8 --- /dev/null +++ b/src/openhuman/referral/README.md @@ -0,0 +1,68 @@ +# referral + +Thin RPC adapter domain for the referral program. It does **not** own any business logic, state, or schema of its own — it makes authenticated `reqwest` calls to the hosted backend's `/referral/*` endpoints and surfaces the raw `data` payloads to the CLI / JSON-RPC clients. It exists primarily because the desktop WebView `fetch` to the backend can fail with a generic "Load failed" (CORS / TLS / WebKit), so these ops reuse the same server-side `reqwest` path as the billing domain. + +## Responsibilities + +- Fetch referral stats (code, link, totals, referred-user rows) via `GET /referral/stats`. +- Claim a referral code for the current user via `POST /referral/claim`, with an optional device fingerprint for abuse signals. +- Resolve and require a backend session token before any call; fail closed with a clear error when no token is stored. +- Trim the referral `code`; trim and drop whitespace-only `deviceFingerprint` before forwarding. +- Wrap backend responses in `RpcOutcome` with grep-friendly log lines. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/referral/mod.rs` | Export-only. Re-exports `ops::*` and the schema/controller pair (`all_referral_controller_schemas`, `all_referral_registered_controllers`, `referral_schemas`). | +| `src/openhuman/referral/ops.rs` | Business logic: `require_token`, `get_stats`, `claim_referral`. Builds a `BackendOAuthClient` against the effective backend URL and issues authed JSON requests. Includes inline tests against an Axum mock backend. | +| `src/openhuman/referral/schemas.rs` | Controller schemas + `handle_*` fns that load config and delegate to `ops`. Defines `ReferralClaimParams` (camelCase deserialization) and helpers (`to_json`, `deserialize_params`, `json_output`). | + +## Public surface + +From `mod.rs` re-exports: + +- `get_stats(config: &Config) -> Result, String>` (via `ops::*`). +- `claim_referral(config: &Config, code: &str, device_fingerprint: Option<&str>) -> Result, String>` (via `ops::*`). +- `all_referral_controller_schemas() -> Vec`. +- `all_referral_registered_controllers() -> Vec`. +- `referral_schemas(function: &str) -> ControllerSchema`. + +(`require_token` is a private helper.) + +## RPC / controllers + +Two controllers in the `referral` namespace, registered into the global registry via `src/core/all.rs`: + +| Method | Inputs | Output | Backend call | +| --- | --- | --- | --- | +| `referral_get_stats` (`referral.get_stats`) | none | `stats` (JSON) | `GET /referral/stats` | +| `referral_claim` (`referral.claim`) | `code` (string, required), `deviceFingerprint` (string, optional) | `result` (JSON) | `POST /referral/claim` | + +An unrecognized `function` name returns an `unknown` placeholder schema with an `error` output. Handlers load `Config` via `config_rpc::load_config_with_timeout()` and return CLI-compatible JSON through `RpcOutcome::into_cli_compatible_json()`. + +## Persistence + +None of its own. The domain is stateless — it reads the backend session token from the credentials store via `get_session_token` but does not persist anything. + +## Dependencies + +- `crate::api::config::effective_backend_api_url` — resolves the effective backend API base URL from `config.api_url`. +- `crate::api::jwt::get_session_token` — reads the stored backend session token. +- `crate::api::BackendOAuthClient` — issues authenticated JSON requests (`authed_json`) to the backend. +- `crate::openhuman::config::Config` — config struct passed into ops; `config::rpc::load_config_with_timeout` is used by the schema handlers. +- `crate::core::all::{ControllerFuture, RegisteredController}` and `crate::core::{ControllerSchema, FieldSchema, TypeSchema}` — controller registry types. +- `crate::rpc::RpcOutcome` — return wrapper carrying value + logs. +- Test-only: `crate::openhuman::credentials::{AuthService, APP_SESSION_PROVIDER, DEFAULT_AUTH_PROFILE_NAME}` for seeding session tokens in unit tests. + +## Used by + +- `src/core/all.rs` — registers `all_referral_registered_controllers()` into the controller registry (line ~213) and `all_referral_controller_schemas()` into the schema list (line ~345), exposing both methods to CLI and JSON-RPC. + +## Notes / gotchas + +- No `types.rs`, `store.rs`, `tools.rs`, or `bus.rs` — this is a pure RPC adapter, not a stateful domain. No agent tools, no event-bus subscribers. +- Both ops fail closed when no session token is stored, with the error `"no backend session token; run auth_store_session first"`. +- Eligibility for `claim` ("only users who have not yet subscribed") is enforced **by the backend**, not in this module — it merely forwards the request. +- Trimming/whitespace-dropping of `deviceFingerprint` happens in both `ops::claim_referral` and the schema handler `handle_referral_claim` (defensive, redundant filtering). +- The module deliberately mirrors the billing domain's server-side `reqwest` path to avoid WebView `fetch` "Load failed" failures. diff --git a/src/openhuman/routing/README.md b/src/openhuman/routing/README.md new file mode 100644 index 000000000..bce6143dd --- /dev/null +++ b/src/openhuman/routing/README.md @@ -0,0 +1,89 @@ +# routing + +Intelligent model routing — a policy-driven layer that sits between callers (agent harness, channels, tools) and the concrete inference providers, deciding per request whether to run inference on a **local** model server or the **remote** OpenHuman backend. It classifies each request by task complexity (from `hint:*` model strings), checks cached local-model health, applies privacy/latency/cost hints, dispatches to the chosen backend, and transparently falls back to remote when a local call fails or returns a low-quality response. It is not an RPC-facing domain — it exposes no controllers, agent tools, event-bus subscribers, or persisted state; its only output side-effect is structured telemetry via `tracing`. + +## Responsibilities + +- Classify a model string (possibly `hint:*`) into a `TaskCategory` (`Lightweight` / `Medium` / `Heavy`). +- Produce a deterministic routing decision `(primary, fallback)` from task category, local availability, and per-call routing hints (privacy, latency budget, cost sensitivity). +- Construct an `IntelligentRoutingProvider` wrapping a remote provider plus a locally-resolved OpenAI-compatible provider (Ollama / LM Studio / llama.cpp / custom OpenAI), honoring the `OPENHUMAN_LOCAL_INFERENCE_URL` env override. +- Probe and cache local model-server health (`GET {base}/api/tags` for Ollama, `GET {base}/models` for OpenAI-compat backends) with a 30 s TTL and 3 s probe timeout. +- On a local primary: dispatch locally, and on error **or** low-quality output retry on remote — unless `privacy_required` forbids leaving the device. +- Heuristically score local responses for low quality (length floor, empty-noise utterances, refusal phrases) to drive fallback. +- Normalize heavy `hint:*` model strings to backend-valid model IDs (`reasoning` → `MODEL_REASONING_V1`, `chat` → `MODEL_REASONING_QUICK_V1`, `agentic` → `MODEL_AGENTIC_V1`, `coding` → `MODEL_CODING_V1`). +- Force remote when native tool-calling is required (tools present) and refuse to silently bypass local routing for streaming. +- Emit a structured `RoutingRecord` (category, target, resolved model, health, fallback flag, latency, tokens, cost) per completed call. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/routing/mod.rs` | Module docstring + `pub mod` decls and `pub use` re-exports of the public surface. | +| `src/openhuman/routing/policy.rs` | `TaskCategory`, `RoutingTarget`, `RoutingHints` (`LatencyBudget`, `CostSensitivity`, `privacy_required`); `classify()` (hint → category) and `decide()` (the pure routing-decision function). Inline tests. | +| `src/openhuman/routing/factory.rs` | `new_provider()` — resolves local provider kind/base URL/health probe from `LocalAiConfig` + env override and assembles an `IntelligentRoutingProvider`. Inline tests. | +| `src/openhuman/routing/provider.rs` | `IntelligentRoutingProvider` — `impl Provider`; resolves targets, dispatches `chat_with_system` / `chat_with_history` / `chat` / streaming, performs fallback, and emits telemetry. Tests in sibling `provider_tests.rs`. | +| `src/openhuman/routing/health.rs` | `LocalHealthChecker` — async, `parking_lot::Mutex`-cached health probe (lock never held across `await`). Inline tests. | +| `src/openhuman/routing/quality.rs` | `is_low_quality()` — allocation-free hot-path heuristic over a `LazyLock` Aho-Corasick refusal DFA, empty-noise token list, and length gate. Inline tests. | +| `src/openhuman/routing/telemetry.rs` | `RoutingRecord` + `emit()` — structured `tracing::info!`/`warn!` under target `"routing"`. Inline tests. | +| `src/openhuman/routing/provider_tests.rs` | Sibling test module for `provider.rs` (`#[path = ...]`). | + +## Public surface + +Re-exported from `mod.rs`: + +- `factory::new_provider` — build an `IntelligentRoutingProvider` from a remote provider + `LocalAiConfig`. +- `health::LocalHealthChecker` — cached local-server health checker. +- `policy::{classify, decide, RoutingTarget, TaskCategory}` — classification + decision primitives. +- `provider::IntelligentRoutingProvider` — the `Provider` implementation. +- `quality::is_low_quality` — response-quality heuristic. +- `telemetry::{emit as emit_routing_record, RoutingRecord}` — telemetry record + emitter. + +Not re-exported but public within the module: `policy::{RoutingHints, LatencyBudget, CostSensitivity}` and `IntelligentRoutingProvider::with_hints`. + +## RPC / controllers + +None. This module exposes no `schemas.rs`, no `all_*_controller_schemas`, and no `openhuman.routing_*` RPC methods. It is consumed in-process by the inference layer. + +## Agent tools + +None (no `tools.rs`). + +## Events + +None published or subscribed (no `bus.rs`). The only observability output is `tracing` telemetry under target `"routing"` (see `telemetry.rs`). + +## Persistence + +None (no `store.rs`). The only state is the in-memory, TTL'd health cache inside `LocalHealthChecker` (`parking_lot::Mutex>`), which is ephemeral and not persisted. + +## Configuration + +- `LocalAiConfig` (`openhuman::config`): `runtime_enabled`, `provider`, `base_url`, `api_key`, `chat_model_id` drive local-provider selection and whether local routing is active at all. +- `OPENHUMAN_LOCAL_INFERENCE_URL` (env): full `/v1` base URL of a local OpenAI-compatible server; when set, takes precedence over `config.base_url` and switches the health probe to `GET {base}/models`. +- Model-ID constants from `openhuman::config`: `MODEL_REASONING_V1`, `MODEL_REASONING_QUICK_V1`, `MODEL_AGENTIC_V1`, `MODEL_CODING_V1`. + +## Dependencies + +- `openhuman::config` — `LocalAiConfig` and backend model-ID constants used for local-provider resolution and heavy-hint normalization. +- `openhuman::inference::provider` — `Provider` trait + `traits::*` (`ChatRequest`, `ChatResponse`, `ChatMessage`, `StreamChunk`/`StreamError`/`StreamOptions`/`StreamResult`, `ProviderCapabilities`, `ToolsPayload`); the routing provider implements `Provider` and wraps two `Box`s. +- `openhuman::inference::provider::compatible` — `OpenAiCompatibleProvider` + `AuthStyle` used to build the local provider in `factory.rs`. +- `openhuman::inference::local` — `ollama_base_url`, `lm_studio::lm_studio_base_url_from_local_ai`, `provider::normalize_provider` for base-URL/provider-kind resolution. +- `openhuman::tools` — `ToolSpec` (passed through `convert_tools` and to detect tool presence forcing remote routing). +- `openhuman::util` — `floor_char_boundary` for safe UTF-8 truncation of log previews. +- External crates: `reqwest` (health probe), `parking_lot` (cache mutex), `aho-corasick` (refusal DFA), `async-trait`, `futures-util`, `anyhow`, `tracing`. + +## Used by + +- `src/openhuman/inference/provider/ops.rs` — calls `crate::openhuman::routing::new_provider(...)` to wrap the backend provider with intelligent routing. +- `src/openhuman/agent/triage/routing.rs` — references the routing wiring (mirrors `routing::factory::new_provider`'s local arm). + +## Notes / gotchas + +- **`hint:chat` goes remote, not local.** Despite being the front-line conversational tier, `classify()` maps it (and any unrecognized `hint:*` or exact model name) to `Heavy`, which always routes remote — the local model is too slow for the TTFT budget that motivated the hint. +- **`privacy_required` fails closed.** It forces local routing with no remote fallback for *every* category (including heavy and when local is unhealthy), and disables streaming (`supports_streaming()` returns `false`). +- **Local streaming is intentionally unsupported.** If policy selects local for a streaming call, the provider returns a single `StreamError::Provider` chunk rather than silently delegating to remote (avoids bypassing privacy/local routing). +- **Tools force remote.** When a `chat` request carries tools and policy chose local, routing overrides to remote (`remote_fallback_model`) because local lacks native tool calling. +- **Fallback also triggers on low quality, not just errors** (`should_fallback` / `is_low_quality`) — the heuristic is deliberately conservative toward flagging low quality, since serving a refusal is more user-visible than an extra remote call. +- **Health cache lock discipline:** `LocalHealthChecker` never holds the `Mutex` across an `await`; it reads/writes the cache, releases, then probes. +- **Token/cost telemetry is only populated for `chat()`** (which carries `ChatResponse.usage`); `chat_with_system` and `chat_with_history` emit records with zeroed token/cost fields. +- Medium tasks default to **remote** unless at least one local-bias hint is set (`LatencyBudget::Low` or `CostSensitivity::High`); lightweight tasks are local-first when local is healthy. diff --git a/src/openhuman/runtime_node/README.md b/src/openhuman/runtime_node/README.md new file mode 100644 index 000000000..80db4b799 --- /dev/null +++ b/src/openhuman/runtime_node/README.md @@ -0,0 +1,102 @@ +# runtime_node + +Managed **Node.js runtime** for the core, plus a thin **tool bridge** that lists and dispatches agent-callable tools through a `javascript` RPC namespace. The runtime half resolves or installs a pinned Node.js toolchain (reusing a compatible host `node` when present, otherwise downloading + SHA-256-verifying + extracting an official distribution from nodejs.org) so that `node_exec` / `npm_exec` / `shell` tools and Node-dependent skills have a trusted `node`/`npm` on a stable path. The bridge half exposes the full agent tool registry over JSON-RPC under `javascript.*` so callers (e.g. an embedded JS host) can enumerate and run tools by name. The public-facing language slot is the sibling [`javascript`](../javascript/) module, which re-exports this module's surface under `javascript`-prefixed names. + +## Responsibilities + +- Detect a compatible system `node` on `PATH` (major-version match) and verify `npm` is also usable before reusing it. +- Resolve/install a managed Node.js toolchain when no compatible system node exists: pick the host archive, fetch `SHASUMS256.txt`, download with streaming SHA-256 verification, extract (`.tar.xz` / `.zip`), and atomically install into a user-owned cache root. +- Memoise the resolved toolchain behind a `tokio::sync::Mutex` so concurrent callers never race the download/extract/install pipeline; offer a non-blocking `try_cached()` peek for transparent `PATH` injection. +- Build the full agent tool registry on demand and expose two RPC controllers: list tool metadata, and execute a named tool returning an MCP-style `ToolResult`. +- Publish `ToolExecutionStarted` / `ToolExecutionCompleted` domain events around bridge tool execution. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/runtime_node/mod.rs` | Export-focused: submodule decls + `pub use` re-exports, including `all_runtime_node_controller_schemas` / `all_runtime_node_registered_controllers`. | +| `src/openhuman/runtime_node/resolver.rs` | Synchronous system-node probe. `detect_system_node`, `parse_node_version`, `SystemNode`. `PATH` walk with execute-bit filtering, `node --version` / `npm --version` probes with a 5s timeout. Major-version match only. | +| `src/openhuman/runtime_node/bootstrap.rs` | Orchestrator. `NodeBootstrap` (serialised + memoised `resolve()`, `try_cached()`), `ResolvedNode`, `NodeSource`. Picks system vs managed, computes the cache root (user cache by default, never workspace-local unless forced), guards against cache-root escape / spoofed installs via canonicalised `starts_with`. | +| `src/openhuman/runtime_node/downloader.rs` | `NodeDistribution` (host triple → archive name/URL), `fetch_shasums`, `download_distribution`. Streams to disk while hashing; **mandatory** SHA-256 match or the partial file is deleted. | +| `src/openhuman/runtime_node/extractor.rs` | `extract_distribution` (`.tar.xz` via `xz2`+`tar`, `.zip` via `zip`, both in `spawn_blocking`), `atomic_install` (rename into place with backup/restore). Asserts a single top-level folder per archive. | +| `src/openhuman/runtime_node/ops.rs` | Bridge logic: `build_runtime_tools` (assembles `SecurityPolicy`, audit logger, `NativeRuntime`, `Memory`, then `tools::all_tools_with_runtime`), `list_tools`, `execute_tool` (event publish + timing). | +| `src/openhuman/runtime_node/rpc.rs` | RPC param structs (`ListToolsParams`, `ExecuteToolParams`) and `*_handler` fns; loads config via `config::rpc` and delegates through the `javascript` alias, wrapping results in `RpcOutcome`. | +| `src/openhuman/runtime_node/schemas.rs` | Controller schemas + registered controllers for `javascript_list_tools` / `javascript_execute_tool`; `handle_*` deserialise params and call `rpc.rs`. | +| `src/openhuman/runtime_node/types.rs` | `RuntimeToolSummary`, `ExecuteToolOutcome` serde types. | + +## Public surface + +From `mod.rs` re-exports: + +- Bootstrap: `NodeBootstrap`, `NodeSource`, `ResolvedNode`. +- Downloader: `download_distribution`, `fetch_shasums`, `NodeDistribution`. +- Extractor: `atomic_install`, `extract_distribution`. +- Resolver: `detect_system_node`, `parse_node_version`, `SystemNode`. +- Bridge ops: `execute_tool`, `list_tools`. +- Types: `RuntimeToolSummary`, `ExecuteToolOutcome` (via `types`). +- Controller registry pair: `all_runtime_node_controller_schemas`, `all_runtime_node_registered_controllers`. + +## RPC / controllers + +Registered under namespace `javascript` (schemas wired into `src/core/all.rs` via the `javascript` module's `all_javascript_*` aliases, not under a `runtime_node` name): + +| Method | Inputs | Output | +| --- | --- | --- | +| `javascript.list_tools` | none | `tools`: array of tool metadata (name, description, category, permission_level, scope, supports_markdown, parameters). | +| `javascript.execute_tool` | `tool_name` (required), `args` (optional, defaults `{}`), `prefer_markdown` (optional bool) | `tool_name`, `elapsed_ms`, `result` (MCP-style `ToolResult`: `{content, is_error, markdownFormatted?}`). | + +Handlers load config via `config::rpc::load_config_with_timeout`, return `RpcOutcome` (`into_cli_compatible_json`). Unknown tool name → error `unknown tool \`\``. + +## Agent tools + +This module owns **no** tools of its own. Instead it builds the *entire* agent tool registry on demand (`tools::all_tools_with_runtime`) to back the `javascript.execute_tool` / `javascript.list_tools` bridge. The actual `node_exec`, `npm_exec`, and `shell` tools live in `src/openhuman/tools/impl/system/` and consume this module's `NodeBootstrap` for binary resolution / `PATH` injection. + +## Events + +`ops::execute_tool` publishes (via `core::event_bus::publish_global`) around each bridge invocation, with `session_id = "javascript"`: + +- `DomainEvent::ToolExecutionStarted` +- `DomainEvent::ToolExecutionCompleted` (with `success`, `elapsed_ms`) + +No event-bus subscribers (`bus.rs`) are defined. + +## Persistence + +No domain `store.rs`. The only on-disk state is the **managed Node.js install cache**, resolved by `NodeBootstrap::cache_root()` (first hit wins): + +1. Explicit `config.node.cache_dir` (honoured verbatim). +2. `dirs::cache_dir()/openhuman/node-runtime` — the default, user-owned. +3. `{workspace}/node-runtime/` — last-resort fallback (warned; less secure). + +Reads `NodeConfig` (`config.node`): `enabled`, `prefer_system`, `version`, `cache_dir` (env overrides like `OPENHUMAN_NODE_ENABLED`, `OPENHUMAN_NODE_VERSION` in config loader). + +## Dependencies + +- `crate::openhuman::config` (`Config`, `schema::NodeConfig`, `rpc`) — runtime config, version/cache settings, RPC config loading. +- `crate::openhuman::tools` (`Tool`, `ToolCallOptions`, `ToolScope`, `all_tools_with_runtime`) — the registry the bridge enumerates/executes. +- `crate::openhuman::security` (`SecurityPolicy`, workspace audit logger) — built per `build_runtime_tools` call to scope tool capability. +- `crate::openhuman::agent::host_runtime` (`NativeRuntime`, `RuntimeAdapter`) — runtime adapter injected into tool construction. +- `crate::openhuman::memory` / `memory_store` (`Memory`, `create_memory_with_local_ai`) — memory backend wired into memory-aware tools. +- `crate::openhuman::skills::types::ToolResult` — result envelope returned by `execute_tool`. +- `crate::openhuman::javascript` — the language-slot alias module the `rpc.rs` handlers call through (`list_tools` / `execute_tool`). +- `crate::core::event_bus` (`publish_global`, `DomainEvent`) — tool execution events. +- `crate::core::all` (`ControllerFuture`, `RegisteredController`) + `crate::core` (`ControllerSchema`, `FieldSchema`, `TypeSchema`) + `crate::rpc::RpcOutcome` — RPC controller plumbing. + +External crates: `reqwest`, `sha2`, `hex`, `xz2`, `tar`, `zip`, `tokio`, `wait_timeout`, `dirs`, `anyhow`, `serde`/`serde_json`, `tracing`, `async-trait`. + +## Used by + +- `src/openhuman/javascript/mod.rs` — re-exports this entire surface under `javascript`-prefixed names (the public language slot). +- `src/openhuman/tools/impl/system/{node_exec,npm_exec,shell}.rs` — hold an `Arc`; `node_exec`/`npm_exec` call `resolve()`, `shell` uses non-blocking `try_cached()` for transparent `PATH` injection. +- `src/openhuman/tools/ops.rs` and `src/openhuman/agent/tools/delegate_to_personality.rs` — reference the bootstrap/runtime surface. +- `src/openhuman/runtime_python/bootstrap.rs` — a sibling runtime modeled on the same pattern. +- `src/core/all.rs` — registers the `javascript.*` controllers via the `javascript` aliases. + +## Notes / gotchas + +- **Naming asymmetry**: the module is `runtime_node` but its RPC namespace and public aliases are `javascript`. The `javascript` module is a deliberate language-slot indirection so a future backend (or `python`/`ruby`) can swap in without churning callers. +- **`build_runtime_tools` is not cheap**: each `list_tools`/`execute_tool` call rebuilds the full tool registry (security policy, audit logger, memory backend) from `Config`. There is no caching at the bridge layer — the memoisation in `bootstrap.rs` is only for Node toolchain resolution, not for tool construction. +- **Integrity is load-bearing, no opt-out**: downloads must match the official `SHASUMS256.txt` digest or the archive is deleted and the call fails. `probe_managed_install` canonicalises and requires the install to live under the resolved cache root to defeat a workspace-vendored fake `node-v*/` tree (PR #723 finding). +- **System-node reuse requires npm**: a compatible `node` with a missing/broken `npm` is rejected so the managed path can supply a complete toolchain (distros that split `nodejs`/`npm`). +- **Version match is major-only** (`parse_node_version`); point releases are accepted. Set `node.prefer_system = false` for strict pinning, or `node.enabled = false` to disable the runtime entirely (then `resolve()` bails). +- **Self-healing cache**: a managed install missing `npm` (e.g. download interrupted after `node` extracted) is treated as unusable and reinstalled rather than reused forever. diff --git a/src/openhuman/runtime_python/README.md b/src/openhuman/runtime_python/README.md new file mode 100644 index 000000000..326026b49 --- /dev/null +++ b/src/openhuman/runtime_python/README.md @@ -0,0 +1,78 @@ +# runtime_python + +Managed Python runtime for Python-backed integrations. This domain owns interpreter discovery and process-launch primitives so callers don't need to care whether Python came from the host or a managed standalone CPython distribution. The immediate use case is launching stdio MCP servers implemented in Python. The shipped intent is a managed CPython distribution downloaded from `astral-sh/python-build-standalone`; a system-interpreter probe is a compatibility / developer override path. + +## Responsibilities + +- Resolve a Python ≥ `minimum_version` (default `3.12.0`) interpreter, memoizing the first success. +- Optionally probe host `PATH` for a compatible interpreter (`prefer_system`). +- Download, SHA-256-verify, extract, and atomically install a managed standalone CPython distribution when no system Python is used/available. +- Spawn line-oriented (unbuffered) Python child processes for stdio protocols such as MCP. +- Read its behavior from `[runtime_python]` config (`RuntimePythonConfig`). + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/runtime_python/mod.rs` | Export-focused module root; module docstring + `pub use` re-exports of the public surface. | +| `src/openhuman/runtime_python/bootstrap.rs` | Orchestrator. `PythonBootstrap` ties resolve → (system probe \| managed install) → memoized `ResolvedPython`; exposes `spawn_stdio`. Holds the per-install file lock, cache-root selection, and managed-install probing. | +| `src/openhuman/runtime_python/resolver.rs` | System Python discovery. Walks candidate commands / `PATH`, probes `--version` (5s timeout), parses semver, enforces the minimum-version floor. | +| `src/openhuman/runtime_python/downloader.rs` | Managed distribution fetch. Queries the GitHub releases API, selects a host-compatible `install_only` asset ≥ minimum, downloads and verifies the published SHA-256 digest. | +| `src/openhuman/runtime_python/extractor.rs` | `tar.gz` extraction (gzip via `flate2`, preserves perms) + `atomic_install` (rename-into-place with backup/rollback). | +| `src/openhuman/runtime_python/process.rs` | `PythonLaunchSpec` + `spawn_stdio_process`: builds a `tokio::process::Command` with piped stdio, `-u`, `kill_on_drop`. | +| `src/openhuman/runtime_python/bootstrap_tests.rs` | Tests for the bootstrap orchestrator (`#[path]`-included). | +| `src/openhuman/runtime_python/resolver_tests.rs` | Tests for version parsing / system detection. | +| `src/openhuman/runtime_python/downloader_tests.rs` | Tests for release-metadata parse and asset selection. | + +## Public surface + +Re-exported from `mod.rs`: + +- `bootstrap`: `PythonBootstrap`, `PythonSource` (`System` / `Managed`), `ResolvedPython` (`python_bin`, `version`, `source`). +- `downloader`: `fetch_release_metadata`, `select_distribution`, `PythonDistribution`. +- `extractor`: `atomic_install`, `extract_distribution`. +- `process`: `PythonLaunchSpec`. +- `resolver`: `detect_system_python`, `parse_python_version`, `PythonVersion`, `SystemPython`. + +Primary entry points: `PythonBootstrap::new(config)`, `.resolve() -> Result`, `.try_cached()`, `.spawn_stdio(&PythonLaunchSpec) -> Result`. + +## RPC / controllers + +None. This domain exposes no JSON-RPC controllers, schemas, or `handle_*` functions — it is a library used by other domains, not directly addressable over RPC. + +## Agent tools + +None. No `tools.rs`; the module owns no agent tools. + +## Events + +None. No `bus.rs`; the module neither publishes nor subscribes to `DomainEvent`s. + +## Persistence + +No structured domain store. Side effects on disk: + +- Managed CPython installs land under the cache root: `config.cache_dir` if set, else `/openhuman/runtime-python`, else `.openhuman/runtime-python`. +- Per-install exclusive file lock at `.lock` (`fs2`) serializes concurrent installs. +- Staging dirs (`.stage--`) and the downloaded archive are removed after a successful atomic install. Existing installs are moved aside to `.old-` and restored on rename failure. + +## Dependencies + +- `crate::openhuman::config::schema::RuntimePythonConfig` — the only intra-crate dependency; drives `enabled`, `minimum_version`, `cache_dir`, `managed_release_tag`, `prefer_system`, `preferred_command`. + +External crates: `reqwest` (HTTP), `serde` (release metadata), `sha2`/`hex` (digest verify), `flate2`/`tar` (extraction), `tokio` (async fs/process + `Mutex`), `fs2` (file lock), `walkdir` (interpreter discovery), `wait-timeout` (version probe timeout), `uuid`, `dirs`, `anyhow`, `tracing`. + +## Used by + +Referenced from `src/openhuman/mod.rs` (module declaration) and surfaced in the capability catalog (`src/openhuman/about_app/catalog.rs`). Config wiring lives in `src/openhuman/config/schema/{runtime_python.rs,types.rs,load.rs,mod.rs}`. No other domain currently constructs `PythonBootstrap` directly in `src/` outside this wiring — the intended consumer is Python-backed integrations such as stdio MCP servers. + +## Notes / gotchas + +- `resolve()` is memoized: the first successful `ResolvedPython` is cached behind a `tokio::Mutex` and returned to all later callers; `try_cached()` peeks without probing. +- When `config.enabled == false`, `resolve()` bails — callers must skip Python-backed features rather than fall back. +- Managed install is only attempted when `prefer_system` is off or the system probe finds nothing compatible; the system path returns `PythonSource::System`, managed returns `PythonSource::Managed`. +- Host-asset selection prefers `install_only_stripped` assets when available; only the platform triples enumerated in `host_asset_suffix` are supported (macOS/Linux/Windows × x86_64/aarch64) — other hosts error. +- Download verification: if release metadata lacks a `digest`, SHA-256 verification is **skipped** with a warning rather than failing. +- The version probe runs ` --version` with a 5s timeout, `CREATE_NO_WINDOW` on Windows, and reads version output from stdout or (fallback) stderr. +- `spawn_stdio_process` defaults to `-u` (unbuffered) and `kill_on_drop(true)` so dropped children don't leak; stdin/stdout/stderr are all piped. +- `bootstrap.rs` defines a private `install_managed()` wrapper that is currently unused by the public path (`resolve()` calls `install_managed_from_api` directly). diff --git a/src/openhuman/scheduler_gate/README.md b/src/openhuman/scheduler_gate/README.md new file mode 100644 index 000000000..5486bc892 --- /dev/null +++ b/src/openhuman/scheduler_gate/README.md @@ -0,0 +1,73 @@ +# scheduler_gate + +Gates background AI work (memory-tree digests, embeddings, summarisation, triage, reflection, local inference) on live host conditions so the process doesn't make the machine visibly lag — especially on battery. It exposes a single process-wide decision point: background workers consult `current_policy()` for a cheap read, or `await wait_for_capacity()` to cooperatively block until the host is ready and hold a slot in a one-permit LLM semaphore. A background sampler refreshes host signals every 30s and recomputes the policy. A separate "signed out" override trumps everything to halt LLM work the moment the app session goes away. + +## Responsibilities + +- Sample host signals on a 30s cadence: power state (on AC / battery charge), recent global CPU usage, and deployment mode (server/container). +- Turn signals + user config into a `Policy` tier: `Aggressive` (server / always-on), `Normal` (desktop with headroom), `Throttled` (busy or on battery), `Paused` (user opted out, on battery with `require_ac_power`, CPU pressure, or signed out). +- Enforce a **process-wide single-slot LLM semaphore** so concurrent local-Ollama / bge-m3 calls can't saturate laptop RAM. +- Provide cooperative backoff: `wait_for_capacity()` resolves immediately in `Aggressive`/`Normal`, sleeps `throttled_backoff_ms` in `Throttled`, and re-polls every `paused_poll_ms` in `Paused` so callers resume the instant the gate flips back on. +- Expose a signed-out kill switch (`set_signed_out` / `is_signed_out`) consumed by the credentials lifecycle and 401-detection sites to stand down all background LLM work. +- Clamp out-of-domain config thresholds defensively so a malformed `config.toml` can't silently disable or force-throttle work. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/scheduler_gate/mod.rs` | Module docstring + re-exports of the public surface. | +| `src/openhuman/scheduler_gate/gate.rs` | Process-wide singleton: cached `State` (config + signals + policy), the 30s sampler task, the single-slot LLM semaphore, `LlmPermit` RAII guard, signed-out override, and `init_global`/`update_config`/`current_policy`/`current_signals`/`wait_for_capacity`. Holds the per-tokio-runtime test-state scaffolding. | +| `src/openhuman/scheduler_gate/policy.rs` | Pure decision logic: `decide(signals, cfg) -> Policy`, the `Policy` and `PauseReason` enums, and their `as_str` / `pause_reason` helpers. Evaluation order: user mode override → server mode → power-aware stand-down → hard CPU ceiling → battery/CPU headroom. | +| `src/openhuman/scheduler_gate/signals.rs` | `Signals` snapshot + `Signals::sample()`. Probes battery (via `starship_battery`), CPU usage (via `sysinfo`, two-refresh delta), and detects server/container mode. Honours `OPENHUMAN_ON_AC_POWER`, `OPENHUMAN_BATTERY_CHARGE`, `OPENHUMAN_DEPLOYMENT` env overrides plus Kubernetes / `/.dockerenv` heuristics. | + +## Public surface + +From `mod.rs`: + +- **Functions** (`gate`): `init_global(&Config)`, `current_policy() -> Policy`, `current_signals() -> Signals`, `wait_for_capacity() -> Option`, `is_signed_out() -> bool`, `set_signed_out(bool)`. +- **Types**: `LlmPermit` (RAII semaphore guard, `#[must_use]`), `Policy` (`Aggressive` / `Normal` / `Throttled` / `Paused { reason }`), `PauseReason` (`UserDisabled` / `OnBattery` / `CpuPressure` / `SignedOut` / `Unknown`), `Signals`. +- Not re-exported but `pub` on `gate`: `update_config(SchedulerGateConfig)`. +- Test-only: `SignedOutTestGuard` (RAII flag snapshot/restore), `try_acquire_llm_permit`, `available_llm_permits`. + +## RPC / controllers + +None. This module exposes no JSON-RPC controllers, schemas, or `handle_*` functions — it is consulted in-process via direct function calls. + +## Agent tools + +None. + +## Events + +No `bus.rs` and no `DomainEvent` publish/subscribe of its own. It is invoked directly; the credentials domain (`credentials/bus.rs`, `credentials/ops.rs`) calls `set_signed_out` from session lifecycle handlers. + +## Persistence + +None. State is process-memory only (`OnceLock>>` + a process-wide `AtomicBool` signed-out flag and `OnceLock>`). It does not persist to disk; the signed-out flag is reseated from the on-disk session at startup by `init_global` callers / credentials code, not stored here. + +## Dependencies + +- `crate::openhuman::config` — reads `Config`, `SchedulerGateConfig` (the `[scheduler_gate]` block: `mode`, `battery_floor`, `cpu_busy_threshold_pct`, `cpu_severe_pct`, `throttled_backoff_ms`, `paused_poll_ms`, `require_ac_power`) and `SchedulerGateMode` (`Auto` / `AlwaysOn` / `Off`). +- External crates: `parking_lot` (RwLock/Mutex), `tokio::sync::Semaphore`, `sysinfo` (CPU), `starship_battery` (power probe), `once_cell` (lazy CPU `System`). + +No dependency on any other `openhuman` domain or on `crate::core::*`. + +## Used by + +Consumed in-process across the codebase (discoverable via `grep scheduler_gate`): + +- **Background workers / pipelines**: `memory/schema.rs`, `memory_queue/worker.rs`, `memory_tree/tree/rpc.rs`, `memory_sync/composio/periodic.rs`, `subconscious/engine.rs`, `learning/reflection.rs`, `autocomplete/core/engine.rs`, `task_sources/route.rs`, `agent/task_dispatcher.rs`, `agent/triage/evaluator.rs`. +- **Inference layer**: `inference/provider/openhuman_backend.rs`, `inference/provider/factory.rs`, `inference/local/service/{vision_embed.rs,public_infer.rs}`, `inference/voice/postprocess.rs`. +- **Credentials lifecycle** (signed-out kill switch): `credentials/ops.rs`, `credentials/bus.rs`. +- **Bootstrap / transport**: `core/jsonrpc.rs` (calls `init_global` during server bootstrap), `core/observability.rs`, plus the domain wiring in `openhuman/mod.rs` and config schema in `config/schema/scheduler_gate.rs`. + +## Notes / gotchas + +- **Single LLM slot is deliberate** (`LLM_SLOTS = 1`): concurrent local Ollama / bge-m3 calls (~1.3 GB resident each) have crashed the user's laptop. Cloud-backend calls bypass this semaphore at the worker layer because they're bandwidth-bound, not RAM-bound. +- **Backoff happens before semaphore acquisition** so a `Paused`/`Throttled` mode doesn't pile tasks into the semaphore wait queue — they sit in the policy poll loop instead. +- **`current_policy()` defaults to `Normal` and `wait_for_capacity()` acquires directly when `STATE` is uninitialised** (unit tests / pre-`init_global` bootstrap) so callers never deadlock on a sampler that will never start. +- **Signed-out override is gated on `STATE.get().is_some()`** on both the reader (`current_policy`/`wait_for_capacity`) and writer (`set_signed_out`) sides. Without this, a stale per-test `signed_out=true` flag (from `clear_session` / 401 / `SessionExpiredSubscriber` tests) would make every later `wait_for_capacity` caller poll forever — the source of the post-#1516 triage-evaluator hangs. +- **Test isolation**: in `cfg(test)` the semaphore and signed-out flag are keyed per tokio runtime ID (`test_state`) so parallel cargo workers and libtest thread reuse don't leak state across `#[tokio::test]`s. `SignedOutTestGuard` snapshots/restores the flag and bypasses the writer-side `STATE` gate. +- **`init_global` is idempotent** (`std::sync::Once`); live config changes go through `update_config`, which recomputes the policy immediately. +- **Server-mode detection** never infers server from "no battery" alone (desktops have none); it requires Linux + no battery + no `DISPLAY`/`WAYLAND_DISPLAY`, or explicit env / k8s / docker signals. +- `PauseReason::OnBattery` and `CpuPressure` are the active power-aware (#1073) reasons; `Unknown` is a placeholder fallback. diff --git a/src/openhuman/screen_intelligence/README.md b/src/openhuman/screen_intelligence/README.md new file mode 100644 index 000000000..f67e756b9 --- /dev/null +++ b/src/openhuman/screen_intelligence/README.md @@ -0,0 +1,93 @@ +# screen_intelligence + +macOS-focused screen capture, accessibility automation, and on-device vision summarization. The module boots an in-process `AccessibilityEngine` singleton that runs consent-gated capture sessions: it polls the foreground window, screenshots the active window via `screencapture -l `, runs a 3-pass OCR + local-LLM pipeline over each frame, and persists synthesized "what the user is doing right now" markdown documents into unified memory. It also exposes permission detection/requests, manual capture/diagnostics, input-action automation, autocomplete suggestions, and a macOS Globe/Fn hotkey listener. Capture is macOS-only in V1; on other platforms session start returns a `macOS-only` error and the embedded server autostart is skipped. + +## Responsibilities + +- Own the process-global `AccessibilityEngine` (state in `state.rs`) and its session lifecycle (enable / `start_session` with consent + TTL clamp 30..3600s / disable / TTL expiry). +- Detect and request macOS privacy permissions (Screen Recording, Accessibility, Input Monitoring, cross-platform Microphone) and open the relevant System Settings panes. +- Run a capture worker (`capture_worker.rs`) that polls foreground context at `baseline_fps`, captures the active window (window-ID only — never fullscreen fallback), applies allow/deny policy, optionally saves PNGs to `{workspace}/screenshots/`, and feeds frames to the vision worker. +- Run a processing worker (`processing_worker.rs`) that drains to the latest frame, compresses the image, runs Apple Vision OCR → vision LLM → synthesis LLM (Ollama), and persists a `VisionSummary` to memory. +- Provide manual / diagnostic capture (`capture_now`, `capture_image_ref_test`, `capture_test`), vision queries (`vision_recent`, `vision_flush`), and input automation (`input_action`) including a `panic_stop` action. +- Maintain in-memory autocomplete context and produce heuristic suggestions. +- Host a standalone `SiServer` (`server.rs`) that drives a capture+vision session in a monitoring loop, for embedded core autostart or CLI use. +- Expose the `openhuman screen-intelligence` CLI (`cli/`) and the `screen_intelligence.*` JSON-RPC controller surface. + +## Key files + +| File | Role | +| --- | --- | +| `mod.rs` | Export-focused. Re-exports `ops as rpc`, all `ops::*`, schema pair, `global_engine`/`AccessibilityEngine`, and `types::*`. | +| `ops.rs` | RPC/CLI handler functions returning `RpcOutcome` (status, permissions, sessions, capture, vision, globe listener) + `accessibility_doctor_cli_json`. | +| `schemas.rs` | `ControllerSchema`s, `all_controller_schemas`, `all_registered_controllers`, and `handle_*` fns delegating to `ops`. | +| `types.rs` | Serde domain types (`AccessibilityStatus`, `SessionStatus`, `CaptureFrame`, `VisionSummary`, `InputActionParams`, params/results). Re-exports permission types from `accessibility`. | +| `state.rs` | `EngineState`, `SessionRuntime`, `AccessibilityEngine`, and the `ACCESSIBILITY_ENGINE` lazy singleton / `global_engine()`. | +| `engine.rs` | Core `impl AccessibilityEngine`: config apply, session lifecycle, status, capture actions, screenshot-to-disk, allow/deny policy. | +| `input.rs` | `impl AccessibilityEngine` for `input_action`, `autocomplete_suggest`, `autocomplete_commit`. | +| `vision.rs` | `impl AccessibilityEngine` for `vision_recent`, `vision_flush`, `analyze_and_persist_frame`. | +| `capture_worker.rs` | Background screenshot loop (foreground poll, window capture, disk save, frame enqueue). | +| `processing_worker.rs` | Background vision pipeline: drain-to-latest, OCR (Apple Vision via `swift`), vision LLM, synthesis LLM, persist. | +| `image_processing.rs` | PNG→resized JPEG compression for the vision LLM (`compress_screenshot`, defaults 1024px / quality 72). | +| `helpers.rs` | Input validation, ephemeral ring-buffer pushes, vision-output parsing, memory persistence (`persist_vision_summary` → `background` namespace), suggestion generation, `truncate_tail`. | +| `limits.rs` | Buffer/string caps (max 120 frames/summaries, 256-char context, etc.). | +| `capture.rs` | `now_ms()` timestamp helper. | +| `permissions.rs` | Empty stub — permission detection moved to the `accessibility` middleware; retained for module-tree compatibility. | +| `server.rs` | `SiServer` runtime, `ServerState`/`SiServerStatus`/`SiServerConfig`, global singleton, `start_if_enabled`, `run_standalone`, benign-failure classifiers. | +| `cli/mod.rs` | `openhuman screen-intelligence` dispatch + shared opts/bootstrap helpers. | +| `cli/{capture,doctor,server,session}.rs` | CLI subcommand impls (`capture`/`vision`, `doctor`, `run`, `status`/`start`/`stop`). | +| `tests.rs`, `engine_tests.rs`, `schemas_tests.rs` | Test suites (sibling-file `#[path]` for engine/schemas). | + +## Public surface + +- `global_engine() -> Arc` and `AccessibilityEngine` (the session engine; impl blocks split across `engine.rs`/`input.rs`/`vision.rs`). +- `ops::*` (re-exported as `rpc`): async handlers e.g. `accessibility_status`, `accessibility_request_permission(s)`, `accessibility_refresh_permissions`, `accessibility_start_session`, `accessibility_stop_session`, `accessibility_capture_now`, `accessibility_capture_image_ref`, `accessibility_capture_test`, `accessibility_input_action`, `accessibility_vision_recent`, `accessibility_vision_flush`, `accessibility_globe_listener_{start,poll,stop}`, `accessibility_doctor_cli_json`. +- `all_screen_intelligence_controller_schemas` / `all_screen_intelligence_registered_controllers` (registry wiring). +- `types::*` — `AccessibilityStatus`, `SessionStatus`, `AccessibilityFeatures`, `CaptureFrame`, `CaptureNowResult`, `CaptureImageRefResult`, `CaptureTestResult`, `VisionSummary`, `VisionRecentResult`, `VisionFlushResult`, `InputActionParams`/`Result`, `StartSessionParams`, `StopSessionParams`, `PermissionRequestParams`, `AppContextInfo`, autocomplete types, plus re-exported `PermissionKind`/`PermissionState`/`PermissionStatus`/`GlobeHotkey*`. +- `server::{SiServer, SiServerConfig, SiServerStatus, ServerState, global_server, try_global_server, start_if_enabled, run_standalone}`. +- `cli::run_screen_intelligence_command`. + +## RPC / controllers + +Namespace `screen_intelligence` (registered via `all_screen_intelligence_registered_controllers` in `src/core/all.rs`). Functions: `status`, `request_permissions`, `request_permission`, `refresh_permissions`, `start_session`, `stop_session`, `capture_now`, `capture_image_ref`, `input_action`, `vision_recent`, `vision_flush`, `capture_test`, `globe_listener_start`, `globe_listener_poll`, `globe_listener_stop`. All `handle_*` fns delegate to `ops` and return CLI-compatible JSON via `RpcOutcome::into_cli_compatible_json`. + +## Agent tools + +None. This module owns no `tools.rs` / agent-tool impls. + +## Events + +None. No `bus.rs` / `EventHandler` impls; the module is driven by RPC/CLI/in-process workers, not the event bus. + +## Persistence + +- **Vision summaries → unified memory** (`helpers::persist_vision_summary`): writes via `MemoryClient::from_workspace_dir(...).put_doc_light(...)` (light path — no vectors/graph). Namespace `background`, source type `screenshot`, category/tag `screen_intelligence`, key `screen_intelligence_{captured_at_ms}_{fnv_hash(id)}`. Body is markdown with YAML frontmatter (app, window, captured ts, confidence, id). +- **Screenshots → disk** (`engine::save_screenshot_to_disk`): `{workspace_dir}/screenshots/{captured_at_ms}_{app_slug}.png` when `keep_screenshots` is set (otherwise a temp file is written for OCR and deleted). +- **In-memory only**: `SessionRuntime` holds ring buffers of recent `CaptureFrame`s and `VisionSummary`s (capped at 120 each) plus autocomplete context — not durably persisted. + +## Dependencies + +- `crate::openhuman::accessibility` — permission detection/requests, `foreground_context`, `AppContext`, window screenshot (`capture_screen_image_ref_for_context`), macOS privacy-pane opener, and the Globe/Fn hotkey listener (`globe_listener_*`). Permission types are re-exported from here. +- `crate::openhuman::config` — loads `Config` / `ScreenIntelligenceConfig` (enabled, vision_enabled, use_vision_model, keep_screenshots, baseline_fps, session_ttl_secs, panic_stop_hotkey, policy_mode, allow/deny lists, autocomplete_enabled) and `workspace_dir`; `local_ai` provider/model settings gate vision. +- `crate::openhuman::inference::local` (as `local_ai`) — Ollama vision + synthesis LLM calls in the processing worker. +- `crate::openhuman::memory_store` — `MemoryClient` / `NamespaceDocumentInput` for persisting vision summaries. +- `crate::core::all` — `ControllerFuture`/`RegisteredController` for the RPC registry; `crate::core::{ControllerSchema, FieldSchema, TypeSchema}` for schemas; `crate::core::logging` for CLI logging; `crate::rpc::RpcOutcome`. +- `crate::openhuman::embeddings::NoopEmbedding` — test-only (`tests.rs`). + +## Used by + +- `src/core/all.rs` — registers controllers/schemas and the namespace description. +- `src/core/cli.rs` — dispatches the `screen-intelligence` CLI subcommand. +- `src/openhuman/config/ops.rs` — applies config changes to the engine. +- `src/openhuman/app_state/ops.rs`, `src/openhuman/credentials/ops.rs` — referenced for lifecycle/state. +- `src/openhuman/tools/local_cli.rs` — referenced from the local CLI tool surface. + +## Notes / gotchas + +- **macOS-only V1.** `enable`/`start_session` return an error off macOS; `start_if_enabled` no-ops on non-macOS and must not initialize the global server (enforced by test). +- **Permission cache per-process.** macOS TCC grants are per-executable and per-process; the running core never sees a freshly-granted permission until it restarts. `refresh_permissions` re-detects (status always calls `detect_permissions()`), but a `restart_core_process` is required to pick up new grants — see GH #133. `AccessibilityStatus` carries `permission_check_process_path` and `core_process` (pid/started_at) so the UI can verify a restart actually happened. +- **Window-ID required.** Capture only proceeds when the foreground context has a `window_id`; there is no fullscreen fallback. +- **Vision pipeline requires local AI.** `analyze_frame` errors unless `local_ai.runtime_enabled=true` and provider is `ollama`. OCR shells out to `swift -e ` with a 30s timeout and `kill_on_drop`. `--no-vision-model` / `--ocr-only` (CLI) sets `use_vision_model=false` at runtime (not persisted) to skip the vision LLM pass. +- **Drain-to-latest.** The processing worker discards stale queued frames and only analyzes the most recent, deduping by `captured_at_ms`; vision-summary pushes also dedupe on `captured_at_ms` so `vision_flush` and the worker channel don't double-store. +- **Lock discipline.** Workers read all needed state under the `Mutex`, then drop the lock before slow I/O (screencapture, disk, LLM); session task handles are aborted/awaited outside the lock to avoid deadlocks. +- **Benign start-session failures** (`macOS-only` off-macOS, `session already active`) are classified down to `info!` so they don't surface as Sentry errors; the `SiServer` cancellation token is swappable so it can restart after logout→login within one process. +- `permissions.rs` is an intentional empty stub; YAML frontmatter escaping in `persist_vision_summary` is best-effort (only quotes/newlines). diff --git a/src/openhuman/service/README.md b/src/openhuman/service/README.md new file mode 100644 index 000000000..91a7d3797 --- /dev/null +++ b/src/openhuman/service/README.md @@ -0,0 +1,111 @@ +# service + +Service-management domain for the OpenHuman core daemon. It installs/uninstalls the core binary as a per-user OS service (macOS LaunchAgent, Linux systemd user unit, Windows scheduled task), drives its install/start/stop/status lifecycle, and orchestrates **self-restart** and **graceful shutdown** of the currently running core process via the event bus. It also persists machine-local daemon-host UI preferences (tray visibility). All of this is exposed transport-agnostically through the `service.*` RPC/CLI controller registry. + +## Responsibilities + +- Install / uninstall the core daemon as a native per-user service and report its `ServiceStatus`. +- Start / stop / query the installed service per OS (`launchctl`, `systemctl --user`, `schtasks`). +- Accept restart/shutdown requests, publish them to the event bus, and (via subscribers) respawn or `process::exit(0)` the running core with a 150ms flush grace window. +- Resolve the daemon executable to launch (env override `OPENHUMAN_CORE_BIN`, else sibling `openhuman-core[-*]` next to the current exe; macOS also searches `../Resources`). +- Persist and read daemon-host UI preferences (`show_tray`) next to the main config. +- Provide a deterministic, file-backed **mock** service backend for E2E tests (`OPENHUMAN_SERVICE_MOCK`). +- Expose `daemon_state.json` path helper consumed by doctor/health reporting. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/service/mod.rs` | Export-focused module root; declares submodules and re-exports the public surface + controller-schema pair. | +| `src/openhuman/service/core.rs` | `ServiceState` / `ServiceStatus` types and the cross-platform `install`/`start`/`stop`/`status`/`uninstall` dispatchers that route to the mock or the per-OS impl via `cfg`. | +| `src/openhuman/service/ops.rs` | RPC handler layer (`service_install`, `service_start`, `service_stop`, `service_status`, `service_restart`, `service_shutdown`, `service_uninstall`, `daemon_host_get`/`set`) returning `RpcOutcome`. Re-exported as `rpc`. | +| `src/openhuman/service/schemas.rs` | Controller schemas + `handle_*` adapters for the `service` namespace; `all_controller_schemas` / `all_registered_controllers`. | +| `src/openhuman/service/restart.rs` | Self-restart orchestration: `service_restart` (publishes event), `trigger_self_restart_now` (respawns current exe with original args), `apply_startup_restart_delay_from_env`, `RestartStatus`. | +| `src/openhuman/service/shutdown.rs` | Graceful-shutdown orchestration: `service_shutdown` (publishes event), `ShutdownStatus`. | +| `src/openhuman/service/bus.rs` | Event-bus subscribers `RestartSubscriber` / `ShutdownSubscriber` (filter domain `system`) and idempotent `register_*_subscriber` helpers; one-shot atomic gates so only the first request acts. | +| `src/openhuman/service/common.rs` | Shared OS helpers: service labels, `resolve_daemon_executable`, `daemon_program_args` (`["run"]`), `xml_escape`, command runners (`run_checked`/`run_capture`/`run_best_effort`/`run_check_silent`), Windows `CREATE_NO_WINDOW` suppression. | +| `src/openhuman/service/macos.rs` | LaunchAgent (`com.openhuman.core.plist`) install/start/stop/status/uninstall via `launchctl`; migrates legacy labels. | +| `src/openhuman/service/linux.rs` | systemd user-unit install/lifecycle via `systemctl --user`. | +| `src/openhuman/service/windows.rs` | Scheduled-task install/lifecycle via `schtasks`. | +| `src/openhuman/service/daemon.rs` | `state_file_path(config)` → `/daemon_state.json`, used by doctor/health. | +| `src/openhuman/service/daemon_host.rs` | `DaemonHostConfig { show_tray }` + async `load_for_config_dir` / `save_for_config_dir` (JSON next to config, `daemon_host_config.json`). | +| `src/openhuman/service/mock.rs` | File-backed deterministic mock backend gated on `OPENHUMAN_SERVICE_MOCK`; supports forced failures and an `agent_running` flag (`mock_agent_running`). | +| `src/openhuman/service/mock_tests.rs` | Sibling test suite for `mock.rs`. | + +## Public surface + +From `mod.rs` re-exports: + +- `core::*` — `ServiceState`, `ServiceStatus`, and the dispatchers `install` / `start` / `stop` / `status` / `uninstall`. +- `ops::*` (also aliased `rpc`) — the async RPC handlers `service_install` … `service_uninstall`, `service_restart`, `service_shutdown`, `daemon_host_get`, `daemon_host_set`. +- `restart::{apply_startup_restart_delay_from_env, RestartStatus}` (and `restart::trigger_self_restart_now` via the `restart` module path). +- `shutdown::ShutdownStatus`. +- `schemas::all_service_controller_schemas` / `all_service_registered_controllers`. +- `daemon::state_file_path`, `daemon_host::{DaemonHostConfig, …}` (via their `pub mod` paths). + +## RPC / controllers + +Namespace `service` (called as `openhuman.service_` / `service.`). All nine handlers go through `RpcOutcome` and are wired into the registry by `crate::core::all` via `all_service_registered_controllers`. + +| Method | Inputs | Output | +| --- | --- | --- | +| `service.install` | — | `ServiceStatus` | +| `service.start` | — | `ServiceStatus` | +| `service.stop` | — | `ServiceStatus` | +| `service.status` | — | `ServiceStatus` | +| `service.uninstall` | — | `ServiceStatus` | +| `service.restart` | `source?`, `reason?` | restart ack JSON (`RestartStatus`) | +| `service.shutdown` | `source?`, `reason?` | shutdown ack JSON (`ShutdownStatus`) | +| `service.daemon_host_get` | — | `DaemonHostConfig` | +| `service.daemon_host_set` | `show_tray` (required) | `DaemonHostConfig` | + +Lifecycle handlers load config via `config::rpc::load_config_with_timeout`; `restart`/`shutdown` are intentionally config-free (they target the running process). + +## Events + +Publishes (in `restart.rs` / `shutdown.rs`) to the global event bus, domain `system`: + +- `DomainEvent::SystemRestartRequested { source, reason }` +- `DomainEvent::SystemShutdownRequested { source, reason }` + +Subscribes (in `bus.rs`): + +- `RestartSubscriber` (`name = "service::restart"`) — on `SystemRestartRequested`, atomically claims a one-shot gate, calls `trigger_self_restart_now` to spawn a replacement process, then `process::exit(0)` after 150ms. +- `ShutdownSubscriber` (`name = "service::shutdown"`) — on `SystemShutdownRequested`, claims its gate and `process::exit(0)` after 150ms (no respawn). + +Both subscribers are registered idempotently from `src/core/jsonrpc.rs` at startup via `register_restart_subscriber` / `register_shutdown_subscriber`; handles are held in process-lifetime `OnceLock`s so they are never dropped. + +## Persistence + +- **Daemon-host prefs** — `daemon_host_config.json` next to the main config (`DaemonHostConfig { show_tray }`, default `true`); read/written async by `daemon_host.rs`. +- **Daemon state path** — `daemon_state.json` next to config (`daemon::state_file_path`), consumed by doctor/health (not written here). +- **Service unit files** — written by the per-OS impls (macOS plist in `~/Library/LaunchAgents`, Linux systemd user unit, Windows scheduled task). +- **Mock state** — `service-mock-state.json` (overridable via `OPENHUMAN_SERVICE_MOCK_STATE_FILE`) tracking `installed`/`running`/`agent_running`/forced `failures`, only when the mock is enabled. + +## Dependencies + +- `crate::openhuman::config` — `Config` (paths, config dir) for every lifecycle/path operation; `config::rpc::load_config_with_timeout` in the schema handlers. +- `crate::core::event_bus` — `DomainEvent`, `EventHandler`, `SubscriptionHandle`, `publish_global` / `subscribe_global` / `init_global` for restart/shutdown orchestration. +- `crate::core` (`ControllerSchema`, `FieldSchema`, `TypeSchema`) and `crate::core::all` (`ControllerFuture`, `RegisteredController`) — controller schema/registration contract. +- `crate::rpc::RpcOutcome` — standard RPC result envelope. +- External: `anyhow`, `serde`/`serde_json`, `tokio`, `async_trait`, plus OS CLIs (`launchctl`, `systemctl`, `schtasks`). + +## Used by + +- `src/core/all.rs` — registers the service controllers (`all_service_registered_controllers`). +- `src/core/jsonrpc.rs` — registers the restart/shutdown event-bus subscribers at startup. +- `src/openhuman/doctor/core.rs` — reads `service::daemon::state_file_path`. +- `src/openhuman/update/ops.rs`, `src/openhuman/config/ops.rs`, `src/openhuman/app_state/ops.rs` — reference `openhuman::service` (status/lifecycle/restart paths). +- `src/lib.rs` — module wiring. + +## Notes / gotchas + +- Restart/shutdown are **two-phase**: the RPC/CLI call only acknowledges and publishes an event; the actual respawn/exit happens in the subscriber, so RPC, CLI, and internal triggers share one path with consistent logging. The subscriber must be registered or requests are no-ops. +- One-shot atomic gates exist in **both** `bus.rs` and `restart.rs` (`RESTART_IN_PROGRESS`) — duplicate restart events are ignored; a failed `trigger_self_restart_now` resets the gate to allow a retry. +- `trigger_self_restart_now` respawns the current exe with the **original argv** (preserving launch mode) and sets `OPENHUMAN_RESTART_DELAY_MS` (default 350) on the child; the child honors it at startup via `apply_startup_restart_delay_from_env` to dodge HTTP-port bind races while the old process releases sockets. +- Self-restart fails if launched with no args (`std::env::args().skip(1)` empty). +- Platform support is compile-gated; on unsupported targets the dispatchers `bail!` with "supported on macOS, Linux, and Windows only". +- `OPENHUMAN_SERVICE_MOCK` short-circuits **every** lifecycle dispatcher to the file-backed mock — used for deterministic E2E; it can also inject forced per-operation failures. +- Windows command spawns set `CREATE_NO_WINDOW` (`common::no_window`) so polled `schtasks /Query` calls don't flash a console. +- Lifecycle RPCs are deliberately **not** unit-tested (they mutate real OS state or kill the process); RPC-adapter coverage lives in `tests/json_rpc_e2e.rs`. `daemon_host_get`/`set` and the restart/shutdown publish paths are unit-tested. +- `daemon.rs::state_file_path` and `common.rs::state_file_path` (mock) both compute paths next to config but for different files (`daemon_state.json` vs the mock state file). diff --git a/src/openhuman/socket/README.md b/src/openhuman/socket/README.md new file mode 100644 index 000000000..fe33ca8a1 --- /dev/null +++ b/src/openhuman/socket/README.md @@ -0,0 +1,102 @@ +# socket + +Persistent, Rust-native Socket.IO client to the OpenHuman backend. The `socket` domain owns a single long-lived `SocketManager` that speaks Engine.IO v4 + Socket.IO v4 directly over a WebSocket (`tokio-tungstenite` + `rustls`), maintains the connection with exponential-backoff reconnection, and routes inbound server events onto the in-process event bus for domain-specific handling (webhooks, channel messages, Composio triggers, device tunnel). Outbound `emit`s and connection lifecycle are exposed over JSON-RPC under the `socket` namespace. + +## Responsibilities + +- Open and maintain a persistent Socket.IO connection to the backend over a raw WebSocket; perform the Engine.IO OPEN and Socket.IO CONNECT handshakes by hand. +- Authenticate the SIO CONNECT with a JWT and reconnect automatically with exponential backoff (1 s → 30 s cap). +- Follow HTTP 3xx redirects during the upgrade (up to 3 hops) so a `http://`-configured `BACKEND_URL` behind a TLS-forcing edge connects cleanly; pin the resolved URL for subsequent reconnects and surface a one-shot "stale BACKEND_URL" warning for permanent redirects. +- Refresh the session token before every reconnect via a `TokenProvider` callback (live re-read of the profile store) instead of caching a single token — fixes the "Invalid token" retry storm (#2892 / TAURI-RUST-9C). +- Fast-fail on a definitively dead token (server "Invalid token" + no fresher token available) rather than burning the whole backoff budget. +- Track connection status / socket id / last user-visible error in shared state and expose it via RPC. +- Parse inbound Socket.IO EVENT frames and publish them as `DomainEvent`s for other domains to consume; emit outbound events. +- Suppress reconnect-storm Sentry noise: route only the 5th consecutive failure through the observability classifier (which demotes offline/transport shapes to a breadcrumb), keep all other retries at `warn`. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/socket/mod.rs` | Module docstring + exports only. Re-exports `SocketManager`, `global_socket_manager`, `set_global_socket_manager`, and the `all_socket_controller_schemas` / `all_socket_registered_controllers` pair. | +| `src/openhuman/socket/manager.rs` | `SocketManager` handle + `SharedState`; global `OnceLock` accessor; `connect` / `connect_with_provider` / `disconnect` / `emit` / `get_state`; spawns the background `ws_loop`. Holds emit/shutdown channels and the loop join handle. | +| `src/openhuman/socket/ws_loop.rs` | The background reconnection loop and a single connection attempt: Engine.IO/Socket.IO handshake, redirect-following connect, ping-timeout deadline, backoff, invalid-token decision logic, failure-escalation logging. | +| `src/openhuman/socket/event_handlers.rs` | Inbound SIO event dispatch (`handle_sio_event`), SIO frame parsing (`parse_sio_event`), outbound frame helper (`emit_via_channel`). Maps event names → `DomainEvent` publishes. Redacts payload content from logs. | +| `src/openhuman/socket/token_provider.rs` | `TokenProvider` type alias + `static_token_provider`, `token_provider_from_config`, and `is_invalid_token_error` (strict double-anchor matcher). | +| `src/openhuman/socket/schemas.rs` | Controller schemas + RPC handlers for the `socket` namespace. | +| `src/openhuman/socket/types.rs` | `WsStream` alias, `ConnectionOutcome` enum, observability event-name constants; re-exports `ConnectionStatus` / `SocketState` from `crate::api::models::socket`. | +| `src/openhuman/socket/ws_loop_tests.rs` | Out-of-line test suite for `ws_loop.rs` (via `#[path = ...]`). | + +## Public surface + +- `SocketManager` — the connection handle. Key methods: `new`, `connect(url, token)`, `connect_with_provider(url, provider)`, `disconnect`, `emit(event, data)`, `get_state() -> SocketState`, `is_connected`, `set_webhook_router` / `webhook_router`. +- `global_socket_manager() -> Option<&'static Arc>` and `set_global_socket_manager(Arc)` — the process-global singleton (set once at bootstrap). +- `all_socket_controller_schemas` / `all_socket_registered_controllers` — controller-registry exports wired into `src/core/all.rs`. + +Internal-only (`pub(crate)` / `pub(super)`): `TokenProvider` and its builders, `SharedState`, `ConnectionOutcome`, `WsStream`, the `ws_loop` and event-handler helpers. + +## RPC / controllers + +Namespace `socket` (called as `openhuman.socket_`): + +| Function | Inputs | Output | Notes | +| --- | --- | --- | --- | +| `connect` | `url` (str, req), `token` (str, req) | `status` | Connect with an explicit static token. | +| `disconnect` | — | `status` | Tear down the loop. | +| `state` | — | `state` (JSON: status, socket_id, error) | | +| `emit` | `event` (str, req), `data` (JSON, opt) | `ok` (bool) | Emit a SIO event to the backend. | +| `connect_with_session` | — | `status` | Loads config, derives the API URL, and connects with a **live-refresh** token provider reading the stored session token from the profile store on every reconnect. | + +All handlers go through `require_manager()` and error with `"SocketManager not initialized"` if the global singleton is unset. + +## Events + +`event_handlers::handle_sio_event` is a thin transport router — it does not run domain logic itself. It mutates connection status for `ready`/`error` and publishes the following `DomainEvent`s via `publish_global` for other domains' bus subscribers: + +| Inbound SIO event | Published `DomainEvent` | Consumer domain | +| --- | --- | --- | +| `webhook:request` | `WebhookIncomingRequest { request, raw_data }` | webhooks (also emits a `webhook:response` 400 on parse failure) | +| `composio:trigger` | `ComposioTriggerReceived { toolkit, trigger, metadata_id, metadata_uuid, payload }` | composio | +| `tunnel:peer-status` | `DevicePeerOnline` / `DevicePeerOffline` | devices | +| `tunnel:frame` | `DeviceTunnelFrame` | devices | +| `tunnel:registered` | `DeviceTunnelRegistered` | devices | +| `tunnel:evicted` | `DevicePeerOffline` | devices | +| `*:message` (suffix match) | `ChannelInboundMessage { event_name, channel, message, sender, reply_target, thread_ts, raw_data }` | channels | + +This module is a **publisher only** — it owns no `bus.rs` / `EventHandler` impls. + +## Persistence + +None of its own. State (`status`, `socket_id`, `error`, attached `WebhookRouter`) lives in-memory in `SharedState` behind `parking_lot::RwLock`. The session token is read on demand from the profile store via `crate::api::jwt::get_session_token` (live-refresh path); there is no `store.rs`. + +## Dependencies + +- `crate::api::models::socket` — `ConnectionStatus`, `SocketState` DTOs. +- `crate::api::socket::websocket_url`, `crate::api::config::effective_backend_api_url`, `crate::api::jwt::get_session_token` — URL derivation and session-token lookup. +- `crate::core::all` — `ControllerFuture`, `RegisteredController` for the controller registry. +- `crate::core::{ControllerSchema, FieldSchema, TypeSchema}` — RPC schema types. +- `crate::core::event_bus` — `publish_global` / `DomainEvent` for routing inbound events. +- `crate::core::observability::report_error_or_expected` — one-shot sustained-outage classification at the failure threshold. +- `crate::openhuman::webhooks` — `WebhookRouter` (attached for parse-error logging / response emission) and `WebhookRequest`. +- `crate::openhuman::composio` — `ComposioTriggerEvent` DTO for `composio:trigger` deserialization. +- `crate::openhuman::devices::tunnel_client` — `TunnelPeerStatus`, `TunnelFrame` DTOs for tunnel events. +- `crate::openhuman::config` — `Config` + `rpc::load_config_with_timeout` for `connect_with_session`. +- `crate::openhuman::util::utf8_safe_prefix_at_byte_boundary` — UTF-8-safe log truncation of raw packets. + +## Used by + +- `src/core/all.rs` — registers the socket controllers. +- `src/core/jsonrpc.rs`, `src/core/observability.rs` — reference the socket namespace/state. +- `src/openhuman/connectivity/rpc.rs` — connectivity/status surfacing. +- `src/openhuman/webhooks/{ops.rs,bus.rs}` — emit webhook responses back through the global manager. +- `src/openhuman/devices/tunnel_client.rs` — emits tunnel frames/registration over the socket. + +## Notes / gotchas + +- **Event-name constants in `types.rs` (`runtime:socket-state-changed`, `server:event`) are grep-anchored** — the frontend subscribes to those exact strings; a rename silently breaks the Tauri event bridge (locked by a test). +- **Payload content is never logged** at any level — webhook bodies / channel messages / Composio payloads can carry PII, secrets, or tokens. Only byte-length and structural shape are logged. This also dodged a UTF-8 char-boundary panic that used to slice raw payloads at byte 500 (OPENHUMAN-TAURI-KC / #1814). +- `connect` rejects an empty/whitespace token immediately rather than spawning a doomed retry loop; `connect_with_provider` does the same eager pre-check via the provider. +- The reconnect loop bounds "fresh-token immediate retry" to **one** per cycle so a provider that returns a different non-empty token every call cannot hot-loop without sleeping or escalating (CodeRabbit Major, #2905). +- A genuinely fresh token from the invalid-token decision is **carried forward** (`pending_token`) so the next iteration uses the exact validated value, avoiding a redundant profile-store lock/disk read. +- Sentry escalation fires exactly once, on the 5th consecutive failure (~15 s of accumulated backoff), and is routed through the observability classifier so offline/transport shapes demote to a breadcrumb (OPENHUMAN-TAURI-8M / -BH). +- Redirect following only persists the "update BACKEND_URL" warning for permanent redirects (301/308); temporary (302/307) hops don't (CodeRabbit, #1547). +- No agent tools and no `bus.rs` — this is a transport domain that publishes events for others to handle. diff --git a/src/openhuman/startup/README.md b/src/openhuman/startup/README.md new file mode 100644 index 000000000..593b8e537 --- /dev/null +++ b/src/openhuman/startup/README.md @@ -0,0 +1,53 @@ +# startup + +Generic OpenHuman process-startup helpers. Currently a thin, stateless module whose sole job is to run one-shot workspace migrations during core boot. It centralizes "do this once when the process comes up" logic so the transport layer (`src/core/jsonrpc.rs`) can fire it without owning migration details. Failures are logged and never abort startup — individual migration helpers own their own idempotency markers. + +## Responsibilities + +- Run workspace migrations at process startup via `run_workspace_migrations(workspace_dir)`. +- Drive the **session-layout** migration (`agent::harness::session::migrate_session_layout_if_needed`) and log its outcome (jsonl/md moved, pruned legacy dirs, warnings). +- Drive the **welcome-to-orchestrator** thread/artifact migration (`threads::migrate_welcome_agent_artifacts`) and log its outcome (threads/transcripts updated, files renamed). +- Swallow migration errors (log `warn`) and fall back to in-place legacy reads so boot always proceeds. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/startup/mod.rs` | Export-only: module docstring + `pub mod ops;` + re-export of `run_workspace_migrations`. | +| `src/openhuman/startup/ops.rs` | Implementation of `run_workspace_migrations` — orchestrates the two workspace migrations and logging. | + +## Public surface + +- `run_workspace_migrations(workspace_dir: &Path)` (re-exported from `ops`) — the only public entry point. Returns nothing; all failure handling is internal/logged. + +## RPC / controllers + +None. The module exposes no controller schemas or `handle_*` functions; it is invoked directly by the transport boot path, not via the controller registry. + +## Agent tools + +None (no `tools.rs`). + +## Events + +None (no `bus.rs`); does not publish or subscribe to `DomainEvent`s. + +## Persistence + +No own state/store. It triggers migrations that mutate on-disk workspace artifacts (session layout files, thread/transcript artifacts) under `workspace_dir`, but the actual persistence and idempotency markers live in the called migration helpers (`agent::harness::session`, `threads`). + +## Dependencies + +- `crate::openhuman::agent::harness::session::migrate_session_layout_if_needed` — performs the session-layout migration (moves jsonl/md, prunes legacy dirs). +- `crate::openhuman::threads::migrate_welcome_agent_artifacts` — performs the welcome-agent → orchestrator artifact migration. + +## Used by + +- `src/core/jsonrpc.rs` (core boot path) — the only caller; invokes `run_workspace_migrations(&workspace_dir)` during core startup, after approval-gate wiring and before MCP registry boot-spawn. + +## Notes / gotchas + +- **Non-fatal by design**: every migration branch logs and continues; a failed migration must never block startup. +- **Idempotency is delegated**: this module does not track whether a migration already ran — it relies on each helper's own `already_done` markers. Re-running is safe. +- **Grep-friendly log prefixes**: `[runtime]` for session-layout, `[migration::welcome-to-orchestrator]` for the thread/artifact migration. +- Module is intentionally minimal — no `types.rs`/`store.rs`/`schemas.rs` because it holds no domain types, no persisted state, and no RPC surface. diff --git a/src/openhuman/subconscious/README.md b/src/openhuman/subconscious/README.md new file mode 100644 index 000000000..6ebc85d8b --- /dev/null +++ b/src/openhuman/subconscious/README.md @@ -0,0 +1,131 @@ +# subconscious + +The subconscious is OpenHuman's background-awareness layer: a SQLite-backed loop that, on each tick, evaluates a set of user/system tasks against a freshly-built "situation report" (derived from the memory tree) using an LLM, then **acts** on actionable tasks, **escalates** ambiguous/risky ones for user approval, or **noops**. In the same tick the LLM also emits proactive **reflections** (#623) — observation-only cards surfaced on the Intelligence tab. The actual periodic loop is owned by the `heartbeat` domain; this module owns task storage, tick evaluation/execution, escalations, and reflections. + +## Responsibilities + +- Maintain a list of `SubconsciousTask`s (system-seeded + user-added) in SQLite; seed three default system tasks on init. +- Run a tick: load due tasks → log them `in_progress` → build a situation report → call the configured LLM → execute `act` tasks, create escalations for `escalate`/`UnapprovedWrite`, mark `noop` otherwise → update log entries in place. +- Route the per-tick evaluation and task execution to a local Ollama/LM Studio model or the OpenHuman cloud, based on config (`workload_local_model("subconscious")` / `subconscious_provider`). +- Classify task write-intent via keyword heuristics (`needs_tools` / `needs_agent`); run read-only tasks analysis-only and escalate any recommended write action for approval. +- Emit, cap (`MAX_REFLECTIONS_PER_TICK = 5`), hydrate, and persist proactive reflections; resolve each reflection's `source_refs` into frozen `SourceChunk` snapshots at tick time. +- Manage escalation lifecycle (pending → approved/dismissed); approving executes the task at full permissions. +- Persist `last_tick_at` across restarts so the situation report only feeds the LLM memory-tree rows newer than the last successful tick (dedupe). +- Provide an overlap guard (generation counter) so a newer tick supersedes an in-flight one and discards its results without advancing the cutoff. +- Expose the full task/log/escalation/reflection surface over JSON-RPC. + +## Key files + +| File | Role | +| --- | --- | +| `mod.rs` | Export-focused (no docstring); re-exports engine, reflection, schemas, source_chunk, and core types. | +| `types.rs` | Domain serde types: `SubconsciousTask`, `TaskSource`, `TaskRecurrence`, `TaskPatch`, `TickDecision`, `TaskEvaluation`, `EvaluationResponse`, `ExecutionResult`, `SubconsciousLogEntry`, `Escalation`/`EscalationPriority`/`EscalationStatus`, `SubconsciousStatus`, `TickResult`. | +| `engine.rs` | `SubconsciousEngine` — the tick loop, evaluation, dispatch (`handle_act`/`handle_escalate`/`handle_noop`), escalation approve/dismiss, provider routing (`resolve_subconscious_route`, `subconscious_provider_unavailable_reason`), LLM response parsing, reflection persistence. | +| `executor.rs` | Per-task execution: routes to local model (text), agentic-v1 full (write-intent), or agentic-v1 analysis-only (read-only). `ExecutionOutcome` (`Completed` / `UnapprovedWrite`), `needs_tools`/`needs_agent` heuristics, 429 retry with backoff, `extract_recommended_action`. | +| `store.rs` | SQLite persistence + DDL for all tables; `with_connection` with busy-timeout + retry (TAURI-RUST-A). Task/log/escalation CRUD, `seed_default_tasks`, `due_tasks`, `compute_next_run` (cron), `get/set_last_tick_at`. | +| `reflection.rs` | `Reflection`, `ReflectionKind`, `ReflectionDraft`; `hydrate_draft`, `apply_cap`, `dedup_key`, `MAX_REFLECTIONS_PER_TICK`. | +| `reflection_store.rs` | SQLite persistence for `subconscious_reflections` + `subconscious_hotness_snapshots`; `list_recent`, `get_reflection`, `add_reflection`, `mark_acted`/`mark_dismissed`, legacy-column + `source_chunks` migrations. | +| `source_chunk.rs` | `SourceChunk` + `resolve_chunks` / `parse_ref` — resolve reflection `source_refs` (`entity:`/`summary:`/`digest:`/…) into frozen content previews (`PREVIEW_MAX_CHARS = 400`). | +| `prompt.rs` | Prompt builders: `build_evaluation_prompt`, `build_text_execution_prompt`, `build_tool_execution_prompt`, `build_analysis_only_prompt`, `load_identity_context` (injects SOUL.md/PROFILE.md). | +| `global.rs` | Engine singleton: `get_or_init_engine`, `bootstrap_after_login`, `stop_heartbeat_loop`, `reset_engine_for_user_switch`. Spawns the `heartbeat` loop and tears it down on logout/user switch. | +| `schemas.rs` | RPC controller schemas + `handle_*` handlers (`subconscious.*`). | +| `decision_log.rs` | In-memory `DecisionLog`/`DecisionRecord` with 24h TTL to avoid re-surfacing the same doc ids. Retained for potential future dedup queries (not wired into the live tick path). | +| `situation_report/` | Situation-report assembly (see below). | +| `*_tests.rs`, `integration_tests.rs` | Sibling and inline test suites. | + +### `situation_report/` submodule + +| File | Role | +| --- | --- | +| `mod.rs` | `build_situation_report` — assembles sections in priority order under a token budget (env, user identifiers, pending tasks, hotness deltas, sealed summaries, L0 digest, recap window, recent reflections); truncates the tail when over budget. | +| `hotness.rs` | Top entity hotness movers since last tick (`mem_tree_entity_hotness`). | +| `summaries.rs` | Recently-sealed summaries (`mem_tree_summaries`). | +| `digest.rs` | Latest global L0 daily digest body. | +| `query_window.rs` | `query_global` recap window since `last_tick_at`. | +| `reflections.rs` | Renders recent reflections as anti-double-emit context. | + +## Public surface + +From `mod.rs`: +- `SubconsciousEngine` (`engine`) — `new`, `from_heartbeat_config`, `run`, `tick`, `status`, `add_task`, `approve_escalation`, `dismiss_escalation`. +- `Reflection`, `ReflectionKind`, `MAX_REFLECTIONS_PER_TICK` (`reflection`). +- `SourceChunk` (`source_chunk`). +- Types: `Escalation`, `EscalationStatus`, `SubconsciousLogEntry`, `SubconsciousStatus`, `SubconsciousTask`, `TaskRecurrence`, `TaskSource`, `TickDecision`, `TickResult`. +- `all_subconscious_controller_schemas` / `all_subconscious_registered_controllers` (`schemas`). +- `global` module functions (`get_or_init_engine`, `bootstrap_after_login`, `stop_heartbeat_loop`, `reset_engine_for_user_switch`) are reachable via `subconscious::global::*`. + +## RPC / controllers + +Namespace `subconscious` (i.e. `openhuman.subconscious_`), all returning `RpcOutcome`: + +| Function | Purpose | +| --- | --- | +| `status` | Engine status (read entirely from DB to avoid blocking on the tick mutex). | +| `trigger` | Manually fire a tick (spawned in the background; returns immediately). | +| `tasks_list` | List tasks (optional `enabled_only`). | +| `tasks_add` | Add a task (`title`, optional `source`). | +| `tasks_update` | Patch `title`/`recurrence` (`once` \| `cron:` \| `pending`)/`enabled`. | +| `tasks_remove` | Delete a task (system tasks cannot be deleted). | +| `log_list` | List execution log entries (optional `task_id`, `limit`). | +| `escalations_list` | List escalations (optional `status`). | +| `escalations_approve` | Approve + execute an escalation. | +| `escalations_dismiss` | Dismiss an escalation without executing. | +| `reflections_list` | List recent reflections (`limit`, `since_ts`). | +| `reflections_act` | Spawn a fresh conversation thread seeded with the reflection body (as an `assistant` message; no LLM turn) and stamp `acted_on_at`; returns `{reflection_id, thread_id}`. | +| `reflections_dismiss` | Set `dismissed_at`. | + +Handlers use the shared bounded `load_config_with_timeout()` loader (30s) to avoid stalling the Intelligence-page 3s poll on a slow keychain. + +## Agent tools + +None. This module owns no `tools.rs` and registers no agent tools. + +## Events + +No `bus.rs`; the module neither publishes nor subscribes to `DomainEvent`s directly. (The tick loop is driven by the `heartbeat` domain, not the event bus.) + +## Persistence + +SQLite at `/subconscious/subconscious.db` (per-user workspace). Tables (`store.rs` DDL): +- `subconscious_tasks` — task definitions (id, title, source, recurrence, enabled, run times, completed, created_at). +- `subconscious_log` — per-tick execution log (decision `in_progress`/`act`/`escalate`/`noop`/`failed`/`cancelled`/`dismissed`, result, duration). +- `subconscious_escalations` — escalations awaiting user input. +- `subconscious_reflections` — proactive reflections incl. `source_refs`, `source_chunks`, lifecycle timestamps. +- `subconscious_hotness_snapshots` — per-entity previous-tick hotness scores for hotness-delta computation. +- `subconscious_state` — KV table holding `last_tick_at` (restart-durable dedupe cutoff). + +`with_connection` runs all DDL + idempotent migrations on every open, with a 5s busy timeout and 3-retry exponential backoff for transient `SQLITE_BUSY`/`SQLITE_LOCKED`. + +## Dependencies + +- `openhuman::config` — `Config`/`HeartbeatConfig`, provider routing (`workload_local_model`, `subconscious_provider`), bounded loaders, `workspace_dir`. +- `openhuman::heartbeat` — `HeartbeatEngine`; `global.rs` spawns the periodic loop that calls `tick`. +- `openhuman::memory::chat` — `build_chat_provider`/`ChatProvider`/`ChatPrompt` for the per-tick LLM evaluation call. +- `openhuman::inference` — local provider factory + `local::ops::agent_chat` for task execution (executor). +- `openhuman::memory_store` — `MemoryClient`/`MemoryClientRef`, tree types; the engine holds a memory client and the situation report reads tree tables. +- `openhuman::memory_tree` — `retrieval::global::query_global` for the recap-window section. +- `openhuman::memory_conversations` — `ensure_thread`/`append_message` for `reflections_act` thread spawning. +- `openhuman::credentials` — `AuthService`/`APP_SESSION_PROVIDER` to check the OpenHuman cloud session bearer for provider availability. +- `openhuman::scheduler_gate` — `is_signed_out()` gate for the cloud provider. +- `openhuman::composio::providers::profile` — connected-account identifiers for the "Your Identifiers" report section (#1365). +- `openhuman::util` — `floor_char_boundary` for budget-safe truncation. +- `core::all` / `core::{ControllerSchema, FieldSchema, TypeSchema}` + `rpc::RpcOutcome` — RPC controller registration. + +## Used by + +- `core::all` / `core::jsonrpc` — registers the subconscious controllers into the RPC surface. +- `openhuman::heartbeat::{engine, rpc}` — drives ticks via the engine; `global::bootstrap_after_login` spawns the heartbeat loop. +- `openhuman::agent::harness::session::builder` and `openhuman::context::prompt::SystemPromptBuilder` — inject reflection `source_chunks` as memory context for threads spawned from a reflection. +- `openhuman::agent::prompts` — references subconscious in prompt assembly. +- `openhuman::channels::providers::web` — chat ingress. +- `openhuman::credentials::ops` — login/logout flow triggers `bootstrap_after_login` / `reset_engine_for_user_switch`. + +## Notes / gotchas + +- **Engine must bootstrap post-login** (`global::bootstrap_after_login`) so `seed_default_tasks` writes to the per-user workspace, not the pre-login global default. `reset_engine_for_user_switch` tears it down on logout/account switch to avoid leaking into the wrong DB. +- **`status` RPC never touches the engine mutex** — it reads counts straight from SQLite, because the engine lock is held for the full tick duration and would otherwise freeze the 3s poll. `consecutive_failures` is therefore reported as `0` from the RPC path (only available from in-memory state). +- **`last_tick_at` is only advanced on success.** Evaluation failure, provider unavailability, or a superseded tick leave the cutoff in place so the next tick re-reads the same window — at the cost of possible re-emitted reflections (there is no insert-time dedupe in `persist_and_surface_reflections`; `dedup_key` exists but is not enforced on insert in the live path). +- **Reflections are observation-only.** The legacy auto-post-into-thread flow was removed; `disposition`/`surfaced_at` columns are dropped via migration and any LLM-emitted `disposition` is ignored by serde. +- **Write-intent gating is heuristic** (`needs_tools`/`needs_agent` keyword matching). Read-only tasks run analysis-only; a `RECOMMENDED ACTION:` line in the output triggers an `UnapprovedWrite` escalation — except on the cloud fallback path for simple text tasks, which deliberately suppresses escalation. +- **`decision_log.rs` is retained but not wired into the live tick** (the comment in `mod.rs` notes it is kept for potential future dedup queries). +- LLM response parsing is best-effort: full envelope → bare evaluations array → all-noop fallback; `extract_json` strips prose around the JSON object/array. diff --git a/src/openhuman/task_sources/README.md b/src/openhuman/task_sources/README.md new file mode 100644 index 000000000..9aeb7ae9c --- /dev/null +++ b/src/openhuman/task_sources/README.md @@ -0,0 +1,115 @@ +# task_sources + +Proactive ingestion of work items from external tools. A **task source** is a user-configured pull from a Composio-backed provider (GitHub, Notion, Linear, ClickUp) with a per-provider filter. A periodic poll fetches matching items through the providers' `fetch_tasks` surface; a fetch → dedup → enrich → route pipeline drops a todo card onto the dedicated `task-sources` thread board and, for proactive sources, dispatches a triage turn so an agent can start working immediately. The domain mirrors the `cron` layering: `mod.rs` is export-only, business logic lives in sibling modules, persistence is SQLite, and the RPC surface is wired through `schemas.rs`. + +## Responsibilities + +- Persist per-source configs (provider + filter + schedule + routing target + optional pinned connection / static executor). +- Periodically poll enabled sources (`periodic.rs`) on a global 10-minute tick honoring per-source `interval_secs` (floored to 60s). +- Translate a typed `FilterSpec` into the provider-agnostic `TaskFetchFilter` (`filter.rs`) and fetch via the registered Composio provider. +- Dedup ingested items with an edit-aware SHA-256 content hash; re-ingest only when the upstream task changed (`store.rs` + `pipeline.rs`). +- Deterministically enrich raw tasks into agent-ready ones — urgency heuristic, summary, linked assignee, templated agent prompt (`enrich.rs`). +- Route enriched tasks onto the `task-sources` thread board as todo cards and, for proactive sources, dispatch a triage turn through the same path Composio webhooks use (`route.rs`). +- Fire a one-shot fetch when a matching Composio connection is created (`bus.rs`). +- Expose an `openhuman.task_sources_*` RPC surface for CRUD, manual fetch, filter preview, ingested-task listing, and status (`schemas.rs` + `ops.rs`). + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/task_sources/mod.rs` | Export-only: module docstring, `mod`/`pub mod` decls, `pub use` re-exports, and the `all_task_sources_*` controller registry pair. | +| `src/openhuman/task_sources/types.rs` | Serde domain types: `ProviderSlug`, `FilterSpec` (provider-tagged enum), `SourceTarget`, `FetchReason`, `TaskSource`, `TaskSourcePatch`, `EnrichedTask`, `FetchOutcome`. | +| `src/openhuman/task_sources/store.rs` | SQLite persistence (`/task_sources/sources.db`): `task_sources` + `ingested_tasks` tables, dedup `content_hash`, card-id ledger, migrate-on-open. | +| `src/openhuman/task_sources/ops.rs` | RPC-facing business logic returning `RpcOutcome`: `list`/`get`/`add`/`update`/`remove`/`fetch`/`list_tasks`/`preview_filter`/`status`. | +| `src/openhuman/task_sources/schemas.rs` | `task_sources` controller schemas + `all_controller_schemas` / `all_registered_controllers` + thin `handle_*` param parsers delegating to `ops.rs`. | +| `src/openhuman/task_sources/pipeline.rs` | `run_source_once` — the infallible fetch → dedup → enrich → route pass shared by poll, manual RPC, and connection hook; publishes domain events. | +| `src/openhuman/task_sources/filter.rs` | `to_fetch_filter` — flattens a `FilterSpec` variant into the shared `TaskFetchFilter`. | +| `src/openhuman/task_sources/enrich.rs` | Deterministic, dependency-free `enrich_task`: urgency heuristic, summary, linked assignee, agent prompt. No LLM call. | +| `src/openhuman/task_sources/route.rs` | `route_enriched` / `add_card` / `board_cards` — appends todo cards to the `task-sources` board (`TASK_SOURCES_THREAD_ID`), removes stale cards on re-ingest, and dispatches a scheduler-gated triage turn for proactive sources. | +| `src/openhuman/task_sources/periodic.rs` | `start_periodic_poll` — global tick scheduler; per-source due-timing in a process-global map; `run_one_tick` is `pub(crate)` for tests. | +| `src/openhuman/task_sources/bus.rs` | `TaskSourcesConnectionSubscriber` + `register_task_sources_subscriber` — one-shot fetch on `ComposioConnectionCreated`. | +| `src/openhuman/task_sources/store_tests.rs` | Sibling test suite for `store.rs`. | +| `src/openhuman/task_sources/pipeline_tests.rs` | Sibling test suite for `pipeline.rs`. | + +## Public surface + +Re-exported from `mod.rs`: + +- Types: `TaskSource`, `TaskSourcePatch`, `FilterSpec`, `ProviderSlug`, `SourceTarget`, `FetchReason`, `EnrichedTask`, `FetchOutcome`; plus `NormalizedTask` / `TaskFetchFilter` re-exported from the composio providers. +- Functions: `start_periodic_poll`, `run_source_once`. +- Constant: `TASK_SOURCES_THREAD_ID` (`"task-sources"`). +- RPC registry: `all_task_sources_controller_schemas`, `all_task_sources_registered_controllers`, `task_sources_schemas`. + +## RPC / controllers + +Namespace `task_sources` (methods `openhuman.task_sources_`): + +| Function | Description | +| --- | --- | +| `list` | List all configured sources. | +| `get` | Fetch one source by id. | +| `add` | Create a source; missing schedule/target/cap fall back to `[task_sources]` config defaults. | +| `update` | Apply a partial `TaskSourcePatch`. | +| `remove` | Delete a source by id (cascades `ingested_tasks`). | +| `fetch` | Fetch one source now (`FetchReason::Manual`) and route new tasks. | +| `list_tasks` | List recently ingested tasks for a source (newest first, default limit 50). | +| `preview_filter` | Dry-run a filter — fetch matching tasks WITHOUT routing/recording. | +| `status` | Domain master switch + default interval + source counts. | + +Handlers parse params and delegate to `ops.rs`; schemas reference `FilterSpec`, `TaskSource`, `TaskSourcePatch`, `FetchOutcome`, `NormalizedTask`. Registered into the global registry via `src/core/all.rs`. + +## Agent tools + +None. This domain owns no `tools.rs` / agent tools. It *produces* work for agents (todo cards + triage turns) rather than exposing callable tools. + +## Events + +Publishes (via `publish_global`, domain `"task_sources"`): + +- `DomainEvent::TaskSourceFetched` — after a successful fetch pass (counts: fetched/routed/skipped). +- `DomainEvent::TaskSourceTaskIngested` — per newly routed task (provider, external_id, title, urgency). +- `DomainEvent::TaskSourceFetchFailed` — on a failed pass (error string). + +Subscribes: + +- `DomainEvent::ComposioConnectionCreated` (domain filter `["composio"]`) via `TaskSourcesConnectionSubscriber` — fires a one-shot `ConnectionCreated` fetch for matching enabled sources. Registered once at startup (`register_task_sources_subscriber`, idempotent `OnceLock` handle). + +Startup wiring lives in `src/core/jsonrpc.rs` (registers the subscriber and starts the periodic poll). + +## Persistence + +SQLite at `/task_sources/sources.db` (WAL, 5s busy timeout, migrate-on-open): + +- **`task_sources`** — configured sources: provider, optional connection_id/name, enabled, filter JSON, interval_secs, target, max_tasks_per_fetch, created_at, last_fetch_at/last_status, assigned_executor. +- **`ingested_tasks`** — per-(source, external_id) dedup ledger: edit-aware `content_hash` (SHA-256 over title/body/status/updated_at/url), normalized task `payload`, `ingested_at`, and `card_id` (board card UUID) so an edited upstream item removes its stale card before re-routing. FK to `task_sources` with `ON DELETE CASCADE`. + +Additive idempotent column migrations (`add_column_if_missing`) backfill `ingested_tasks.card_id` and `task_sources.assigned_executor` on older DBs. App-level defaults (enabled flag, default interval, per-fetch cap, auto_proactive) live in config (`TaskSourcesConfig`), not the store. + +## Dependencies + +- `crate::core::all` — `ControllerFuture`, `RegisteredController` for the RPC registry. +- `crate::core::event_bus` — `publish_global`, `subscribe_global`, `DomainEvent`, `EventHandler`, `SubscriptionHandle` for event publish/subscribe. +- `crate::openhuman::config` (+ `config::rpc`) — `Config`, `load_config_with_timeout`; reads the `[task_sources]` block for defaults and the master switch. +- `crate::openhuman::memory_sync::composio::providers` — `get_provider`, `ProviderContext`, `NormalizedTask`, `TaskFetchFilter`, `ComposioProvider::fetch_tasks`; the actual external fetch + normalized task shape. +- `crate::openhuman::agent::triage` — `run_triage`, `apply_decision`, `TriageOutcome`, `TriggerEnvelope`; dispatches the proactive agent turn for `AgentTodoProactive` sources. +- `crate::openhuman::todos` (`todos::ops`) — `add`/`remove`, `BoardLocation`, `CardPatch`; the thread-scoped board cards are stored here. Also references `agent::task_board::TaskBoardCard` for `board_cards`. +- `crate::openhuman::scheduler_gate` — `wait_for_capacity` capacity semaphore; gates proactive triage turns behind background-AI throttling. + +## Used by + +- `src/core/all.rs` — registers controllers + schemas into the global RPC registry. +- `src/core/jsonrpc.rs` — at startup registers the connection subscriber and starts the periodic poll. +- `src/core/event_bus/events.rs` — defines/classifies the three `TaskSource*` event variants under domain `"task_sources"`. +- `src/openhuman/config/schema/` — `TaskSourcesConfig` block feeding domain defaults. + +## Notes / gotchas + +- **Periodic cadence is coarse.** `TICK_SECONDS = 600` is the effective lower bound: any `interval_secs` shorter than 10 minutes is rounded up to the tick. A misconfigured `interval_secs = 0` is floored to `MIN_INTERVAL_SECONDS = 60`. The first immediate-fire tick is skipped so startup isn't slammed. +- **Pipeline is infallible at the boundary.** `run_source_once` captures any error into `FetchOutcome::error` (and a failure event) so the scheduler loop never unwinds. +- **Route-then-mark ordering.** A task is marked ingested only after routing succeeds, so a routing failure retries next pass instead of being silently dropped. +- **Edit-aware dedup.** `content_hash` includes `url` deliberately (it drives card notes/metadata and external write-back); a changed hash re-ingests and removes the stale board card via the persisted `card_id`. +- **Static executor routing (G7).** A source's optional `assigned_executor` is pre-stamped onto each card's `assigned_agent` so the dispatcher can run it deterministically without the LLM router. `add` applies it as a follow-up patch to keep `store::add_source`'s signature stable. +- **`route.rs` is the only writer of card `source_metadata`** (provider/source_id/external_id/urgency, plus url and — GitHub-only — repo). +- **`update_source` TOCTOU.** Documented theoretical read-modify-write window across three connections; acceptable at settings-panel scale. +- **Enrichment is intentionally LLM-free** — deterministic and unit-testable; the heavy reasoning happens in the downstream triage turn. +- `clear_all` exists for the E2E `test_reset` RPC. diff --git a/src/openhuman/team/README.md b/src/openhuman/team/README.md new file mode 100644 index 000000000..ac632152d --- /dev/null +++ b/src/openhuman/team/README.md @@ -0,0 +1,90 @@ +# team + +Team management RPC adapters. This domain is a **thin proxy to the hosted backend**: every operation forwards an authenticated HTTP request to the OpenHuman backend (`/teams/*`) and returns the raw JSON response verbatim. It owns **no local state, no domain types, and no server-side authorization** — the backend enforces team ownership, role permissions, and tenant isolation; non-authorized callers receive the backend's 401/403 surfaced as an RPC error string. Covers team CRUD, membership, role changes, invites, usage, and active-team switching. + +## Responsibilities + +- Fetch the current user's active-team usage (`/teams/me/usage`). +- List teams for the authenticated user, fetch a single team, create / update / delete teams. +- Switch the active team, leave a team, join a team via invite code. +- List, create, and revoke team invites. +- Remove members and change member roles. +- Validate inputs (non-empty trimmed ids/names/codes) **before** any network call, with deterministic field-precedence in error messages. +- Percent-encode path segments safely (no path-injection via team/user ids). + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/team/mod.rs` | Export-only: declares `ops`/`schemas`, re-exports all ops fns and the controller-schema pair. | +| `src/openhuman/team/ops.rs` | Business logic — async fns that build a path, attach the session JWT, call the backend, and wrap the response in `RpcOutcome::single_log`. Contains URL-path builder + id-normalization helpers and their unit tests. | +| `src/openhuman/team/schemas.rs` | Controller schemas (`all_team_controller_schemas`, `all_team_registered_controllers`, `team_schemas`), param structs, and `handle_*` adapters that load config, deserialize params, and delegate to `ops`. | +| `src/openhuman/team/schemas_tests.rs` | Sibling test module for `schemas.rs` (wired via `#[path]`). | + +## Public surface + +Re-exported from `mod.rs`: + +- **Ops (all `async fn(... ) -> Result, String>`):** `get_usage`, `list_members`, `list_teams`, `get_team`, `create_team`, `update_team`, `delete_team`, `switch_team`, `leave_team`, `join_team`, `create_invite`, `remove_member`, `change_member_role`, `list_invites`, `revoke_invite`. +- **Schemas:** `all_team_controller_schemas`, `all_team_registered_controllers`, `team_schemas`. + +Internal helpers in `ops.rs` (`require_token`, `normalize_id`, `build_api_path`, `get_authed_value`) are private. + +## RPC / controllers + +Namespace `team`. Registered controllers (RPC method `openhuman.team_`): + +| Method | Backend call | Required inputs | Optional | +| --- | --- | --- | --- | +| `team_get_usage` | `GET /teams/me/usage` | — | — | +| `team_list_members` | `GET /teams/:teamId/members` | `teamId` | — | +| `team_list_teams` | `GET /teams` | — | — | +| `team_get_team` | `GET /teams/:teamId` | `teamId` | — | +| `team_create_team` | `POST /teams` | `name` | — | +| `team_update_team` | `PUT /teams/:teamId` | `teamId` | `name` | +| `team_delete_team` | `DELETE /teams/:teamId` | `teamId` | — | +| `team_switch_team` | `POST /teams/:teamId/switch` | `teamId` | — | +| `team_leave_team` | `POST /teams/:teamId/leave` | `teamId` | — | +| `team_join_team` | `POST /teams/join` | `code` | — | +| `team_create_invite` | `POST /teams/:teamId/invites` | `teamId` | `maxUses`, `expiresInDays` | +| `team_list_invites` | `GET /teams/:teamId/invites` | `teamId` | — | +| `team_revoke_invite` | `DELETE /teams/:teamId/invites/:inviteId` | `teamId`, `inviteId` | — | +| `team_remove_member` | `DELETE /teams/:teamId/members/:userId` | `teamId`, `userId` | — | +| `team_change_member_role` | `PUT /teams/:teamId/members/:userId/role` | `teamId`, `userId`, `role` | — | + +Outputs are raw backend JSON (`result` field; array for list endpoints). Params are camelCase; `handle_*` fns deserialize via `serde_json::from_value` and surface `invalid params: …` on failure. + +## Agent tools + +None. This domain exposes no agent tools. + +## Events + +None. No `bus.rs`; publishes/subscribes to no `DomainEvent`s. + +## Persistence + +None local. State lives in the hosted backend. The only stored value it reads is the app-session JWT, fetched via `crate::api::jwt::get_session_token` (written by `auth_store_session`); it is sent as `Authorization: Bearer …` and never logged. + +## Dependencies + +- `crate::api::config::effective_backend_api_url` — resolves the backend base URL from `Config.api_url`. +- `crate::api::jwt::get_session_token` — pulls the session JWT used to authenticate every request. +- `crate::api::BackendOAuthClient` — HTTP client; `authed_json(token, method, path, body)` performs the authed call. +- `crate::openhuman::config::Config` — config passed into every op; `config::rpc::load_config_with_timeout` loads it inside each `handle_*`. +- `crate::core::all::{ControllerFuture, RegisteredController}` and `crate::core::{ControllerSchema, FieldSchema, TypeSchema}` — controller registry types. +- `crate::rpc::RpcOutcome` — return wrapper (`single_log`). +- `reqwest` (`Method`, `Url`) for HTTP + path building; `serde` / `serde_json` for params and bodies. + +## Used by + +- `src/core/all.rs` — registers `all_team_registered_controllers()` into the controller registry and `all_team_controller_schemas()` into the schema list (the standard controller-only exposure path). No domain branches in `cli.rs` / `jsonrpc.rs`. + +## Notes / gotchas + +- **Pure proxy.** No type modeling of teams/members/invites — everything is `serde_json::Value` passthrough; the backend is the source of truth. +- **Error rendering:** `get_authed_value` maps failures with `{e:#}` (full anyhow chain), deliberately not `e.to_string()`, so the underlying cause (DNS/TLS/timeout/non-2xx) is preserved for Sentry rather than the truncated `backend request GET /teams` label (see the in-code note referencing OPENHUMAN-TAURI-AD / TAURI-B2). +- **Path safety:** `build_api_path` clears and pushes segments through `url::path_segments_mut`, percent-encoding reserved chars, spaces, and Unicode — ids like `team/with?reserved` cannot escape their segment. +- **Validation precedence is deterministic** (tested): `team_id` is normalized before `user_id` before `role`, so error messages are stable regardless of which other fields are also invalid. +- `update_team` only includes `name` in the body when present and non-empty after trimming; an empty/whitespace name is dropped rather than sent. +- Missing/blank session token yields `no backend session token; run auth_store_session first` before any network attempt. diff --git a/src/openhuman/test_support/README.md b/src/openhuman/test_support/README.md new file mode 100644 index 000000000..f78965453 --- /dev/null +++ b/src/openhuman/test_support/README.md @@ -0,0 +1,78 @@ +# test_support + +Test-support domain: wipe-and-reset plus read-only introspection RPCs that let E2E specs drive the in-process core between tests without restarting the process. `openhuman.test_reset` returns the running core to a fresh-install baseline; the `test_support.*` introspection RPCs let specs verify that a UI action actually flowed through to disk and to live Rust state (workspace tree, files, the `IN_FLIGHT` chat map, wallet prepared quotes). Everything here is gated behind the `/rpc` bearer token (only written in debug builds) and, for the destructive reset, an explicit `OPENHUMAN_E2E_MODE` env flag — so it is effectively unreachable in release. + +## Responsibilities + +- Reset persistent core state in-place to the "fresh install" baseline: no authenticated user (`active_user.toml` removed, `api_key` cleared), onboarding not completed (`onboarding_completed=false`, `chat_onboarding_completed=false`), no cron jobs, and a wiped memory tree (chunks, summaries, content dirs, sync cursors). +- Short-circuit and surface errors on any individual wipe step (partial resets are treated as worse than a clear failure). +- Gate `reset` behind `OPENHUMAN_E2E_MODE` (`1`/`true`/`TRUE`/`yes`/`YES`). +- Provide narrow, read-only introspection RPCs: resolve the active workspace root, list workspace files (depth- and count-capped), read a workspace file (lossy UTF-8, 1 MiB cap), snapshot the in-flight chat map, and snapshot the wallet prepared-quote store. +- Enforce workspace-jail path safety: reject `..` escapes, canonicalize the root (handles macOS `/var`→`/private/var` symlink), and skip symlinks during directory walks so listings can't escape the workspace. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/test_support/mod.rs` | Export-only module: declares `introspect`, `rpc`, `schemas`; re-exports `all_test_support_controller_schemas` / `all_test_support_registered_controllers`. | +| `src/openhuman/test_support/rpc.rs` | `openhuman.test_reset` implementation — `reset()` wipes cron, memory tree, config fields, and active user; returns a `ResetSummary`. Includes the `OPENHUMAN_E2E_MODE` guard and inline tests. | +| `src/openhuman/test_support/introspect.rs` | Read-only introspection RPCs: `workspace_root`, `list_workspace_files`, `read_workspace_file`, `in_flight_chats`, `wallet_prepared_quotes`, plus the `resolve_workspace_relative` path guard and BFS `walk_dir`. | +| `src/openhuman/test_support/schemas.rs` | `ControllerSchema` definitions, the registered-controller list, and `handle_*` dispatchers that delegate to `rpc`/`introspect` and serialize via `RpcOutcome::into_cli_compatible_json`. | + +## Public surface + +From `mod.rs`: +- `all_test_support_controller_schemas()` — `Vec` for the six controllers. +- `all_test_support_registered_controllers()` — `Vec` (schema + handler pairs). + +From `rpc` / `introspect` (used by handlers, also `pub`): +- `rpc::reset() -> RpcOutcome`, `rpc::reset_json()` (raw JSON envelope convenience). +- `introspect::workspace_root()`, `list_workspace_files(rel_root, max_depth)`, `read_workspace_file(rel_path, max_bytes)`, `in_flight_chats()`, `wallet_prepared_quotes()`. +- Result types: `ResetSummary`, `WorkspaceRoot`, `ListEntry`/`ListResult`, `ReadFileResult`, `InFlightEntryView`/`InFlightResult`, `PreparedQuotesResult`. + +## RPC / controllers + +Six controllers, registered into the global registry via `src/core/all.rs`: + +| Namespace.function | Inputs | Purpose | +| --- | --- | --- | +| `test.reset` | none | Wipe auth, onboarding, cron, and memory tree to a fresh-install baseline. Requires `OPENHUMAN_E2E_MODE`. | +| `test_support.workspace_root` | none | Return active `workspace_dir` path + existence. | +| `test_support.list_workspace_files` | `rel_root?`, `max_depth?` | Recursive listing, capped at depth 6 / 2000 entries. | +| `test_support.read_workspace_file` | `rel_path`, `max_bytes?` | Lossy-UTF-8 file read, capped at 1 MiB; rejects `..` escapes. | +| `test_support.in_flight_chats` | none | Snapshot the `IN_FLIGHT` chat map (`(client_id, thread_id)` → `request_id`). | +| `test_support.wallet_prepared_quotes` | none | Snapshot the in-memory wallet prepared-quote store. | + +Note the `reset` controller uses namespace `test` (method `openhuman.test_reset`) while the introspection controllers use namespace `test_support`. + +## Persistence + +This module owns no state of its own — it mutates/reads state owned by other domains: +- **Wipes**: cron jobs (`cron::clear_all_jobs`), memory tree rows/content dirs/sync state (`memory::read_rpc::wipe_all_rpc`), config fields (`onboarding_completed`, `chat_onboarding_completed`, `api_key`), and `active_user.toml` (`config::clear_active_user` under `default_root_openhuman_dir`). +- **Reads**: workspace files under `Config::workspace_dir`, the in-process `IN_FLIGHT` chat map, and the in-memory wallet prepared-quote store. + +## Dependencies + +- `crate::openhuman::config` — `Config::load_or_init`/`save`, `workspace_dir`, `clear_active_user`, `default_root_openhuman_dir`; config is the source of truth for the workspace root and the fields reset wipes. +- `crate::openhuman::cron` — `clear_all_jobs` to wipe scheduled jobs during reset. +- `crate::openhuman::memory::read_rpc` — `wipe_all_rpc` to clear memory-tree rows, content dirs, and sync state. +- `crate::openhuman::channels::providers::web` — `in_flight_entries_for_test` to snapshot the live `IN_FLIGHT` chat map. +- `crate::openhuman::wallet` — `prepared_quotes_for_test` and `PreparedTransaction` to snapshot prepared quotes. +- `crate::core::all` — `ControllerFuture`, `RegisteredController` for handler wiring. +- `crate::core::{ControllerSchema, FieldSchema, TypeSchema}` — controller schema types. +- `crate::rpc::RpcOutcome` — standard RPC result envelope. + +## Used by + +- `src/core/all.rs` — registers this module's controllers and schemas into the global RPC registry (`all_test_support_registered_controllers` / `all_test_support_controller_schemas`). +- `src/openhuman/mod.rs` — declares `pub mod test_support`. +- Consumed at runtime by E2E specs (WDIO) calling `openhuman.test_reset` and `openhuman.test_support_*` over JSON-RPC. + +## Notes / gotchas + +- **Double gating**: `reset` is fail-closed behind both the `/rpc` bearer token (debug-only token file) and `OPENHUMAN_E2E_MODE`; introspection RPCs rely on the bearer token alone. Not reachable in release builds by design. +- **In-process reset**: the core process is not restarted; specs reload the webview afterward so the renderer also starts blank. +- **Add new persistent state to `reset`**: per the module docstring, any new domain that survives `test_reset` is a leak that lets specs interfere — extend `rpc::reset` when adding persistent state. +- **Path-jail symlink handling**: `resolve_workspace_relative` canonicalizes the root first (macOS `/var`→`/private/var`); `walk_dir` uses `symlink_metadata` and skips symlinks so listings can't follow links out of the workspace. +- **`read_workspace_file` byte accounting**: `returned_bytes` is the raw byte count before lossy UTF-8 conversion (which substitutes U+FFFD and can change length) so specs can assert byte-accurate truncation. +- **`list_workspace_files` schema omits `entries`** for brevity, but the runtime payload includes them; `max_depth` defaults to 2, clamped to 6; entry cap is 2000. diff --git a/src/openhuman/text_input/README.md b/src/openhuman/text_input/README.md new file mode 100644 index 000000000..e4fe74def --- /dev/null +++ b/src/openhuman/text_input/README.md @@ -0,0 +1,70 @@ +# text_input + +Text input intelligence — read, insert, and preview ("ghost text") suggestions in the OS-focused input field. A thin orchestration layer: it owns the request/response types, RPC controller surface, and a standalone CLI, but delegates **all** platform work (focus inspection, text application, overlay rendering) to `accessibility::*`. Consumed by autocomplete, voice control, and other text-aware features. + +## Responsibilities + +- Read the currently focused text field's contents, app name, role, selection, and (optionally) screen bounds; flag whether the field belongs to a terminal app. +- Insert text into the focused field, with optional focus validation (expected app / role) before writing. +- Show a ghost-text overlay near the focused field with a TTL auto-dismiss, and dismiss it. +- Accept a ghost suggestion atomically (dismiss overlay → validate focus → insert). +- Expose all of the above as `text_input.*` JSON-RPC controllers. +- Provide a standalone `openhuman text-input` CLI for testing read/insert/ghost/dismiss flows and a lightweight dev JSON-RPC server, without launching the full desktop app. + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/text_input/mod.rs` | Export-only. Re-exports `ops` (as `rpc`), `types`, and the schema registry pair (`all_text_input_controller_schemas` / `all_text_input_registered_controllers`). | +| `src/openhuman/text_input/types.rs` | Serde request/response types (`ReadFieldParams`/`Result`, `InsertTextParams`/`Result`, `ShowGhostTextParams`/`Result`, `DismissGhostTextResult`, `AcceptGhostTextParams`/`Result`, `FieldBounds`) + `FieldBounds ↔ accessibility::ElementBounds` conversions. | +| `src/openhuman/text_input/ops.rs` | Business logic (canonical handler file). Async fns `read_field`, `insert_text`, `show_ghost`, `dismiss_ghost`, `accept_ghost` returning `RpcOutcome`; all delegate to `accessibility::*`. | +| `src/openhuman/text_input/schemas.rs` | Controller schemas + `handle_*` wrappers that deserialize params and call `ops`; defines `all_controller_schemas` / `all_registered_controllers`. | +| `src/openhuman/text_input/cli.rs` | `openhuman text-input ` CLI: `run` (dev JSON-RPC server on port 7798), `read`, `insert`, `ghost`, `dismiss`. | + +## Public surface + +From `mod.rs` re-exports: + +- `ops::*` (also aliased as `rpc`): `read_field`, `insert_text`, `show_ghost`, `dismiss_ghost`, `accept_ghost`. +- `types::*`: `ReadFieldParams`, `ReadFieldResult`, `InsertTextParams`, `InsertTextResult`, `ShowGhostTextParams`, `ShowGhostTextResult`, `DismissGhostTextResult`, `AcceptGhostTextParams`, `AcceptGhostTextResult`, `FieldBounds`. +- `all_text_input_controller_schemas()` / `all_text_input_registered_controllers()` — registry entry points. +- `cli::run_text_input_command(args)` — `pub(crate)` CLI dispatch. + +## RPC / controllers + +Namespace `text_input` (5 controllers): + +| Method | Inputs | Output | +| --- | --- | --- | +| `text_input.read_field` | `include_bounds?` (bool) | `ReadFieldResult` (`app_name`, `role`, `text`, `selected_text`, `bounds?`, `is_terminal`) | +| `text_input.insert_text` | `text` (required), `validate_focus?`, `expected_app?`, `expected_role?` | `InsertTextResult` (`inserted`, `error?`) | +| `text_input.show_ghost` | `text` (required), `ttl_ms?` (default 3000), `bounds?` (JSON `{x,y,width,height}`) | `ShowGhostTextResult` (`shown`, `error?`) | +| `text_input.dismiss_ghost` | — | `DismissGhostTextResult` (`dismissed`) | +| `text_input.accept_ghost` | `text` (required), `validate_focus?`, `expected_app?`, `expected_role?` | `AcceptGhostTextResult` (`inserted`, `error?`) | + +Registered into the global registry via `src/core/all.rs` (`controllers.extend(...)` and `schemas.extend(...)`). The capability catalog (`about_app` via `all.rs`) describes it as "Read, insert, and preview text in the OS-focused input field." + +## Dependencies + +- **`openhuman::accessibility`** — does all platform work: `focused_text_context_verbose` (read field), `is_terminal_app`, `apply_text_to_focused_field` (insert), `show_overlay` / `hide_overlay` (ghost text), `validate_focused_target` (focus validation), and `ElementBounds` (bounds type mirrored by `FieldBounds`). +- **`core::all`** — `ControllerFuture`, `RegisteredController` for controller registration. +- **`core::{ControllerSchema, FieldSchema, TypeSchema}`** — schema definitions. +- **`rpc::RpcOutcome`** — handler return wrapper. +- **`core::logging`**, **`core::types`** (`RpcRequest`/`RpcSuccess`/`RpcFailure`/`RpcError`), **`core::jsonrpc`** (`default_state`, `invoke_method`) — used only by the dev CLI in `cli.rs`. + +## Used by + +- `src/core/all.rs` — registers the controllers/schemas and the capability-catalog description. +- `src/core/cli.rs` — dispatches the `text-input` top-level subcommand to `cli::run_text_input_command`. + +(Note: `src/openhuman/voice/server.rs` references its own `super::text_input` symbol, not this domain module.) + +## Notes / gotchas + +- **Pure orchestration, no state.** No `store.rs`, no persistence, no event-bus subscribers, no agent tools, no config reads — every call resolves against live OS focus at request time. +- **Error contract differs per method.** `insert_text` / `accept_ghost` wrap platform failures as `Ok(RpcOutcome { value: { inserted: false, error: Some(..) } })` (never `Err`), so JSON-RPC callers always get a structured result. `read_field` and `show_ghost` may bubble accessibility errors up as `Err` (e.g. when reading focused-field bounds fails). Empty `text` is rejected with `Err` up front. +- **`dismiss_ghost` is idempotent** — it discards any `hide_overlay()` error and always returns `dismissed: true`. +- **Focus validation triggers** when `validate_focus` is true **or** either `expected_app` / `expected_role` is set. +- **`show_ghost` bounds fallback:** if no `bounds` provided, it reads the focused field; if that has no bounds, defaults to `{0,0,200,24}`. +- **`read_field` params tolerate omitted `include_bounds`** (`Option` + `#[serde(default)]`); its handler uses `deserialize_params_or_default` so an empty param map yields defaults rather than erroring. +- **Dev CLI** (`text-input run`) stands up a minimal Axum server (`/health`, `/rpc`) routing through `core::jsonrpc::invoke_method`, default port 7798. Useful for exercising the domain from a terminal without the desktop app. Unit tests deliberately only pin guard-clause/validation logic since post-guard paths require a live OS display. diff --git a/src/openhuman/threads/README.md b/src/openhuman/threads/README.md new file mode 100644 index 000000000..a4f601641 --- /dev/null +++ b/src/openhuman/threads/README.md @@ -0,0 +1,98 @@ +# threads + +Conversation thread and message management. Owns the RPC surface and controller registry for thread lifecycle (create / list / upsert / delete / purge), per-thread message CRUD, AI-assisted thread titling, persisted in-flight turn snapshots (for cold-boot recovery of interrupted agent turns), the per-thread kanban task board passthrough, and a one-shot welcome-agent → orchestrator workspace migration. Persistence is delegated to `memory::conversations` (JSONL thread/message store) and to a dedicated `turn_state` snapshot store; this module is the RPC/controller layer over both. + +## Responsibilities + +- List, create (`upsert` with caller-supplied id or `create_new` with auto-generated id + placeholder title), delete, and purge conversation threads. +- List, append, and metadata-patch messages within a thread. +- Update thread labels and user-specified titles. +- Generate a durable thread title from the first user message + assistant reply via the inference provider, with deterministic fallbacks (derive title from the user message; skip non-placeholder titles). +- Maintain restart-survivable snapshots of in-flight agent turns (`turn_state`): get / list / clear over RPC; written by the web-channel progress bridge via `TurnStateMirror`. +- Expose a per-thread kanban task board get/put that proxies to `agent::task_board`. +- On thread delete/purge, invalidate the in-process web-channel session and clean up orphaned turn snapshots. +- One-shot idempotent migration of legacy welcome-agent artifacts (strip `onboarding` label, rename `welcome*` session transcripts/markdown to `orchestrator*`). + +## Key files + +| File | Role | +| --- | --- | +| `src/openhuman/threads/mod.rs` | Export-focused: module docstring, `mod` decls, re-exports (`ThreadsError`, `THREAD_NOT_FOUND_KIND`, controller schema/registry pair, welcome-migration entry point). | +| `src/openhuman/threads/ops.rs` | Business logic / RPC entry points returning `RpcOutcome>`. All thread + message + turn-state ops live here; wraps results in `ApiEnvelope` with request-id/count meta. | +| `src/openhuman/threads/schemas.rs` | `ControllerSchema` definitions, `all_controller_schemas`, `all_registered_controllers`, and `handle_*` functions delegating to `ops`. Also hosts the `task_board_get`/`task_board_put` handlers (proxy to `agent::task_board`). | +| `src/openhuman/threads/error.rs` | `ThreadsError` taxonomy (`NotFound` / `Message`); encodes `NotFound` as a `StructuredRpcError` (`kind: "ThreadNotFound"`, `expected_user_state: true`) at the controller boundary so the frontend handles stale thread refs without string-matching or Sentry noise. | +| `src/openhuman/threads/title.rs` | Pure, provider-free helpers: placeholder-title detection, raw-title sanitization, whitespace collapse, fallback title from user message, prompt builder, log fingerprint. Heavily unit-tested. | +| `src/openhuman/threads/welcome_migration.rs` | One-shot, marker-guarded migration of legacy welcome-agent threads/transcripts to orchestrator naming. | +| `src/openhuman/threads/turn_state/mod.rs` | Submodule export hub for in-flight turn snapshots. | +| `src/openhuman/threads/turn_state/types.rs` | Wire/storage types: `TurnState`, `TurnLifecycle`, `TurnPhase`, `ToolTimelineEntry`/`Status`, `SubagentActivity`/`ToolCall`, and the get/list/clear request/response payloads. camelCase to mirror `chatRuntimeSlice.ts`. | +| `src/openhuman/threads/turn_state/store.rs` | `TurnStateStore` — atomic per-thread JSON snapshot store (tempfile + persist + dir fsync), process-wide mutex; `put`/`get`/`delete`/`list`/`clear_all`/`mark_all_interrupted` plus free-fn wrappers. | +| `src/openhuman/threads/turn_state/mirror.rs` | `TurnStateMirror` — translates `agent::progress::AgentProgress` events into `TurnState` mutations, flushing at iteration/tool boundaries; deletes snapshot on completion, flags `Interrupted` otherwise. | +| `*_tests.rs` | Sibling test suites: `ops_tests.rs`, `schemas_tests.rs`, `turn_state/store_tests.rs`, `turn_state/mirror_tests.rs` (plus inline tests in `error.rs`, `title.rs`, `welcome_migration.rs`). | + +## Public surface + +From `mod.rs`: +- `ThreadsError`, `THREAD_NOT_FOUND_KIND` (re-exported from `error`). +- `all_threads_controller_schemas` / `all_threads_registered_controllers` (re-exported from `schemas`). +- `migrate_welcome_agent_artifacts`, `WelcomeMigrationResult` (re-exported from `welcome_migration`). + +From `turn_state`: `TurnStateMirror`, `TurnStateStore`, `TurnState` + lifecycle/phase/timeline/subagent types and the turn-state RPC request/response types. + +## RPC / controllers + +Namespace `threads` (JSON-RPC `openhuman.threads_`). Schemas + handlers registered via `all_registered_controllers`: + +| Function | Op | +| --- | --- | +| `list` | List thread summaries. | +| `upsert` | Create/refresh a thread (caller id, title, created_at, optional labels). | +| `create_new` | New thread with auto id + `Chat