feat(mcp): ship MCP registry — unhide UI, transport-aware install, reconfigure, registry creds (#3039) (#3058)

Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai>

docs/MCP_SETUP_AGENT.md

@@ -1,12 +1,16 @@
-# MCP Setup Agent — design sketch
+# MCP Setup Agent
 
 A sub-agent that walks the user through installing, configuring, and
 connecting an MCP server from one of the upstream registries
 (`mcp_registry::registries`: Smithery, modelcontextprotocol/registry).
 
-This document is a **design sketch** for follow-up implementation. Nothing
-here is wired up yet beyond the underlying primitives in
-`src/openhuman/mcp_registry/`.
+**Status: implemented** (issue #3039). The agent archetype, its five
+`mcp_setup_*` tools, the opaque-secret request/submit flow, and the
+`install_and_connect` commit path are all live. The orchestrator delegates to
+it via the `setup_mcp_server` delegate (the `mcp_setup` entry in
+`src/openhuman/agent_registry/agents/orchestrator/agent.toml`), so a chat turn
+like *"set up the Notion MCP server"* routes here. The sections below describe
+how the flow works; paths reflect the shipped layout.
 
 ---
 
@@ -31,7 +35,7 @@ subprocess, and the persistence.
 
 ## Tool surface
 
-Four tools registered behind a `mcp_setup_*` namespace. All tool inputs
+Five tools registered behind a `mcp_setup_*` namespace. All tool inputs
 and outputs are JSON; secret values **never** appear in either direction.
 
 | Tool | Input | Output | Notes |
@@ -40,7 +44,7 @@ and outputs are JSON; secret values **never** appear in either direction.
 | `mcp_setup_get` | `{ qualified_name }` | `{ detail, required_env_keys }` | Wraps `registry_get`; pre-computes `required_env_keys` from the `config_schema` (same logic as `ops::collect_required_env_keys`). |
 | `mcp_setup_request_secret` | `{ key_name, prompt }` | `{ ref: "secret://<opaque>" }` | Triggers an out-of-band UI prompt. Returns an opaque ref; raw value is held in a process-local in-memory map keyed by ref. |
 | `mcp_setup_test_connection` | `{ qualified_name, env_refs: { KEY: "secret://…" } }` | `{ ok, tools?: [McpTool], error?: string }` | Spawns the candidate subprocess in a **scratch** workspace, resolves refs to values just-in-time, runs `initialize` + `tools/list`, tears it down. No persistence. |
-| `mcp_setup_install_and_connect` | `{ qualified_name, env_refs }` | `{ server_id, status, tools: [McpTool] }` | Resolves refs, persists the install + `mcp_client_env` rows, calls `connections::connect`. Refs are consumed (removed from the in-memory map) regardless of outcome. |
+| `mcp_setup_install_and_connect` | `{ qualified_name, env_refs }` | `{ server_id, status, tools: [McpTool] }` | Resolves refs, persists the install + `mcp_client_env` rows, calls `connections::connect`. Refs are always consumed (removed from the in-memory map) regardless of outcome — on failure the agent must re-prompt via `mcp_setup_request_secret`. |
 
 ---
 
@@ -68,10 +72,11 @@ Lifecycle of `SETUP_SECRETS`:
 - Process-local `OnceLock<RwLock<HashMap<RefId, SecretEntry>>>`.
 - Entries TTL out after, say, 15 min (defends against stranded secrets if
   the conversation is abandoned mid-flow).
-- `mcp_setup_install_and_connect` consumes refs on success: pulls each
-  value, writes it to the `mcp_client_env` table (existing persistence,
-  already keyed by `server_id`), removes the ref. On failure refs are
-  left intact so the agent can retry without re-prompting the user.
+- `mcp_setup_install_and_connect` consumes refs regardless of outcome:
+  pulls each value, writes it to the `mcp_client_env` table (existing
+  persistence, already keyed by `server_id`), and removes the ref from
+  the in-memory map. On failure the agent should re-prompt via
+  `mcp_setup_request_secret` to collect fresh refs for a retry.
 - On core shutdown the map is dropped — refs do not survive restart.
 
 `RefId` is a short random hex string. **No structure or hint of the
@@ -96,17 +101,18 @@ values.
 
 ## Where the agent lives
 
-Follow the existing sub-agent pattern (`src/openhuman/agent/harness/`):
+Follows the existing sub-agent pattern (`src/openhuman/agent_registry/`):
 
-- New archetype TOML at `app/src/lib/ai/agents/mcp_setup.toml` (loaded by
-  `AgentDefinitionRegistry::init_global`).
-- Prompt + tool allowlist scoped tight: only the four `mcp_setup_*` tools
-  plus the standard `chat` / `ask_user` primitives. **No** general
-  filesystem, network, or shell tools — the agent shouldn't be able to
-  exfiltrate a leaked ref even if one shows up.
-- Triggered by the main agent via `spawn_subagent("mcp_setup", { goal })`
-  or by an explicit UI affordance ("Add MCP server…" button that opens a
-  thread pinned to this archetype).
+- Archetype TOML at `src/openhuman/agent_registry/agents/mcp_setup/agent.toml`
+  (loaded by the agent registry loader).
+- Prompt + tool allowlist scoped tight: only the five `mcp_setup_*` tools
+  plus `ask_user_clarification`. **No** general filesystem, network, or shell
+  tools — the agent shouldn't be able to exfiltrate a leaked ref even if one
+  shows up. (`submit_secret` is intentionally NOT in the agent allowlist — the
+  UI calls it out-of-band via the socket bridge.)
+- Triggered by the orchestrator's `setup_mcp_server` delegate (the `mcp_setup`
+  entry in `agent_registry/agents/orchestrator/agent.toml`), or directly from
+  chat when the user asks to add/install/set up an MCP server.
 
 ---
 
@@ -127,7 +133,8 @@ Following the project's `Specify → Rust → JSON-RPC → UI → tests` flow:
    modal listening on a new socket event `mcp_setup_request_secret`),
    submit POSTs the value to a Tauri command that calls into core to
    register the ref.
-4. **Archetype** + system prompt at `app/src/lib/ai/agents/mcp_setup.toml`.
+4. **Archetype** + system prompt at
+   `src/openhuman/agent_registry/agents/mcp_setup/agent.toml`.
 5. **Tests**:
    - Unit: ref lifecycle (mint → resolve → consume → TTL expiry).
    - Integration (`tests/mcp_registry_e2e.rs` style): full flow against

docs/TEST-COVERAGE-MATRIX.md

@@ -418,6 +418,9 @@ Canonical mapping of every product feature to its test source(s). Drives gap-fil
 | 11.1.6 | SearXNG MCP search         | RU    | `src/openhuman/integrations/searxng.rs`, `src/openhuman/mcp_server/tools.rs`, `src/openhuman/tools/schemas.rs`      | ✅     | Self-hosted search config, normalized results, MCP argument validation, and mocked HTTP execution |
 | 11.1.7 | Bundled prompt resources   | RU    | `src/openhuman/mcp_server/resources.rs`, `src/openhuman/mcp_server/protocol.rs`                                     | ✅     | `resources/list` catalog + `resources/read` happy path, -32002 unknown URI, -32602 missing param, catalog-mirrors-BUILTINS parity test |
 | 11.1.8 | Resource templates list    | RU    | `src/openhuman/mcp_server/resources.rs`, `src/openhuman/mcp_server/protocol.rs`                                     | ✅     | `resources/templates/list` returns `{resourceTemplates: []}` (static catalog), tolerates unknown/cursor params |
+| 11.1.10 | MCP registry install→connect→tool_call | RI    | `tests/json_rpc_e2e.rs` (`mcp_clients_install_connect_tool_call_happy_path`), `tests/mcp_registry_e2e.rs`, `src/openhuman/mcp_registry/setup_ops.rs` (this PR, #3039) | ✅     | HTTP-RPC happy path install→connect→tool_call→update_env against `test-mcp-stub`; transport-aware install (stdio + http_remote) via `build_install_transport` |
+| 11.1.11 | MCP env reconfigure + registry creds | RI/VU | `tests/json_rpc_e2e.rs` (`mcp_clients_registry_settings_roundtrip`), `src/openhuman/mcp_registry/registries/mcp_official.rs`, `app/src/components/channels/mcp/InstalledServerDetail.test.tsx` (this PR, #3039) | ✅     | `update_env` persist+reconnect; `registry_settings` get/set with secrets write-only (config-first, env-fallback); reconfigure form validation |
+| 11.1.12 | MCP UI surface + setup-agent client | VU    | `app/src/components/channels/mcp/InstallDialog.test.tsx`, `app/src/services/api/mcpClientsApi.test.ts`, `app/src/services/api/mcpSetupApi.test.ts` (this PR, #3039) | ✅     | Skills `?tab=mcp` renders `McpServersTab` (not Coming Soon); auto-connect on install (best-effort); typed `mcpSetupApi` wrapper |
 
 <!-- 11.1.9 Vault Markdown Writes — removed: Knowledge Vaults dropped (vault domain + VaultPanel deleted). -->