feat: fix external MCP server integration and add streaming tool-call support

Rebased and cleaned-up version of PR #113 by @mricharz, resolved against current main (including Codex engine, Gemini thought_signature, and agent manager fixes merged since the original PR). MCP Transport & Client: - StreamableHTTPTransport with session tracking, SSE parsing, timeouts - MCPClient.initialize() sends proper MCP handshake (protocolVersion, capabilities, clientInfo) + notifications/initialized - Fix StdioTransport constructor: command=[command] + args - MCPRequest.to_dict() with notification support (id=None) External MCP Discovery: - _discover_external_mcp supports both url (HTTP) and command (stdio) - Per-server include_tools / exclude_tools filtering - MCP clients persisted on JarvisSystem for runtime lifetime Streaming Tool-Call Support (stream_full): - StreamChunk dataclass in _stubs.py (content, tool_calls, finish_reason, usage) - Default stream_full() on InferenceEngine ABC wraps stream() for backward compat - _OpenAICompatibleEngine.stream_full() with SSE parsing - CloudEngine: _stream_full_openai (OpenAI/OpenRouter/MiniMax/Codex routing) _stream_full_anthropic (event-based → OpenAI delta format) - InstrumentedEngine, MultiEngine: stream_full delegation - GuardrailsEngine: stream_full with post-hoc security scanning (FIXED: original PR bypassed output scanning — now accumulates and scans like stream()) Other improvements: - _prepare_anthropic_messages() extracted to eliminate duplication - Default tool_choice=auto when tools are provided (OpenAI compat engines) - MCP tool injection into managed agent streaming path - Documentation: docs/user-guide/mcp-external-servers.md Tests: ~59 new tests across 8 test files, all passing. Closes PR #113 Co-Authored-By: mricharz <mricharz@users.noreply.github.com> Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-07-31 03:12:16 +00:00 · 2026-03-27 18:09:54 +00:00
parent 957ce83955
commit b39dbedcc4
27 changed files with 3228 additions and 438 deletions
@@ -626,7 +626,20 @@ These events enable the telemetry and trace systems to record detailed interacti

 ## Managed Agent Streaming

-The Managed Agent API (`/v1/managed-agents/{id}/messages`) supports real-time SSE streaming. Send a message with `stream: true` to receive the agent's response as a Server-Sent Events stream instead of the default asynchronous queue mode.
+The Managed Agent API (`/v1/managed-agents/{id}/messages`) supports **real LLM token streaming** via SSE. Send a message with `stream: true` to receive the model's response tokens as they are generated, rather than waiting for the full response.
+
+### How It Works
+
+The streaming endpoint calls `engine.stream_full()` directly, which yields `StreamChunk` objects containing content tokens, tool-call fragments, and finish reasons. This provides genuine token-by-token streaming from the LLM -- not a post-hoc word replay.
+
+For multi-turn tool-calling agents, the streaming loop automatically:
+
+1. Yields content tokens to the client as they arrive.
+2. Accumulates tool-call fragments (OpenAI sends these incrementally).
+3. Executes tools when `finish_reason="tool_calls"` is received.
+4. Emits tool results as named SSE events (`event: tool_result`).
+5. Feeds results back to the LLM for the next turn.
+6. Repeats until the model produces a final text response or `max_turns` is reached.

 ### Streaming Messages

@@ -639,19 +652,21 @@ curl -N -X POST http://localhost:8000/v1/managed-agents/{id}/messages \
 The response follows the OpenAI SSE format:

 1. **Content chunks** -- `data: {"choices": [{"delta": {"content": "token"}}]}`
-2. **Tool results** (if the agent used tools) -- `event: tool_results\ndata: {"results": [...]}`
-3. **Final chunk** -- `data: {"choices": [{"delta": {}, "finish_reason": "stop"}]}`
-4. **Done sentinel** -- `data: [DONE]`
+2. **Tool calls** (if the model requests tool use) -- `event: tool_calls\ndata: {"calls": [{"tool_name": "...", "arguments": "..."}]}`
+3. **Tool results** -- `event: tool_result\ndata: {"tool_name": "...", "output": "..."}`
+4. **Final chunk** -- `data: {"choices": [{"delta": {}, "finish_reason": "stop"}]}`
+5. **Done sentinel** -- `data: [DONE]`

 When `stream: false` (the default), the endpoint behaves exactly as before -- the message is queued and the agent must be triggered separately via `/run`.

 ### Behavior Details

- The user message is always stored in the database before the agent runs.
- After streaming completes, the full agent response is persisted as an `agent_to_user` message.
- The agent is instantiated from the managed agent's stored `agent_type` and `config`.
- Conversation history from prior messages is automatically loaded as context.
+- The user message is always stored in the database before streaming starts.
+- After streaming completes, the full collected response is persisted as an `agent_to_user` message.
+- Conversation history from prior messages is automatically loaded as LLM context.
+- The engine's `stream_full()` method is used for real token streaming. Engines that do not override it fall back to the default implementation which wraps the plain `stream()` method.
 - If the engine is not available on the server, a `503` error is returned.
+- Tool execution during streaming uses the `ToolRegistry` to find and instantiate tools.

 ### Python Example

@@ -0,0 +1,158 @@
+# External MCP Server Integration
+
+OpenJarvis can extend agent capabilities by connecting to external [Model Context Protocol (MCP)](https://modelcontextprotocol.io/) servers. This allows agents to use tools provided by services like Home Assistant, databases, custom APIs, or any MCP-compatible server -- without writing custom tool code.
+
+## How It Works
+
+When OpenJarvis starts, it reads the `[tools.mcp]` section in `config.toml`. For each configured server, it:
+
+1. Opens a connection using the appropriate transport (Streamable HTTP or stdio).
+2. Performs the MCP initialize handshake (protocol version negotiation and `initialized` notification).
+3. Discovers available tools via `tools/list`.
+4. Wraps each discovered tool as a standard `BaseTool` so agents can call them like any built-in tool.
+
+If a server is unreachable or returns an error, OpenJarvis logs a warning and continues loading the remaining servers. One broken server does not prevent other tools from being available.
+
+## Configuration
+
+External MCP servers are configured in `config.toml` under `[tools.mcp]`:
+
+```toml
+[tools.mcp]
+enabled = true
+servers = '[{"name": "homeassistant", "url": "http://172.16.3.1:9583/private_abc123"}]'
+```
+
+The `servers` value is a **JSON-encoded string** containing an array of server objects. Each object defines one external MCP server.
+
+!!! note
+    The value must be a JSON string (with single-quote TOML delimiters around it), not a native TOML array. This is because the configuration system passes it through as a single string field.
+
+## Server Config Schema
+
+Each server object supports the following fields:
+
+| Field            | Type           | Required | Description                                              |
+|------------------|----------------|----------|----------------------------------------------------------|
+| `name`           | string         | No       | Human-readable name used in log messages. Defaults to `<unnamed>`. |
+| `url`            | string         | No*      | URL for Streamable HTTP transport.                       |
+| `command`         | string         | No*      | Command to launch a stdio-based MCP server.              |
+| `args`           | list of strings| No       | Arguments passed to the stdio command.                   |
+| `include_tools`  | list of strings| No       | Whitelist of tool names to import. Only these tools are loaded. |
+| `exclude_tools`  | list of strings| No       | Blacklist of tool names to skip. All other tools are loaded. |
+
+*Either `url` or `command` must be provided. If neither is set, the server is skipped with a warning.
+
+When both `include_tools` and `exclude_tools` are specified, the whitelist is applied first, then the blacklist filters the result.
+
+## Examples
+
+### Home Assistant via Streamable HTTP
+
+Connect to the [ha-mcp](https://github.com/tevonsb/ha-mcp) Home Assistant add-on:
+
+```toml
+[tools.mcp]
+enabled = true
+servers = '[{"name": "homeassistant", "url": "http://172.16.3.1:9583/private_abc123"}]'
+```
+
+This discovers all HA tools (entity control, automations, history, etc.) and makes them available to agents.
+
+### Stdio Server
+
+Launch a local MCP server as a subprocess:
+
+```toml
+[tools.mcp]
+enabled = true
+servers = '[{"name": "myserver", "command": "python", "args": ["-m", "my_mcp_server"]}]'
+```
+
+OpenJarvis starts the process automatically, communicates via JSON-RPC over stdin/stdout, and terminates it on shutdown.
+
+### Multiple Servers
+
+```toml
+[tools.mcp]
+enabled = true
+servers = '[{"name": "homeassistant", "url": "http://172.16.3.1:9583/private_abc123"}, {"name": "database", "command": "db-mcp-server", "args": ["--db", "postgres://localhost/mydb"]}]'
+```
+
+### Tool Filtering
+
+When a server exposes many tools but you only need a few, use `include_tools` to whitelist:
+
+```toml
+[tools.mcp]
+enabled = true
+servers = '[{"name": "ha", "url": "http://172.16.3.1:9583/private_abc123", "include_tools": ["hassTurnOn", "hassTurnOff", "hassGetState"]}]'
+```
+
+To load everything except specific tools, use `exclude_tools`:
+
+```toml
+[tools.mcp]
+enabled = true
+servers = '[{"name": "ha", "url": "http://172.16.3.1:9583/private_abc123", "exclude_tools": ["hassCreateBackup", "hassDeleteBackup"]}]'
+```
+
+## Transport Types
+
+### Streamable HTTP
+
+Used when the `url` field is set. The transport sends JSON-RPC requests as HTTP POST to the given URL using `httpx`. It tracks the `Mcp-Session-Id` header across requests as required by the MCP Streamable HTTP specification.
+
+**When to use:** Remote MCP servers, services running as HTTP endpoints (e.g., Home Assistant MCP add-on, cloud-hosted MCP servers).
+
+**Connection parameters:**
+
+- Connect timeout: 10 seconds
+- Request timeout: 60 seconds
+
+### Stdio
+
+Used when the `command` field is set. OpenJarvis spawns the command as a subprocess and communicates via JSON-RPC lines on stdin/stdout.
+
+**When to use:** Local MCP servers distributed as CLI tools, development/testing, servers that require filesystem access on the same machine.
+
+!!! info "SSETransport alias"
+    `SSETransport` is provided as a backward-compatible alias for `StreamableHTTPTransport`. Both refer to the same implementation.
+
+## Error Handling
+
+OpenJarvis handles MCP server failures gracefully:
+
+- **Server unreachable:** A warning is logged and the server is skipped. All other servers and built-in tools continue to load normally.
+- **Timeout:** HTTP requests time out after 60 seconds. The server is skipped with a warning.
+- **Invalid config:** If the `servers` JSON is malformed or a server entry has neither `url` nor `command`, a warning is logged and that entry is skipped.
+- **Tool discovery failure:** If `tools/list` fails on a server, the error is caught and the server is skipped.
+- **Runtime tool call failure:** If a tool call to an external MCP server fails at runtime, it returns a `ToolResult` with `success=False` and the error message.
+
+No single server failure causes OpenJarvis to crash or prevents other tools from working.
+
+## Troubleshooting
+
+### Server not discovered
+
+1. Check that `[tools.mcp]` has `enabled = true`.
+2. Verify the `servers` JSON is valid. A common mistake is using TOML arrays instead of a JSON string.
+3. Check the OpenJarvis logs for warnings like `Failed to discover external MCP tools`.
+
+### Connection refused / timeout
+
+1. Verify the server is running and reachable from the OpenJarvis host: `curl -v http://host:port/`.
+2. Check firewall rules between the OpenJarvis container and the MCP server.
+3. For Docker deployments, ensure both containers are on the same network or use host IPs.
+
+### Tools not appearing
+
+1. Run with debug logging to see which tools were discovered.
+2. Check if `include_tools` or `exclude_tools` filters are too restrictive.
+3. Verify the MCP server actually exposes tools via `tools/list` (some servers only expose resources or prompts).
+
+### Stdio server crashes immediately
+
+1. Test the command manually: `python -m my_mcp_server` should start and wait for input on stdin.
+2. Check stderr output in the OpenJarvis logs for error messages from the subprocess.
+3. Ensure all dependencies for the MCP server are installed in the same environment.
@@ -204,7 +204,7 @@ All built-in tools are registered via `@ToolRegistry.register()` and are availab
 | **Scheduler** | `pause_scheduled_task` | Pause an active scheduled task |
 | **Scheduler** | `resume_scheduled_task` | Resume a paused scheduled task |
 | **Scheduler** | `cancel_scheduled_task` | Cancel a scheduled task permanently |
-| **Integration** | `mcp_adapter` | Bridge to external MCP tool servers |
+| **Integration** | `mcp_adapter` | Bridge to external MCP tool servers (see [External MCP Servers](mcp-external-servers.md)) |

 ---