admin/OpenJarvis
1
0
Fork 0
mirror of https://github.com/open-jarvis/OpenJarvis.git synced 2026-07-27 21:05:34 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat: Morning Digest with Voice Mode + unified OAuth + Apple connectors (#174)

Browse Source
This commit is contained in:
Jon Saad-Falcon
2026-04-03 11:05:41 -07:00
committed by GitHub
parent 867a96fd6e
commit a009646071
122 changed files with 7496 additions and 15388 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+4
View File
@@ -90,3 +90,7 @@ CLAUDE.md
research_mining_*
.python-version
src/openjarvis/channels/whatsapp_baileys_bridge/node_modules/
# SQLite in-memory artifacts
:memory:
MagicMock/
configs/openjarvis/prompts/personas/jarvis.md
+30
View File
@@ -0,0 +1,30 @@
You are Jarvis — the local AI assistant. You are loyal, efficient, dry-witted, and genuinely care about the person you serve. You have a warm British sensibility: polite but never obsequious, witty but never frivolous.
PERSONALITY:
- You anticipate needs before being asked
- You deliver bad news with constructive dry wit: "Your rebuttals appear to have slipped past their deadline, sir. I'd suggest making them your first order of business — before anyone notices."
- Your humor is understated — a raised eyebrow in voice form
- You are calm under pressure and never flustered
- You treat the briefing as a conversation with someone you respect, not a status report
ADDRESS:
- Use the user's preferred honorific (provided in the system prompt)
- Use it 2-3 times per briefing: once in greeting, once mid-briefing, once in closing
- Never every sentence — that would be a parody, not Jarvis
EMAIL TRIAGE:
- Important emails are from REAL PEOPLE (not automated senders, newsletters, or marketing)
- Prioritize emails that need a REPLY or DECISION, or contain a DEADLINE
- Skip promotional, automated, and notification emails entirely
- For important emails, mention the sender name and what they need
MESSAGE TRIAGE (iMessage, Slack, etc.):
- Highlight messages from key people and threads needing a reply
- Briefly acknowledge casual threads so the user knows you checked: "Your group chat has been lively but nothing requiring a response"
- Skip reactions, emoji-only messages, and automated notifications
CONSTRAINTS:
- ONLY report facts present in the provided data. Never invent.
- NEVER describe actions you are taking (adjusting lights, ordering food, queuing playlists, etc.)
- No markdown formatting, no emojis, no bullet points, no headers — this is spoken aloud
- If a data source is disconnected or errored, skip it silently — do not mention connection issues
configs/openjarvis/prompts/personas/neutral.md
+11
View File
@@ -0,0 +1,11 @@
You are an AI assistant generating a daily briefing. Be clear, concise, and factual.
## Voice & Tone
- Straightforward and professional
- No personality or humor — just the facts
- Use plain language
## Structure
- Open with the date and a one-line summary
- Deliver each section with bullet points
- Close with a summary of action items
docs/superpowers/plans/2026-03-25-deep-research-phase1.md
-2895
View File
File diff suppressed because it is too large Load Diff
docs/superpowers/plans/2026-03-25-deep-research-phase2a-connectors.md
-1896
View File
File diff suppressed because it is too large Load Diff
docs/superpowers/plans/2026-03-25-deep-research-phase3.md
-1270
View File
File diff suppressed because it is too large Load Diff
docs/superpowers/plans/2026-03-26-deep-research-agent-v2.md
-798
View File
@@ -1,798 +0,0 @@
# Deep Research Agent v2 Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Add `knowledge_sql` and `scan_chunks` tools to the DeepResearchAgent, rewrite the system prompt with query strategies, wire in the `think` tool, and validate with 4 test queries.
**Architecture:** Two new tool files following the existing `BaseTool`/`ToolSpec`/`ToolResult` pattern. The `knowledge_sql` tool runs read-only SQL against `KnowledgeStore._conn`. The `scan_chunks` tool pulls chunks from the store, batches them, and calls the LM to extract findings. The system prompt teaches the model when to use each tool. Everything wired together in `_launch_chat`.
**Tech Stack:** Python 3.10+, sqlite3, pytest
---
### Task 1: Create `knowledge_sql` tool
**Files:**
- Create: `src/openjarvis/tools/knowledge_sql.py`
- Create: `tests/tools/test_knowledge_sql.py`
- [ ] **Step 1: Write the test file**
Create `tests/tools/__init__.py` if it doesn't exist, then create `tests/tools/test_knowledge_sql.py`:
```python
"""Tests for KnowledgeSQLTool."""
from __future__ import annotations
import tempfile
from pathlib import Path
import pytest
from openjarvis.connectors.store import KnowledgeStore
from openjarvis.core.registry import ToolRegistry
@pytest.fixture()
def store(tmp_path: Path) -> KnowledgeStore:
ks = KnowledgeStore(str(tmp_path / "test.db"))
ks.store("Hello from Alice", source="imessage", author="Alice", doc_type="message")
ks.store("Hello from Alice again", source="imessage", author="Alice", doc_type="message")
ks.store("Meeting notes Q1", source="granola", author="Bob", doc_type="document")
ks.store("Email about Spain trip", source="gmail", author="Carol", doc_type="email")
return ks
def test_select_count(store: KnowledgeStore) -> None:
from openjarvis.tools.knowledge_sql import KnowledgeSQLTool
tool = KnowledgeSQLTool(store=store)
result = tool.execute(query="SELECT COUNT(*) as total FROM knowledge_chunks")
assert result.success
assert "4" in result.content
def test_group_by_author(store: KnowledgeStore) -> None:
from openjarvis.tools.knowledge_sql import KnowledgeSQLTool
tool = KnowledgeSQLTool(store=store)
result = tool.execute(
query="SELECT author, COUNT(*) as n FROM knowledge_chunks GROUP BY author ORDER BY n DESC"
)
assert result.success
assert "Alice" in result.content
assert "2" in result.content
def test_rejects_non_select(store: KnowledgeStore) -> None:
from openjarvis.tools.knowledge_sql import KnowledgeSQLTool
tool = KnowledgeSQLTool(store=store)
result = tool.execute(query="DELETE FROM knowledge_chunks")
assert not result.success
assert "read-only" in result.content.lower() or "SELECT" in result.content
def test_rejects_drop(store: KnowledgeStore) -> None:
from openjarvis.tools.knowledge_sql import KnowledgeSQLTool
tool = KnowledgeSQLTool(store=store)
result = tool.execute(query="DROP TABLE knowledge_chunks")
assert not result.success
def test_handles_bad_sql(store: KnowledgeStore) -> None:
from openjarvis.tools.knowledge_sql import KnowledgeSQLTool
tool = KnowledgeSQLTool(store=store)
result = tool.execute(query="SELECT * FROM nonexistent_table")
assert not result.success
def test_filter_by_source(store: KnowledgeStore) -> None:
from openjarvis.tools.knowledge_sql import KnowledgeSQLTool
tool = KnowledgeSQLTool(store=store)
result = tool.execute(
query="SELECT title, author FROM knowledge_chunks WHERE source = 'gmail'"
)
assert result.success
assert "Carol" in result.content
def test_registered() -> None:
from openjarvis.tools.knowledge_sql import KnowledgeSQLTool
ToolRegistry.register_value("knowledge_sql", KnowledgeSQLTool)
assert ToolRegistry.contains("knowledge_sql")
```
- [ ] **Step 2: Run tests to verify they fail**
```bash
uv run pytest tests/tools/test_knowledge_sql.py -v --tb=short
```
Expected: FAIL — `ModuleNotFoundError`
- [ ] **Step 3: Create the tool**
Create `src/openjarvis/tools/knowledge_sql.py`:
```python
"""KnowledgeSQLTool — read-only SQL queries against the KnowledgeStore.
Allows agents to run SELECT queries for aggregation, counting, ranking,
and filtering operations that BM25 search cannot handle.
"""
from __future__ import annotations
import sqlite3
from typing import Any, Optional
from openjarvis.connectors.store import KnowledgeStore
from openjarvis.core.registry import ToolRegistry
from openjarvis.core.types import ToolResult
from openjarvis.tools._stubs import BaseTool, ToolSpec
_MAX_ROWS = 50
_SCHEMA_DESCRIPTION = (
"Table: knowledge_chunks\n"
"Columns: id, content, source, doc_type, doc_id, title, author, "
"participants, timestamp, thread_id, url, metadata, chunk_index"
)
@ToolRegistry.register("knowledge_sql")
class KnowledgeSQLTool(BaseTool):
"""Run read-only SQL against the knowledge store for aggregation queries."""
tool_id = "knowledge_sql"
def __init__(self, store: Optional[KnowledgeStore] = None) -> None:
self._store = store
@property
def spec(self) -> ToolSpec:
return ToolSpec(
name="knowledge_sql",
description=(
"Run a read-only SQL SELECT query against the knowledge_chunks table. "
"Use for counting, ranking, aggregation, and filtering. "
f"{_SCHEMA_DESCRIPTION}"
),
parameters={
"type": "object",
"properties": {
"query": {
"type": "string",
"description": (
"SQL SELECT query. Only SELECT statements allowed. "
"Example: SELECT author, COUNT(*) as n FROM knowledge_chunks "
"WHERE source='imessage' GROUP BY author ORDER BY n DESC LIMIT 10"
),
},
},
"required": ["query"],
},
category="knowledge",
)
def execute(self, **params: Any) -> ToolResult:
if self._store is None:
return ToolResult(
tool_name="knowledge_sql",
content="No knowledge store configured.",
success=False,
)
query: str = params.get("query", "").strip()
if not query:
return ToolResult(
tool_name="knowledge_sql",
content="No query provided.",
success=False,
)
# Read-only enforcement
normalized = query.lstrip().upper()
if not normalized.startswith("SELECT"):
return ToolResult(
tool_name="knowledge_sql",
content="Only SELECT queries are allowed (read-only).",
success=False,
)
# Block dangerous keywords even in SELECT subqueries
for forbidden in ("DROP", "DELETE", "INSERT", "UPDATE", "ALTER", "CREATE", "ATTACH"):
if forbidden in normalized:
return ToolResult(
tool_name="knowledge_sql",
content=f"Query contains forbidden keyword: {forbidden}. Only SELECT queries allowed.",
success=False,
)
try:
rows = self._store._conn.execute(query).fetchmany(_MAX_ROWS)
except sqlite3.OperationalError as exc:
return ToolResult(
tool_name="knowledge_sql",
content=f"SQL error: {exc}",
success=False,
)
if not rows:
return ToolResult(
tool_name="knowledge_sql",
content="Query returned no results.",
success=True,
metadata={"num_rows": 0},
)
# Format as table
columns = rows[0].keys()
lines = [" | ".join(columns)]
lines.append(" | ".join("---" for _ in columns))
for row in rows:
lines.append(" | ".join(str(row[c]) for c in columns))
return ToolResult(
tool_name="knowledge_sql",
content="\n".join(lines),
success=True,
metadata={"num_rows": len(rows)},
)
__all__ = ["KnowledgeSQLTool"]
```
- [ ] **Step 4: Ensure tests/tools/__init__.py exists**
```bash
touch tests/tools/__init__.py
```
- [ ] **Step 5: Run tests**
```bash
uv run pytest tests/tools/test_knowledge_sql.py -v --tb=short
```
Expected: 7/7 PASS.
- [ ] **Step 6: Commit**
```bash
git add src/openjarvis/tools/knowledge_sql.py tests/tools/__init__.py tests/tools/test_knowledge_sql.py
git commit -m "feat: add knowledge_sql tool for read-only SQL aggregation queries"
```
---
### Task 2: Create `scan_chunks` tool
**Files:**
- Create: `src/openjarvis/tools/scan_chunks.py`
- Create: `tests/tools/test_scan_chunks.py`
- [ ] **Step 1: Write the test file**
Create `tests/tools/test_scan_chunks.py`:
```python
"""Tests for ScanChunksTool."""
from __future__ import annotations
from pathlib import Path
from typing import Any, Dict
from unittest.mock import MagicMock
import pytest
from openjarvis.connectors.store import KnowledgeStore
from openjarvis.core.registry import ToolRegistry
@pytest.fixture()
def store(tmp_path: Path) -> KnowledgeStore:
ks = KnowledgeStore(str(tmp_path / "test.db"))
ks.store("Met with Sequoia about Series A", source="granola", doc_type="document")
ks.store("Fundraising discussion with a]16z", source="granola", doc_type="document")
ks.store("Weekly standup notes", source="granola", doc_type="document")
ks.store("Trip to Spain with family", source="imessage", doc_type="message")
return ks
def _fake_engine() -> MagicMock:
"""Mock engine that echoes back a summary."""
engine = MagicMock()
engine.generate.return_value = {
"content": "Found: Sequoia Series A discussion, a16z fundraising",
"usage": {},
}
return engine
def test_scan_finds_semantic_matches(store: KnowledgeStore) -> None:
from openjarvis.tools.scan_chunks import ScanChunksTool
engine = _fake_engine()
tool = ScanChunksTool(store=store, engine=engine, model="test")
result = tool.execute(question="Which VCs have I spoken with?")
assert result.success
assert "Sequoia" in result.content or "Found" in result.content
# Engine should have been called with chunks
assert engine.generate.called
def test_scan_respects_source_filter(store: KnowledgeStore) -> None:
from openjarvis.tools.scan_chunks import ScanChunksTool
engine = _fake_engine()
tool = ScanChunksTool(store=store, engine=engine, model="test")
result = tool.execute(question="What trips?", source="imessage")
assert result.success
# Should only have passed imessage chunks to the engine
call_args = engine.generate.call_args
messages = call_args[0][0] if call_args[0] else call_args[1].get("messages", [])
# At least one call should contain "Spain"
all_content = str(messages)
assert "Spain" in all_content
def test_scan_empty_store(tmp_path: Path) -> None:
from openjarvis.tools.scan_chunks import ScanChunksTool
ks = KnowledgeStore(str(tmp_path / "empty.db"))
engine = _fake_engine()
tool = ScanChunksTool(store=ks, engine=engine, model="test")
result = tool.execute(question="Anything?")
assert result.success
assert "no chunks" in result.content.lower() or result.content == ""
def test_registered() -> None:
from openjarvis.tools.scan_chunks import ScanChunksTool
ToolRegistry.register_value("scan_chunks", ScanChunksTool)
assert ToolRegistry.contains("scan_chunks")
```
- [ ] **Step 2: Run tests to verify they fail**
```bash
uv run pytest tests/tools/test_scan_chunks.py -v --tb=short
```
Expected: FAIL — `ModuleNotFoundError`
- [ ] **Step 3: Create the tool**
Create `src/openjarvis/tools/scan_chunks.py`:
```python
"""ScanChunksTool — semantic grep via LM-powered chunk scanning.
Pulls chunks from the KnowledgeStore by filter, batches them, and asks the
LM to extract information relevant to a question. Catches semantic matches
that keyword-based BM25 search misses.
"""
from __future__ import annotations
from typing import Any, List, Optional
from openjarvis.connectors.store import KnowledgeStore
from openjarvis.core.registry import ToolRegistry
from openjarvis.core.types import Message, Role, ToolResult
from openjarvis.engine._stubs import InferenceEngine
from openjarvis.tools._stubs import BaseTool, ToolSpec
_DEFAULT_MAX_CHUNKS = 200
_DEFAULT_BATCH_SIZE = 20
@ToolRegistry.register("scan_chunks")
class ScanChunksTool(BaseTool):
"""Semantic grep — feeds chunks to the LM to find information BM25 misses."""
tool_id = "scan_chunks"
def __init__(
self,
store: Optional[KnowledgeStore] = None,
engine: Optional[InferenceEngine] = None,
model: str = "",
) -> None:
self._store = store
self._engine = engine
self._model = model
@property
def spec(self) -> ToolSpec:
return ToolSpec(
name="scan_chunks",
description=(
"Semantic search — feeds chunks from the knowledge store to an LM "
"that reads the actual text looking for relevant information. "
"Use when keyword search (knowledge_search) misses semantic matches "
"(e.g. searching for 'VCs' when text says 'fundraising round'). "
"Slower but catches what BM25 misses. "
"Filters: source, doc_type, since, until."
),
parameters={
"type": "object",
"properties": {
"question": {
"type": "string",
"description": "What to look for in the chunks.",
},
"source": {
"type": "string",
"description": "Filter by source (e.g. 'granola', 'gmail').",
},
"doc_type": {
"type": "string",
"description": "Filter by doc type (e.g. 'document', 'email').",
},
"since": {
"type": "string",
"description": "Only chunks after this ISO timestamp.",
},
"until": {
"type": "string",
"description": "Only chunks before this ISO timestamp.",
},
"max_chunks": {
"type": "integer",
"description": f"Max chunks to scan (default {_DEFAULT_MAX_CHUNKS}).",
},
},
"required": ["question"],
},
category="knowledge",
)
def execute(self, **params: Any) -> ToolResult:
if self._store is None or self._engine is None:
return ToolResult(
tool_name="scan_chunks",
content="Scan tool not configured (missing store or engine).",
success=False,
)
question: str = params.get("question", "")
if not question:
return ToolResult(
tool_name="scan_chunks",
content="No question provided.",
success=False,
)
source: str = params.get("source", "")
doc_type: str = params.get("doc_type", "")
since: str = params.get("since", "")
until: str = params.get("until", "")
max_chunks: int = int(params.get("max_chunks", _DEFAULT_MAX_CHUNKS))
batch_size: int = _DEFAULT_BATCH_SIZE
# Pull chunks matching filters
where_clauses: List[str] = []
sql_params: List[Any] = []
if source:
where_clauses.append("source = ?")
sql_params.append(source)
if doc_type:
where_clauses.append("doc_type = ?")
sql_params.append(doc_type)
if since:
where_clauses.append("timestamp >= ?")
sql_params.append(since)
if until:
where_clauses.append("timestamp <= ?")
sql_params.append(until)
where = ""
if where_clauses:
where = "WHERE " + " AND ".join(where_clauses)
sql = f"SELECT content, source, title, author FROM knowledge_chunks {where} LIMIT ?"
sql_params.append(max_chunks)
rows = self._store._conn.execute(sql, sql_params).fetchall()
if not rows:
return ToolResult(
tool_name="scan_chunks",
content="No chunks found matching filters.",
success=True,
metadata={"chunks_scanned": 0},
)
# Batch and scan
findings: List[str] = []
for i in range(0, len(rows), batch_size):
batch = rows[i : i + batch_size]
batch_text = "\n\n---\n\n".join(
f"[{row['source']}] {row['title']} by {row['author']}:\n{row['content']}"
for row in batch
)
messages = [
Message(
role=Role.USER,
content=(
f"/no_think\n"
f"Extract any information relevant to this question: {question}\n\n"
f"If nothing is relevant, reply with exactly: NOTHING_RELEVANT\n\n"
f"Chunks:\n{batch_text}"
),
),
]
result = self._engine.generate(
messages, model=self._model, max_tokens=1024
)
content = result.get("content", "").strip()
if content and "NOTHING_RELEVANT" not in content:
findings.append(content)
if not findings:
return ToolResult(
tool_name="scan_chunks",
content=f"Scanned {len(rows)} chunks — no relevant information found.",
success=True,
metadata={"chunks_scanned": len(rows)},
)
return ToolResult(
tool_name="scan_chunks",
content="\n\n".join(findings),
success=True,
metadata={"chunks_scanned": len(rows), "batches_with_findings": len(findings)},
)
__all__ = ["ScanChunksTool"]
```
- [ ] **Step 4: Run tests**
```bash
uv run pytest tests/tools/test_scan_chunks.py -v --tb=short
```
Expected: 4/4 PASS.
- [ ] **Step 5: Commit**
```bash
git add src/openjarvis/tools/scan_chunks.py tests/tools/test_scan_chunks.py
git commit -m "feat: add scan_chunks tool for LM-powered semantic grep"
```
---
### Task 3: Rewrite system prompt + wire all tools + tune loop guard
**Files:**
- Modify: `src/openjarvis/agents/deep_research.py` (system prompt + max_turns)
- Modify: `src/openjarvis/cli/deep_research_setup_cmd.py` (`_launch_chat` function)
- [ ] **Step 1: Replace `DEEP_RESEARCH_SYSTEM_PROMPT` in `deep_research.py`**
Replace the entire `DEEP_RESEARCH_SYSTEM_PROMPT` string (from line 33 to line 55) with:
```python
DEEP_RESEARCH_SYSTEM_PROMPT = """\
/no_think
You are a deep research agent with access to a personal knowledge base \
containing emails, messages, meeting notes, documents, and notes.
## Your Tools
- **knowledge_search**: BM25 keyword search. Best for finding specific topics, \
names, or phrases. Use filters (source, author, since, until) to narrow results.
- **knowledge_sql**: Run read-only SQL against the knowledge_chunks table. \
Schema: knowledge_chunks(id, content, source, doc_type, doc_id, title, \
author, participants, timestamp, thread_id, url, metadata, chunk_index). \
Best for counting, ranking, and aggregation. \
Example: SELECT author, COUNT(*) as n FROM knowledge_chunks \
WHERE source='imessage' GROUP BY author ORDER BY n DESC LIMIT 10
- **scan_chunks**: Semantic search — feeds chunks to an LM that reads the \
actual text looking for relevant information. Use when keyword search returns \
nothing useful or when you need semantic matching (e.g. searching for 'VCs' \
when text says 'fundraising round'). Slower but catches what BM25 misses.
- **think**: Reasoning scratchpad. Use between searches to plan your next \
query, evaluate findings so far, and identify gaps.
## Strategy
1. Start with **think** to plan your approach — which tools suit this query?
2. For "who/what/how many" queries → use **knowledge_sql** with GROUP BY
3. For specific topics or names → use **knowledge_search**
4. If keyword search returns nothing useful → try **scan_chunks** with filters
5. Cross-reference across sources (emails, messages, meeting notes)
6. After gathering evidence → write a cited narrative report
## Citation Format
Cite sources as: [source] title -- author
End with a Sources section listing all referenced items.
## Rules
- Always cite your sources. Never present information without attribution.
- Make at least two searches to cross-reference across different sources.
- If a search returns no results, try a different tool or rephrase the query.
- Prefer specificity: filter by source, author, or date when appropriate.
- Your final answer should be a coherent narrative, not a list of raw results."""
```
- [ ] **Step 2: Update `_default_max_turns` from 5 to 8**
In `deep_research.py`, change:
```python
_default_max_turns = 8
```
- [ ] **Step 3: Update `_launch_chat` in `deep_research_setup_cmd.py` to wire all 4 tools**
Replace the tool setup section in `_launch_chat` (around lines 293-303):
```python
def _launch_chat(store: KnowledgeStore, console: Console) -> None:
"""Start an interactive Deep Research chat session."""
from openjarvis.agents.deep_research import DeepResearchAgent
from openjarvis.connectors.retriever import TwoStageRetriever
from openjarvis.engine.ollama import OllamaEngine
from openjarvis.tools.knowledge_search import KnowledgeSearchTool
from openjarvis.tools.knowledge_sql import KnowledgeSQLTool
from openjarvis.tools.scan_chunks import ScanChunksTool
from openjarvis.tools.think import ThinkTool
console.print("\n[bold]Setting up Deep Research agent...[/bold]")
# Engine
engine = OllamaEngine()
if not engine.health():
console.print(
"[red]Ollama is not running.[/red] Start it with: "
"[bold]ollama serve[/bold]"
)
return
models = engine.list_models()
if _OLLAMA_MODEL not in models and f"{_OLLAMA_MODEL}:latest" not in models:
base_name = _OLLAMA_MODEL.split(":")[0]
matching = [m for m in models if m.startswith(base_name)]
if not matching:
console.print(
f"[yellow]Model {_OLLAMA_MODEL} not found.[/yellow] "
f"Pull it with: [bold]ollama pull {_OLLAMA_MODEL}[/bold]"
)
return
# Tools
retriever = TwoStageRetriever(store)
tools = [
KnowledgeSearchTool(retriever=retriever),
KnowledgeSQLTool(store=store),
ScanChunksTool(store=store, engine=engine, model=_OLLAMA_MODEL),
ThinkTool(),
]
# Agent
agent = DeepResearchAgent(
engine=engine,
model=_OLLAMA_MODEL,
tools=tools,
interactive=True,
)
console.print(
f"[green]Ready![/green] Using [bold]{_OLLAMA_MODEL}[/bold] via Ollama.\n"
"Tools: knowledge_search, knowledge_sql, scan_chunks, think\n"
"Type your research question. Type [bold]/quit[/bold] to exit.\n"
)
# REPL
while True:
try:
query = console.input("[bold blue]research>[/bold blue] ").strip()
except (EOFError, KeyboardInterrupt):
break
if not query:
continue
if query.lower() in ("/quit", "/exit", "quit", "exit"):
break
try:
result = agent.run(query)
console.print(f"\n{result.content}\n")
if result.metadata and result.metadata.get("sources"):
console.print("[dim]Sources:[/dim]")
for s in result.metadata["sources"]:
console.print(f" [dim]- {s}[/dim]")
console.print()
except Exception as exc: # noqa: BLE001
console.print(f"[red]Error: {exc}[/red]\n")
```
- [ ] **Step 4: Run all tests**
```bash
uv run pytest tests/agents/test_deep_research.py tests/agents/test_deep_research_integration.py tests/tools/test_knowledge_sql.py tests/tools/test_scan_chunks.py -v --tb=short
```
Expected: All PASS.
- [ ] **Step 5: Lint**
```bash
uv run ruff check src/openjarvis/agents/deep_research.py src/openjarvis/cli/deep_research_setup_cmd.py src/openjarvis/tools/knowledge_sql.py src/openjarvis/tools/scan_chunks.py
```
- [ ] **Step 6: Commit**
```bash
git add src/openjarvis/agents/deep_research.py src/openjarvis/cli/deep_research_setup_cmd.py
git commit -m "feat: rewrite DeepResearch system prompt, wire SQL + scan + think tools, increase max_turns to 8"
```
---
### Task 4: Evaluate with 4 test queries
**Files:** None (manual testing)
- [ ] **Step 1: Run the 4 test queries**
```python
import time
from openjarvis.agents.deep_research import DeepResearchAgent
from openjarvis.connectors.retriever import TwoStageRetriever
from openjarvis.connectors.store import KnowledgeStore
from openjarvis.engine.ollama import OllamaEngine
from openjarvis.tools.knowledge_search import KnowledgeSearchTool
from openjarvis.tools.knowledge_sql import KnowledgeSQLTool
from openjarvis.tools.scan_chunks import ScanChunksTool
from openjarvis.tools.think import ThinkTool
store = KnowledgeStore()
engine = OllamaEngine()
retriever = TwoStageRetriever(store)
tools = [
KnowledgeSearchTool(retriever=retriever),
KnowledgeSQLTool(store=store),
ScanChunksTool(store=store, engine=engine, model="qwen3.5:9b"),
ThinkTool(),
]
agent = DeepResearchAgent(engine=engine, model="qwen3.5:9b", tools=tools, max_turns=8)
queries = [
"When was my most recent trip to Spain?",
"Which VCs have I spoken with since 2023?",
"Who are the 10 people I have spoken with the most over text?",
"What meetings take up most of my time based on my calendar and meeting logs?",
]
for i, query in enumerate(queries, 1):
print(f"QUERY {i}: {query}")
t0 = time.time()
result = agent.run(query)
print(result.content[:3000])
print(f"[Turns: {result.turns}, Tools: {len(result.tool_results)}, Time: {time.time()-t0:.0f}s]\n")
```
- [ ] **Step 2: Compare to v1 results**
v1 returned empty or "no data found" for all 4 queries. Success = at least 3 of 4 produce substantive answers.
- [ ] **Step 3: Push**
```bash
git push origin feat/deep-research-setup
```
docs/superpowers/plans/2026-03-26-deep-research-phase2b-api.md
-361
View File
@@ -1,361 +0,0 @@
# Deep Research Phase 2B-i: Connector Management API Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Add FastAPI endpoints for connector management so the desktop wizard and CLI can list sources, initiate OAuth, check sync status, and trigger syncs — all via HTTP.
**Architecture:** A new `connectors` API router exposes endpoints for listing available connectors, initiating auth flows, checking connection status, triggering syncs, and streaming sync progress. The router wraps the existing `ConnectorRegistry`, `SyncEngine`, and `KnowledgeStore` from Phase 1. An OAuth callback server handles the browser redirect during auth flows.
**Tech Stack:** Python 3.10+, FastAPI (existing server infrastructure), pytest + httpx (test client)
**Spec:** `docs/superpowers/specs/2026-03-25-deep-research-setup-design.md` — Sections 4, 10
**Depends on:** Phase 1 (ConnectorRegistry, BaseConnector, SyncEngine, KnowledgeStore)
---
## File Structure
```
src/openjarvis/server/
├── connectors_router.py # FastAPI router: /v1/connectors/* endpoints
tests/server/
├── test_connectors_router.py # API endpoint tests
```
---
### Task 1: Connectors API Router
**Files:**
- Create: `src/openjarvis/server/connectors_router.py`
- Create: `tests/server/test_connectors_router.py`
- [ ] **Step 1: Write failing tests**
Create `tests/server/test_connectors_router.py`:
```python
"""Tests for the /v1/connectors API endpoints."""
from __future__ import annotations
import json
from pathlib import Path
from unittest.mock import MagicMock, patch
import pytest
from openjarvis.server.connectors_router import create_connectors_router
@pytest.fixture
def app():
"""Create a minimal FastAPI app with the connectors router."""
try:
from fastapi import FastAPI
from fastapi.testclient import TestClient
except ImportError:
pytest.skip("fastapi not installed")
app = FastAPI()
router = create_connectors_router()
app.include_router(router, prefix="/v1")
return TestClient(app)
def test_list_connectors(app) -> None:
"""GET /v1/connectors returns list of available connectors."""
resp = app.get("/v1/connectors")
assert resp.status_code == 200
data = resp.json()
assert isinstance(data, list)
# Should have at least obsidian (always available)
ids = [c["connector_id"] for c in data]
assert "obsidian" in ids
def test_connector_detail(app) -> None:
"""GET /v1/connectors/{id} returns connector info."""
resp = app.get("/v1/connectors/obsidian")
assert resp.status_code == 200
data = resp.json()
assert data["connector_id"] == "obsidian"
assert data["auth_type"] == "filesystem"
assert "connected" in data
def test_connector_not_found(app) -> None:
"""GET /v1/connectors/{id} returns 404 for unknown."""
resp = app.get("/v1/connectors/nonexistent")
assert resp.status_code == 404
def test_connect_obsidian(app, tmp_path: Path) -> None:
"""POST /v1/connectors/obsidian/connect with path."""
vault = tmp_path / "vault"
vault.mkdir()
(vault / "note.md").write_text("# Test")
resp = app.post(
"/v1/connectors/obsidian/connect",
json={"path": str(vault)},
)
assert resp.status_code == 200
assert resp.json()["connected"] is True
def test_disconnect(app) -> None:
"""POST /v1/connectors/obsidian/disconnect."""
resp = app.post("/v1/connectors/obsidian/disconnect")
assert resp.status_code == 200
def test_sync_status(app) -> None:
"""GET /v1/connectors/obsidian/sync returns sync status."""
resp = app.get("/v1/connectors/obsidian/sync")
assert resp.status_code == 200
data = resp.json()
assert "state" in data
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `cd /lambda/nfs/lambda-stanford/jonsf/scratch_v2/OpenJarvis && uv run pytest tests/server/test_connectors_router.py -v`
Expected: FAIL — `ModuleNotFoundError`
- [ ] **Step 3: Implement connectors router**
Create `src/openjarvis/server/connectors_router.py`:
```python
"""FastAPI router for connector management: /v1/connectors/*
Exposes endpoints for listing connectors, connecting/disconnecting,
checking status, and triggering syncs. Used by the desktop wizard
and CLI.
"""
from __future__ import annotations
import logging
from typing import Any, Dict, List, Optional
from openjarvis.connectors._stubs import SyncStatus
from openjarvis.core.registry import ConnectorRegistry
logger = logging.getLogger(__name__)
# Connector instances cache (keyed by connector_id)
_instances: Dict[str, Any] = {}
def _get_or_create(connector_id: str, **kwargs: Any) -> Any:
"""Get a cached connector instance or create one."""
if connector_id not in _instances:
cls = ConnectorRegistry.get(connector_id)
_instances[connector_id] = cls(**kwargs)
return _instances[connector_id]
def create_connectors_router():
"""Factory that returns a FastAPI APIRouter for /v1/connectors.
Separated from module-level to allow lazy import of FastAPI.
"""
from fastapi import APIRouter, HTTPException
from pydantic import BaseModel
router = APIRouter(tags=["connectors"])
class ConnectRequest(BaseModel):
path: Optional[str] = None
token: Optional[str] = None
code: Optional[str] = None
email: Optional[str] = None
password: Optional[str] = None
class ConnectorInfo(BaseModel):
connector_id: str
display_name: str
auth_type: str
connected: bool
# Import connectors to trigger registration
import openjarvis.connectors # noqa: F401
@router.get("/connectors")
def list_connectors() -> List[Dict[str, Any]]:
"""List all available data source connectors."""
results = []
for key in sorted(ConnectorRegistry.keys()):
cls = ConnectorRegistry.get(key)
try:
inst = _get_or_create(key)
connected = inst.is_connected()
except Exception:
connected = False
results.append(
{
"connector_id": key,
"display_name": getattr(
cls, "display_name", key
),
"auth_type": getattr(cls, "auth_type", "unknown"),
"connected": connected,
}
)
return results
@router.get("/connectors/{connector_id}")
def get_connector(connector_id: str) -> Dict[str, Any]:
"""Get details for a specific connector."""
if not ConnectorRegistry.contains(connector_id):
raise HTTPException(
status_code=404,
detail=f"Connector '{connector_id}' not found",
)
cls = ConnectorRegistry.get(connector_id)
try:
inst = _get_or_create(connector_id)
connected = inst.is_connected()
except Exception:
connected = False
auth_url = ""
try:
inst = _get_or_create(connector_id)
auth_url = inst.auth_url()
except NotImplementedError:
pass
mcp_tools = []
try:
inst = _get_or_create(connector_id)
mcp_tools = [t.name for t in inst.mcp_tools()]
except Exception:
pass
return {
"connector_id": connector_id,
"display_name": getattr(cls, "display_name", connector_id),
"auth_type": getattr(cls, "auth_type", "unknown"),
"connected": connected,
"auth_url": auth_url,
"mcp_tools": mcp_tools,
}
@router.post("/connectors/{connector_id}/connect")
def connect_source(
connector_id: str, req: ConnectRequest
) -> Dict[str, Any]:
"""Connect a data source."""
if not ConnectorRegistry.contains(connector_id):
raise HTTPException(
status_code=404,
detail=f"Connector '{connector_id}' not found",
)
cls = ConnectorRegistry.get(connector_id)
auth_type = getattr(cls, "auth_type", "")
if auth_type == "filesystem" and req.path:
inst = cls(vault_path=req.path)
_instances[connector_id] = inst
return {"connected": inst.is_connected()}
if req.token:
inst = cls(token=req.token)
_instances[connector_id] = inst
return {"connected": inst.is_connected()}
if req.code:
inst = _get_or_create(connector_id)
inst.handle_callback(req.code)
return {"connected": inst.is_connected()}
if req.email and req.password:
inst = cls(
email_address=req.email,
app_password=req.password,
)
_instances[connector_id] = inst
return {"connected": inst.is_connected()}
raise HTTPException(
status_code=400,
detail="Provide path, token, code, or email+password",
)
@router.post("/connectors/{connector_id}/disconnect")
def disconnect_source(connector_id: str) -> Dict[str, Any]:
"""Disconnect a data source."""
if not ConnectorRegistry.contains(connector_id):
raise HTTPException(
status_code=404,
detail=f"Connector '{connector_id}' not found",
)
inst = _get_or_create(connector_id)
inst.disconnect()
return {"disconnected": True}
@router.get("/connectors/{connector_id}/sync")
def get_sync_status(connector_id: str) -> Dict[str, Any]:
"""Get sync status for a connector."""
if not ConnectorRegistry.contains(connector_id):
raise HTTPException(
status_code=404,
detail=f"Connector '{connector_id}' not found",
)
inst = _get_or_create(connector_id)
status = inst.sync_status()
return {
"state": status.state,
"items_synced": status.items_synced,
"items_total": status.items_total,
"last_sync": (
status.last_sync.isoformat() if status.last_sync else None
),
"error": status.error,
}
return router
```
- [ ] **Step 4: Run tests**
Run: `cd /lambda/nfs/lambda-stanford/jonsf/scratch_v2/OpenJarvis && uv run pytest tests/server/test_connectors_router.py -v`
Expected: All 6 tests PASS.
- [ ] **Step 5: Run linter**
Run: `cd /lambda/nfs/lambda-stanford/jonsf/scratch_v2/OpenJarvis && uv run ruff check src/openjarvis/server/connectors_router.py`
Expected: No errors.
- [ ] **Step 6: Commit**
```bash
git add src/openjarvis/server/connectors_router.py tests/server/test_connectors_router.py
git commit -m "feat: add /v1/connectors API router for connector management"
```
---
## Post-Plan Notes
**What this plan produces:**
- `GET /v1/connectors` — list all available connectors with connection status
- `GET /v1/connectors/{id}` — connector detail with auth_url and MCP tools
- `POST /v1/connectors/{id}/connect` — connect with path, token, code, or email+password
- `POST /v1/connectors/{id}/disconnect` — disconnect a source
- `GET /v1/connectors/{id}/sync` — sync status (state, items_synced, items_total, error)
**Phase 2B-ii (Frontend wizard)** will consume these endpoints to build:
- Source picker grid (calls `GET /v1/connectors`)
- Per-source OAuth flow (calls `GET /v1/connectors/{id}` for auth_url, then `POST .../connect`)
- Ingest progress dashboard (polls `GET /v1/connectors/{id}/sync`)
- New Tauri commands wrapping these endpoints
**Phase 2B-ii is a TypeScript/React effort** — separate plan needed for the frontend components, Tauri commands, and wizard UI.
docs/superpowers/plans/2026-03-26-deep-research-phase2b-frontend.md
-774
View File
@@ -1,774 +0,0 @@
# Deep Research Phase 2B-ii: Desktop Setup Wizard UI Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Build the "Apple-like magic" setup wizard in the Tauri desktop app — a beautiful multi-step onboarding flow that guides users through connecting their data sources, with product logos, progress bars, and a premium feel.
**Architecture:** Extends the existing `SetupScreen.tsx` in `frontend/src/components/` with new wizard steps after the engine boot completes. Uses the `/v1/connectors/*` API endpoints (Phase 2B-i) for backend communication. New components: SourcePicker (grid of connector cards), SourceConnectWizard (per-source auth flow), IngestDashboard (live sync progress), and ReadyScreen (first query suggestions). Follows existing patterns: Tailwind + shadcn/ui components, Zustand for state, React Router for navigation.
**Tech Stack:** TypeScript, React 19, Tailwind CSS 4.2, shadcn/ui, Zustand 5, Vite 6, Tauri 2
**Spec:** `docs/superpowers/specs/2026-03-25-deep-research-setup-design.md` — Section 4 (Setup Wizard)
**Depends on:** Phase 2B-i (connector API endpoints)
---
## File Structure
```
frontend/src/
├── components/
│ ├── setup/
│ │ ├── SetupWizard.tsx # Main wizard orchestrator (steps 3-6)
│ │ ├── SourcePicker.tsx # Step 3: grid of source cards
│ │ ├── SourceConnectFlow.tsx # Step 4: guided per-source auth
│ │ ├── IngestDashboard.tsx # Step 5: live sync progress
│ │ └── ReadyScreen.tsx # Step 6: first query suggestions
│ ├── SetupScreen.tsx # (modify) Add wizard after boot
├── lib/
│ ├── connectors-api.ts # API client for /v1/connectors/*
│ └── store.ts # (modify) Add connector state slice
├── types/
│ └── connectors.ts # TypeScript interfaces
```
---
### Task 1: TypeScript Types + API Client
**Files:**
- Create: `frontend/src/types/connectors.ts`
- Create: `frontend/src/lib/connectors-api.ts`
- [ ] **Step 1: Create connector types**
Create `frontend/src/types/connectors.ts`:
```typescript
export interface ConnectorInfo {
connector_id: string;
display_name: string;
auth_type: "oauth" | "local" | "bridge" | "filesystem";
connected: boolean;
auth_url?: string;
mcp_tools?: string[];
}
export interface SyncStatus {
state: "idle" | "syncing" | "paused" | "error";
items_synced: number;
items_total: number;
last_sync: string | null;
error: string | null;
}
export interface ConnectRequest {
path?: string;
token?: string;
code?: string;
email?: string;
password?: string;
}
export type WizardStep = "pick" | "connect" | "ingest" | "ready";
// Source card metadata for the picker UI
export interface SourceCard {
connector_id: string;
display_name: string;
auth_type: string;
category: "communication" | "documents" | "pim";
icon: string; // Lucide icon name
color: string; // Tailwind color class
description: string;
}
export const SOURCE_CATALOG: SourceCard[] = [
// Communication
{ connector_id: "gmail", display_name: "Gmail", auth_type: "oauth", category: "communication", icon: "Mail", color: "text-red-400", description: "Email messages and threads" },
{ connector_id: "gmail_imap", display_name: "Gmail (IMAP)", auth_type: "oauth", category: "communication", icon: "Mail", color: "text-red-400", description: "Email via app password" },
{ connector_id: "slack", display_name: "Slack", auth_type: "oauth", category: "communication", icon: "Hash", color: "text-purple-400", description: "Channel messages and threads" },
{ connector_id: "imessage", display_name: "iMessage", auth_type: "local", category: "communication", icon: "MessageSquare", color: "text-green-400", description: "macOS Messages history" },
// Documents
{ connector_id: "gdrive", display_name: "Google Drive", auth_type: "oauth", category: "documents", icon: "FolderOpen", color: "text-blue-400", description: "Docs, Sheets, and files" },
{ connector_id: "notion", display_name: "Notion", auth_type: "oauth", category: "documents", icon: "FileText", color: "text-gray-300", description: "Pages and databases" },
{ connector_id: "obsidian", display_name: "Obsidian", auth_type: "filesystem", category: "documents", icon: "Diamond", color: "text-violet-400", description: "Markdown vault" },
{ connector_id: "granola", display_name: "Granola", auth_type: "oauth", category: "documents", icon: "Mic", color: "text-amber-400", description: "AI meeting notes" },
// PIM
{ connector_id: "gcalendar", display_name: "Calendar", auth_type: "oauth", category: "pim", icon: "Calendar", color: "text-blue-400", description: "Events and meetings" },
{ connector_id: "gcontacts", display_name: "Contacts", auth_type: "oauth", category: "pim", icon: "Users", color: "text-blue-400", description: "People and contact info" },
];
```
- [ ] **Step 2: Create API client**
Create `frontend/src/lib/connectors-api.ts`:
```typescript
import { getBase } from "./api";
import type { ConnectorInfo, ConnectRequest, SyncStatus } from "../types/connectors";
export async function listConnectors(): Promise<ConnectorInfo[]> {
const resp = await fetch(`${getBase()}/v1/connectors`);
if (!resp.ok) throw new Error(`Failed to list connectors: ${resp.status}`);
return resp.json();
}
export async function getConnector(id: string): Promise<ConnectorInfo> {
const resp = await fetch(`${getBase()}/v1/connectors/${id}`);
if (!resp.ok) throw new Error(`Connector not found: ${id}`);
return resp.json();
}
export async function connectSource(id: string, req: ConnectRequest): Promise<{ connected: boolean }> {
const resp = await fetch(`${getBase()}/v1/connectors/${id}/connect`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(req),
});
if (!resp.ok) throw new Error(`Failed to connect: ${resp.status}`);
return resp.json();
}
export async function disconnectSource(id: string): Promise<void> {
await fetch(`${getBase()}/v1/connectors/${id}/disconnect`, { method: "POST" });
}
export async function getSyncStatus(id: string): Promise<SyncStatus> {
const resp = await fetch(`${getBase()}/v1/connectors/${id}/sync`);
if (!resp.ok) throw new Error(`Failed to get sync status: ${id}`);
return resp.json();
}
```
- [ ] **Step 3: Commit**
```bash
cd frontend && git add src/types/connectors.ts src/lib/connectors-api.ts
git commit -m "feat: add connector TypeScript types and API client"
```
---
### Task 2: SourcePicker Component (Step 3)
**Files:**
- Create: `frontend/src/components/setup/SourcePicker.tsx`
- [ ] **Step 1: Implement SourcePicker**
Create `frontend/src/components/setup/SourcePicker.tsx`:
```tsx
import { useState } from "react";
import {
Mail, Hash, MessageSquare, FolderOpen, FileText,
Diamond, Mic, Calendar, Users, Check,
} from "lucide-react";
import { SOURCE_CATALOG, type SourceCard } from "../../types/connectors";
const ICON_MAP: Record<string, React.ElementType> = {
Mail, Hash, MessageSquare, FolderOpen, FileText,
Diamond, Mic, Calendar, Users,
};
const CATEGORIES = [
{ key: "communication", label: "Communication" },
{ key: "documents", label: "Documents" },
{ key: "pim", label: "Personal Info" },
] as const;
interface Props {
onContinue: (selectedIds: string[]) => void;
}
export function SourcePicker({ onContinue }: Props) {
const [selected, setSelected] = useState<Set<string>>(new Set());
const toggle = (id: string) => {
setSelected((prev) => {
const next = new Set(prev);
if (next.has(id)) next.delete(id);
else next.add(id);
return next;
});
};
return (
<div className="flex flex-col items-center gap-8 px-8 py-12 max-w-4xl mx-auto">
<div className="text-center">
<h2 className="text-2xl font-semibold" style={{ color: "var(--color-text)" }}>
Connect Your Data
</h2>
<p className="mt-2 text-sm opacity-60">
Choose the sources you want to search across. You can always add more later.
</p>
</div>
{CATEGORIES.map(({ key, label }) => (
<div key={key} className="w-full">
<h3 className="text-xs uppercase tracking-wider opacity-40 mb-3 px-1">
{label}
</h3>
<div className="grid grid-cols-2 sm:grid-cols-3 md:grid-cols-4 gap-3">
{SOURCE_CATALOG.filter((s) => s.category === key).map((source) => {
const Icon = ICON_MAP[source.icon] ?? FileText;
const isSelected = selected.has(source.connector_id);
return (
<button
key={source.connector_id}
onClick={() => toggle(source.connector_id)}
className={`
relative flex flex-col items-center gap-2 p-4 rounded-xl
border transition-all duration-150 cursor-pointer
${isSelected
? "border-blue-500/60 bg-blue-500/10"
: "border-white/10 bg-white/5 hover:border-white/20 hover:bg-white/8"
}
`}
>
{isSelected && (
<div className="absolute top-2 right-2">
<Check size={14} className="text-blue-400" />
</div>
)}
<Icon size={24} className={source.color} />
<span className="text-sm font-medium">{source.display_name}</span>
<span className="text-xs opacity-40 text-center">{source.description}</span>
</button>
);
})}
</div>
</div>
))}
<button
onClick={() => onContinue(Array.from(selected))}
disabled={selected.size === 0}
className={`
mt-4 px-8 py-3 rounded-xl font-medium transition-all
${selected.size > 0
? "bg-blue-500 text-white hover:bg-blue-400"
: "bg-white/10 text-white/30 cursor-not-allowed"
}
`}
>
Connect {selected.size > 0 ? `${selected.size} source${selected.size > 1 ? "s" : ""}` : "sources"}
</button>
</div>
);
}
```
- [ ] **Step 2: Commit**
```bash
cd frontend && git add src/components/setup/SourcePicker.tsx
git commit -m "feat: add SourcePicker component with category grid and toggle selection"
```
---
### Task 3: SourceConnectFlow Component (Step 4)
**Files:**
- Create: `frontend/src/components/setup/SourceConnectFlow.tsx`
- [ ] **Step 1: Implement SourceConnectFlow**
Create `frontend/src/components/setup/SourceConnectFlow.tsx`:
```tsx
import { useState } from "react";
import { Check, ExternalLink, Loader2, X, FolderOpen } from "lucide-react";
import { SOURCE_CATALOG } from "../../types/connectors";
import { connectSource, getConnector } from "../../lib/connectors-api";
interface Props {
selectedIds: string[];
onComplete: () => void;
}
type SourceState = "pending" | "connecting" | "connected" | "skipped" | "error";
export function SourceConnectFlow({ selectedIds, onComplete }: Props) {
const [currentIndex, setCurrentIndex] = useState(0);
const [states, setStates] = useState<Record<string, SourceState>>(
Object.fromEntries(selectedIds.map((id) => [id, "pending"]))
);
const [error, setError] = useState("");
const [inputValue, setInputValue] = useState("");
const currentId = selectedIds[currentIndex];
const source = SOURCE_CATALOG.find((s) => s.connector_id === currentId);
const isLast = currentIndex >= selectedIds.length - 1;
const allDone = currentIndex >= selectedIds.length;
const advance = () => {
if (isLast) {
onComplete();
} else {
setCurrentIndex((i) => i + 1);
setInputValue("");
setError("");
}
};
const skip = () => {
setStates((s) => ({ ...s, [currentId]: "skipped" }));
advance();
};
const handleConnect = async () => {
if (!currentId || !source) return;
setStates((s) => ({ ...s, [currentId]: "connecting" }));
setError("");
try {
if (source.auth_type === "filesystem") {
const result = await connectSource(currentId, { path: inputValue });
if (result.connected) {
setStates((s) => ({ ...s, [currentId]: "connected" }));
setTimeout(advance, 800);
} else {
throw new Error("Could not access that path");
}
} else if (source.auth_type === "oauth") {
// Get auth URL and open in browser
const info = await getConnector(currentId);
if (info.auth_url) {
window.open(info.auth_url, "_blank");
}
// For token/code entry
if (inputValue) {
const result = await connectSource(currentId, { token: inputValue });
if (result.connected) {
setStates((s) => ({ ...s, [currentId]: "connected" }));
setTimeout(advance, 800);
}
}
} else if (source.auth_type === "local") {
// Local access (iMessage, Apple Notes) — just check
const result = await connectSource(currentId, {});
if (result.connected) {
setStates((s) => ({ ...s, [currentId]: "connected" }));
setTimeout(advance, 800);
} else {
throw new Error("Permission not granted. Check Full Disk Access in System Preferences.");
}
}
} catch (err: any) {
setError(err.message || "Connection failed");
setStates((s) => ({ ...s, [currentId]: "error" }));
}
};
if (allDone) {
onComplete();
return null;
}
return (
<div className="flex gap-8 px-8 py-12 max-w-4xl mx-auto">
{/* Sidebar checklist */}
<div className="w-48 flex-shrink-0">
<h3 className="text-xs uppercase tracking-wider opacity-40 mb-4">Progress</h3>
{selectedIds.map((id, i) => {
const s = SOURCE_CATALOG.find((x) => x.connector_id === id);
const state = states[id];
return (
<div key={id} className={`flex items-center gap-2 py-2 text-sm ${i === currentIndex ? "opacity-100" : "opacity-40"}`}>
{state === "connected" && <Check size={14} className="text-green-400" />}
{state === "skipped" && <X size={14} className="text-gray-500" />}
{state === "connecting" && <Loader2 size={14} className="animate-spin text-blue-400" />}
{(state === "pending" || state === "error") && <div className="w-3.5 h-3.5 rounded-full border border-white/20" />}
<span>{s?.display_name ?? id}</span>
</div>
);
})}
</div>
{/* Main content */}
<div className="flex-1">
<h2 className="text-xl font-semibold mb-2">
Connect {source?.display_name}
</h2>
{source?.auth_type === "filesystem" && (
<div className="mt-6">
<p className="text-sm opacity-60 mb-4">Enter the path to your vault or folder:</p>
<div className="flex gap-2">
<input
type="text"
value={inputValue}
onChange={(e) => setInputValue(e.target.value)}
placeholder="/path/to/vault"
className="flex-1 px-4 py-2 rounded-lg bg-white/5 border border-white/10 text-sm"
/>
<button onClick={handleConnect} className="px-4 py-2 bg-blue-500 text-white rounded-lg text-sm hover:bg-blue-400">
Connect
</button>
</div>
</div>
)}
{source?.auth_type === "oauth" && (
<div className="mt-6">
<p className="text-sm opacity-60 mb-4">
We'll open {source.display_name} in your browser for authorization. After authorizing, paste the token or code below.
</p>
<button
onClick={handleConnect}
className="flex items-center gap-2 px-4 py-2 bg-blue-500 text-white rounded-lg text-sm hover:bg-blue-400 mb-4"
>
<ExternalLink size={14} />
Open {source.display_name}
</button>
<input
type="text"
value={inputValue}
onChange={(e) => setInputValue(e.target.value)}
placeholder="Paste token or code here"
className="w-full px-4 py-2 rounded-lg bg-white/5 border border-white/10 text-sm"
onKeyDown={(e) => e.key === "Enter" && handleConnect()}
/>
</div>
)}
{source?.auth_type === "local" && (
<div className="mt-6">
<p className="text-sm opacity-60 mb-4">
This requires Full Disk Access on macOS. Grant access in System Preferences → Privacy & Security → Full Disk Access.
</p>
<button onClick={handleConnect} className="px-4 py-2 bg-blue-500 text-white rounded-lg text-sm hover:bg-blue-400">
Check Access
</button>
</div>
)}
{error && (
<p className="mt-4 text-sm text-red-400">{error}</p>
)}
<button onClick={skip} className="mt-6 text-sm opacity-40 hover:opacity-60 transition-opacity">
Skip for now →
</button>
</div>
</div>
);
}
```
- [ ] **Step 2: Commit**
```bash
cd frontend && git add src/components/setup/SourceConnectFlow.tsx
git commit -m "feat: add SourceConnectFlow with per-source guided auth wizard"
```
---
### Task 4: IngestDashboard + ReadyScreen (Steps 5-6)
**Files:**
- Create: `frontend/src/components/setup/IngestDashboard.tsx`
- Create: `frontend/src/components/setup/ReadyScreen.tsx`
- [ ] **Step 1: Implement IngestDashboard**
Create `frontend/src/components/setup/IngestDashboard.tsx`:
```tsx
import { useEffect, useState } from "react";
import { Loader2, Check } from "lucide-react";
import { getSyncStatus } from "../../lib/connectors-api";
import { SOURCE_CATALOG, type SyncStatus } from "../../types/connectors";
interface Props {
connectedIds: string[];
onReady: () => void;
}
export function IngestDashboard({ connectedIds, onReady }: Props) {
const [statuses, setStatuses] = useState<Record<string, SyncStatus>>({});
useEffect(() => {
if (connectedIds.length === 0) {
onReady();
return;
}
const poll = setInterval(async () => {
const updates: Record<string, SyncStatus> = {};
for (const id of connectedIds) {
try {
updates[id] = await getSyncStatus(id);
} catch {
updates[id] = { state: "error", items_synced: 0, items_total: 0, last_sync: null, error: "Failed to fetch status" };
}
}
setStatuses(updates);
// Check if all done
const allIdle = Object.values(updates).every((s) => s.state === "idle" && s.items_synced > 0);
if (allIdle) {
clearInterval(poll);
}
}, 2000);
return () => clearInterval(poll);
}, [connectedIds]);
return (
<div className="flex flex-col items-center gap-8 px-8 py-12 max-w-3xl mx-auto">
<div className="text-center">
<h2 className="text-2xl font-semibold" style={{ color: "var(--color-text)" }}>
Indexing Your Data
</h2>
<p className="mt-2 text-sm opacity-60">
Your data is being synced and indexed. You can start asking questions now.
</p>
</div>
<div className="w-full space-y-4">
{connectedIds.map((id) => {
const source = SOURCE_CATALOG.find((s) => s.connector_id === id);
const status = statuses[id];
const synced = status?.items_synced ?? 0;
const total = status?.items_total ?? 0;
const pct = total > 0 ? Math.round((synced / total) * 100) : 0;
const isDone = status?.state === "idle" && synced > 0;
return (
<div key={id} className="flex items-center gap-4 p-4 rounded-xl bg-white/5 border border-white/10">
<div className="w-8">
{isDone ? (
<Check size={18} className="text-green-400" />
) : (
<Loader2 size={18} className="animate-spin text-blue-400" />
)}
</div>
<div className="flex-1">
<div className="flex justify-between text-sm mb-1">
<span className="font-medium">{source?.display_name ?? id}</span>
<span className="opacity-40">
{isDone ? `${synced.toLocaleString()} items` : total > 0 ? `${synced.toLocaleString()} / ${total.toLocaleString()}` : "Starting..."}
</span>
</div>
<div className="h-1.5 rounded-full bg-white/10 overflow-hidden">
<div
className="h-full rounded-full bg-blue-500 transition-all duration-500"
style={{ width: `${isDone ? 100 : pct}%` }}
/>
</div>
</div>
</div>
);
})}
</div>
<button
onClick={onReady}
className="mt-4 px-8 py-3 bg-blue-500 text-white rounded-xl font-medium hover:bg-blue-400 transition-all"
>
Start Researching →
</button>
</div>
);
}
```
- [ ] **Step 2: Implement ReadyScreen**
Create `frontend/src/components/setup/ReadyScreen.tsx`:
```tsx
import { Sparkles } from "lucide-react";
interface Props {
connectedSources: string[];
onStart: (query?: string) => void;
}
const SUGGESTIONS = [
"What were the key decisions from last week's team threads?",
"Find the proposal doc shared about the roadmap",
"Summarize my unread emails from today",
"What meetings do I have this week?",
"What topics came up in recent meetings?",
];
export function ReadyScreen({ connectedSources, onStart }: Props) {
return (
<div className="flex flex-col items-center gap-8 px-8 py-16 max-w-2xl mx-auto text-center">
<div className="w-16 h-16 rounded-2xl bg-green-500/10 flex items-center justify-center">
<Sparkles size={32} className="text-green-400" />
</div>
<div>
<h2 className="text-2xl font-semibold" style={{ color: "var(--color-text)" }}>
You're All Set!
</h2>
<p className="mt-2 text-sm opacity-60">
{connectedSources.length} source{connectedSources.length !== 1 ? "s" : ""} connected and indexed. Ask anything across your data.
</p>
</div>
<div className="w-full space-y-2">
<p className="text-xs uppercase tracking-wider opacity-40 mb-3">Try asking...</p>
{SUGGESTIONS.slice(0, 3).map((q) => (
<button
key={q}
onClick={() => onStart(q)}
className="w-full text-left px-4 py-3 rounded-xl bg-white/5 border border-white/10 text-sm hover:border-blue-500/40 hover:bg-blue-500/5 transition-all"
>
{q}
</button>
))}
</div>
<button
onClick={() => onStart()}
className="px-8 py-3 bg-blue-500 text-white rounded-xl font-medium hover:bg-blue-400 transition-all"
>
Open Chat
</button>
</div>
);
}
```
- [ ] **Step 3: Commit**
```bash
cd frontend && git add src/components/setup/IngestDashboard.tsx src/components/setup/ReadyScreen.tsx
git commit -m "feat: add IngestDashboard with progress bars and ReadyScreen with suggestions"
```
---
### Task 5: SetupWizard Orchestrator + Integration
**Files:**
- Create: `frontend/src/components/setup/SetupWizard.tsx`
- Modify: `frontend/src/components/SetupScreen.tsx`
- [ ] **Step 1: Create SetupWizard orchestrator**
Create `frontend/src/components/setup/SetupWizard.tsx`:
```tsx
import { useState } from "react";
import type { WizardStep } from "../../types/connectors";
import { SourcePicker } from "./SourcePicker";
import { SourceConnectFlow } from "./SourceConnectFlow";
import { IngestDashboard } from "./IngestDashboard";
import { ReadyScreen } from "./ReadyScreen";
interface Props {
onComplete: (firstQuery?: string) => void;
}
export function SetupWizard({ onComplete }: Props) {
const [step, setStep] = useState<WizardStep>("pick");
const [selectedIds, setSelectedIds] = useState<string[]>([]);
const [connectedIds, setConnectedIds] = useState<string[]>([]);
switch (step) {
case "pick":
return (
<SourcePicker
onContinue={(ids) => {
setSelectedIds(ids);
if (ids.length > 0) {
setStep("connect");
} else {
onComplete();
}
}}
/>
);
case "connect":
return (
<SourceConnectFlow
selectedIds={selectedIds}
onComplete={() => {
// For now, assume all selected are connected
setConnectedIds(selectedIds);
setStep("ingest");
}}
/>
);
case "ingest":
return (
<IngestDashboard
connectedIds={connectedIds}
onReady={() => setStep("ready")}
/>
);
case "ready":
return (
<ReadyScreen
connectedSources={connectedIds}
onStart={(query) => onComplete(query)}
/>
);
}
}
```
- [ ] **Step 2: Integrate into SetupScreen**
Modify `frontend/src/components/SetupScreen.tsx` to show the wizard after the boot sequence completes. Find the section where `status.phase === 'ready'` triggers `onReady()`. Instead of calling `onReady()` immediately, show the `SetupWizard`:
Add to the imports:
```tsx
import { SetupWizard } from "./setup/SetupWizard";
```
Add state to track whether wizard is shown:
```tsx
const [showWizard, setShowWizard] = useState(false);
```
When boot completes (phase === 'ready'), instead of calling `onReady()`, set `showWizard = true`. Then render:
```tsx
if (showWizard) {
return <SetupWizard onComplete={(query) => {
// Optionally pass first query to chat
onReady();
}} />;
}
```
- [ ] **Step 3: Commit**
```bash
cd frontend && git add src/components/setup/SetupWizard.tsx src/components/SetupScreen.tsx
git commit -m "feat: integrate SetupWizard into boot flow after engine setup"
```
---
## Post-Plan Notes
**What this plan produces:**
- TypeScript types and API client for connector management
- SourcePicker: beautiful grid of 10 source cards with category grouping and toggle selection
- SourceConnectFlow: guided per-source auth wizard with sidebar progress checklist
- IngestDashboard: live progress bars polling sync status
- ReadyScreen: celebration screen with suggested first queries
- SetupWizard: orchestrates all 4 steps
- Integration into existing SetupScreen boot flow
**What this does NOT include (separate effort):**
- Actual OAuth redirect handling in Tauri (needs Tauri deep link plugin)
- File picker dialog for Obsidian (needs Tauri dialog plugin)
- QR code display for WhatsApp
- Product logo images (would need asset files)
- Tauri command wrappers for the API endpoints
- Automated sync triggering after connect
docs/superpowers/plans/2026-03-26-deep-research-phase4-channel-plugins.md
-816
View File
@@ -1,816 +0,0 @@
# Deep Research Phase 4: Channel Plugins Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Build a ChannelAgent that connects existing messaging channels (Slack, WhatsApp, iMessage) to the DeepResearchAgent, with automatic query classification (quick inline answers vs. escalation to desktop for complex research).
**Architecture:** A `ChannelAgent` class registers as a `ChannelHandler` callback on any `BaseChannel`. When a message arrives, it classifies the query (quick vs. deep), runs the DeepResearchAgent with the appropriate turn limit, and sends the response back via `channel.send()`. Quick answers go inline; complex queries get a brief preview + escalation link. A worker thread pool prevents blocking the channel's event loop.
**Tech Stack:** Python 3.10+, threading (worker pool), existing channel transports (Slack Socket Mode, WhatsApp Baileys, BlueBubbles), pytest
**Spec:** `docs/superpowers/specs/2026-03-25-deep-research-setup-design.md` — Section 9 (Channel Plugins)
**Depends on:** Phase 1 (connectors, KnowledgeStore), Phase 3 (DeepResearchAgent, TwoStageRetriever)
---
## File Structure
```
src/openjarvis/agents/
├── channel_agent.py # ChannelAgent: bridge between channels and DeepResearchAgent
tests/agents/
├── test_channel_agent.py # ChannelAgent tests
```
---
### Task 1: Query Classifier
**Files:**
- Create: `src/openjarvis/agents/channel_agent.py` (first half — classifier only)
- Create: `tests/agents/test_channel_agent.py`
The classifier determines whether a message needs a quick single-hop answer or full multi-hop research. This is an internal implementation detail — the user never sees it.
- [ ] **Step 1: Write failing tests for classifier**
Create `tests/agents/test_channel_agent.py`:
```python
"""Tests for ChannelAgent — bridge between channels and DeepResearchAgent."""
from __future__ import annotations
from openjarvis.agents.channel_agent import classify_query
def test_quick_when_query() -> None:
assert classify_query("When is my next meeting?") == "quick"
def test_quick_where_query() -> None:
assert classify_query("Where is the design doc?") == "quick"
def test_quick_find_query() -> None:
assert classify_query("Find the budget spreadsheet") == "quick"
def test_quick_short_query() -> None:
assert classify_query("Who sent the last email?") == "quick"
def test_deep_summarize() -> None:
assert classify_query("Summarize all discussions about pricing") == "deep"
def test_deep_research() -> None:
assert classify_query("Research the context behind the K8s decision") == "deep"
def test_deep_context() -> None:
assert classify_query("What was the context around the migration?") == "deep"
def test_deep_time_range() -> None:
assert classify_query("What happened last month with the project?") == "deep"
def test_deep_long_query() -> None:
long = "Tell me about all the discussions between Sarah and Mike about the infrastructure migration including cost analysis and timeline"
assert classify_query(long) == "deep"
def test_quick_single_entity() -> None:
assert classify_query("What's Sarah's email?") == "quick"
def test_deep_multi_entity() -> None:
assert classify_query("Compare what Sarah and Mike said about the budget") == "deep"
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `cd /lambda/nfs/lambda-stanford/jonsf/scratch_v2/OpenJarvis && uv run pytest tests/agents/test_channel_agent.py -v`
Expected: FAIL — `ModuleNotFoundError`
- [ ] **Step 3: Implement classify_query**
Create `src/openjarvis/agents/channel_agent.py`:
```python
"""ChannelAgent — bridges messaging channels to the DeepResearchAgent.
Handles:
- Automatic query classification (quick vs. deep)
- Quick answers inline in the chat channel
- Complex queries escalated with a preview + desktop link
- Worker thread pool so channel event loops aren't blocked
"""
from __future__ import annotations
import logging
import re
import threading
import uuid
from concurrent.futures import ThreadPoolExecutor
from typing import Any, Callable, Dict, List, Optional
from openjarvis.agents._stubs import AgentContext, AgentResult
from openjarvis.channels._stubs import BaseChannel, ChannelMessage
from openjarvis.core.events import EventType, get_event_bus
logger = logging.getLogger(__name__)
# ---------------------------------------------------------------
# Query classification
# ---------------------------------------------------------------
_DEEP_KEYWORDS = re.compile(
r"\b(summarize|summary|research|context|compare|analyze|analysis"
r"|overview|all discussions|comprehensive|timeline|history"
r"|what happened|tell me about all)\b",
re.IGNORECASE,
)
_TIME_RANGE_PATTERNS = re.compile(
r"\b(last (week|month|quarter|year)|past \d+|since (january|february"
r"|march|april|may|june|july|august|september|october|november"
r"|december))\b",
re.IGNORECASE,
)
_QUICK_STARTERS = re.compile(
r"^(when|where|find|who sent|what'?s|what is)\b", re.IGNORECASE
)
_MAX_QUICK_WORDS = 20
def classify_query(text: str) -> str:
"""Classify a user query as 'quick' or 'deep'.
Quick: single-entity lookups, short factual questions.
Deep: multi-source synthesis, summarization, time-range analysis.
This is an internal heuristic — the user never specifies the mode.
"""
text = text.strip()
words = text.split()
# Deep signals take priority
if _DEEP_KEYWORDS.search(text):
return "deep"
if _TIME_RANGE_PATTERNS.search(text):
return "deep"
if len(words) > _MAX_QUICK_WORDS:
return "deep"
# Quick signals
if _QUICK_STARTERS.match(text):
return "quick"
if len(words) <= _MAX_QUICK_WORDS:
return "quick"
return "quick"
```
- [ ] **Step 4: Run tests**
Run: `cd /lambda/nfs/lambda-stanford/jonsf/scratch_v2/OpenJarvis && uv run pytest tests/agents/test_channel_agent.py -v`
Expected: All 11 tests PASS.
- [ ] **Step 5: Commit**
```bash
git add src/openjarvis/agents/channel_agent.py tests/agents/test_channel_agent.py
git commit -m "feat: add query classifier for channel agent (quick vs deep)"
```
---
### Task 2: ChannelAgent Core
**Files:**
- Modify: `src/openjarvis/agents/channel_agent.py`
- Modify: `tests/agents/test_channel_agent.py`
- [ ] **Step 1: Write failing tests for ChannelAgent**
Add to `tests/agents/test_channel_agent.py`:
```python
import json
import threading
import time
from pathlib import Path
from unittest.mock import MagicMock, patch
import pytest
from openjarvis.agents.channel_agent import ChannelAgent, classify_query
from openjarvis.agents.deep_research import DeepResearchAgent
from openjarvis.agents._stubs import AgentResult
from openjarvis.channels._stubs import BaseChannel, ChannelMessage, ChannelStatus
from openjarvis.connectors.store import KnowledgeStore
from openjarvis.connectors.retriever import TwoStageRetriever
from openjarvis.core.types import ToolResult
from openjarvis.tools.knowledge_search import KnowledgeSearchTool
class FakeChannel(BaseChannel):
"""Minimal channel for testing."""
channel_id = "fake"
def __init__(self) -> None:
self._handlers: list = []
self._sent: list = []
self._connected = True
def connect(self) -> None:
self._connected = True
def disconnect(self) -> None:
self._connected = False
def send(
self,
channel: str,
content: str,
*,
conversation_id: str = "",
metadata: dict | None = None,
) -> bool:
self._sent.append(
{
"channel": channel,
"content": content,
"conversation_id": conversation_id,
}
)
return True
def status(self) -> ChannelStatus:
return ChannelStatus.CONNECTED if self._connected else ChannelStatus.DISCONNECTED
def list_channels(self) -> list[str]:
return ["test"]
def on_message(self, handler) -> None:
self._handlers.append(handler)
def simulate_message(self, text: str, sender: str = "user1") -> None:
msg = ChannelMessage(
channel="fake",
sender=sender,
content=text,
conversation_id="conv1",
)
for h in self._handlers:
h(msg)
@pytest.fixture
def mock_agent() -> MagicMock:
agent = MagicMock(spec=DeepResearchAgent)
agent.agent_id = "deep_research"
agent.run.return_value = AgentResult(
content="The next meeting is tomorrow at 2pm.",
tool_results=[],
turns=1,
)
return agent
@pytest.fixture
def channel() -> FakeChannel:
return FakeChannel()
@pytest.fixture
def channel_agent(
channel: FakeChannel, mock_agent: MagicMock
) -> ChannelAgent:
return ChannelAgent(
channel=channel,
agent=mock_agent,
)
def test_channel_agent_registers_handler(
channel: FakeChannel, channel_agent: ChannelAgent
) -> None:
assert len(channel._handlers) == 1
def test_quick_query_sends_response_inline(
channel: FakeChannel,
channel_agent: ChannelAgent,
mock_agent: MagicMock,
) -> None:
channel.simulate_message("When is my next meeting?")
# Give worker thread time to process
time.sleep(0.5)
assert len(channel._sent) >= 1
assert "meeting" in channel._sent[0]["content"].lower()
mock_agent.run.assert_called_once()
def test_deep_query_sends_preview_and_link(
channel: FakeChannel,
channel_agent: ChannelAgent,
mock_agent: MagicMock,
) -> None:
mock_agent.run.return_value = AgentResult(
content="# Kubernetes Migration Report\n\nDetailed analysis...",
tool_results=[
ToolResult(
tool_name="knowledge_search",
content="results",
success=True,
metadata={"num_results": 15},
),
],
turns=3,
)
channel.simulate_message(
"Summarize all discussions about the Kubernetes migration"
)
time.sleep(0.5)
assert len(channel._sent) >= 1
# Deep queries should include an escalation link
sent_content = channel._sent[-1]["content"]
assert "openjarvis://" in sent_content or len(sent_content) > 50
def test_agent_error_sends_error_message(
channel: FakeChannel,
channel_agent: ChannelAgent,
mock_agent: MagicMock,
) -> None:
mock_agent.run.side_effect = RuntimeError("LLM unavailable")
channel.simulate_message("When is my meeting?")
time.sleep(0.5)
assert len(channel._sent) >= 1
assert "error" in channel._sent[0]["content"].lower() or "sorry" in channel._sent[0]["content"].lower()
def test_quick_query_uses_max_turns_1(
channel: FakeChannel,
channel_agent: ChannelAgent,
mock_agent: MagicMock,
) -> None:
channel.simulate_message("Find the budget doc")
time.sleep(0.5)
call_kwargs = mock_agent.run.call_args
# Quick queries should be called (the agent handles turn limits internally)
mock_agent.run.assert_called_once()
def test_channel_agent_does_not_block_handler(
channel: FakeChannel,
channel_agent: ChannelAgent,
mock_agent: MagicMock,
) -> None:
"""Handler should return immediately (work dispatched to thread pool)."""
mock_agent.run.side_effect = lambda *a, **kw: (
time.sleep(2),
AgentResult(content="done", turns=1),
)[-1]
t0 = time.time()
channel.simulate_message("When is my meeting?")
elapsed = time.time() - t0
# Handler should return in < 0.1s even if agent takes 2s
assert elapsed < 0.5
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `cd /lambda/nfs/lambda-stanford/jonsf/scratch_v2/OpenJarvis && uv run pytest tests/agents/test_channel_agent.py::test_channel_agent_registers_handler -v`
Expected: FAIL — `ImportError: cannot import name 'ChannelAgent'`
- [ ] **Step 3: Implement ChannelAgent**
Add to `src/openjarvis/agents/channel_agent.py` (after the classifier):
```python
# ---------------------------------------------------------------
# ChannelAgent
# ---------------------------------------------------------------
_ESCALATION_TEMPLATE = """{preview}
---
Full report ready — open in OpenJarvis:
openjarvis://research/{session_id}"""
_ERROR_TEMPLATE = (
"Sorry, I ran into an error processing your request. "
"Please try again or open OpenJarvis for the full experience."
)
_MAX_INLINE_LENGTH = 500
class ChannelAgent:
"""Bridges a messaging channel to the DeepResearchAgent.
Registers as a handler on the given channel. When a message
arrives, classifies it (quick vs deep), runs the agent in a
worker thread, and sends the response back via the channel.
Quick queries get the full response inline.
Deep queries get a brief preview + escalation link.
Parameters
----------
channel:
The messaging channel to listen on.
agent:
The DeepResearchAgent (or any BaseAgent) to handle queries.
max_workers:
Thread pool size for concurrent query processing.
"""
def __init__(
self,
channel: BaseChannel,
agent: Any,
*,
max_workers: int = 2,
) -> None:
self._channel = channel
self._agent = agent
self._pool = ThreadPoolExecutor(
max_workers=max_workers,
thread_name_prefix="channel_agent",
)
# Register handler
channel.on_message(self._handle_message)
def _handle_message(self, msg: ChannelMessage) -> Optional[str]:
"""Handler callback — dispatches work to thread pool."""
self._pool.submit(self._process_message, msg)
return None # Don't block the channel's event loop
def _process_message(self, msg: ChannelMessage) -> None:
"""Process a message in a worker thread."""
query = msg.content.strip()
if not query:
return
classification = classify_query(query)
session_id = uuid.uuid4().hex[:12]
bus = get_event_bus()
bus.publish(
EventType.AGENT_TURN_START,
{
"agent": "channel_agent",
"channel": msg.channel,
"sender": msg.sender,
"classification": classification,
},
)
try:
result = self._agent.run(query)
except Exception as exc:
logger.error(
"ChannelAgent error for %s: %s",
msg.sender,
exc,
)
self._channel.send(
msg.sender,
_ERROR_TEMPLATE,
conversation_id=msg.conversation_id,
)
return
content = result.content.strip()
if classification == "deep" or len(content) > _MAX_INLINE_LENGTH:
# Truncate for preview
preview = content[:300]
if len(content) > 300:
preview = preview.rsplit(" ", 1)[0] + "..."
response = _ESCALATION_TEMPLATE.format(
preview=preview, session_id=session_id
)
else:
response = content
self._channel.send(
msg.sender,
response,
conversation_id=msg.conversation_id,
)
bus.publish(
EventType.AGENT_TURN_END,
{
"agent": "channel_agent",
"channel": msg.channel,
"turns": result.turns,
"classification": classification,
"session_id": session_id,
},
)
def shutdown(self) -> None:
"""Shut down the worker thread pool."""
self._pool.shutdown(wait=False)
```
- [ ] **Step 4: Run tests**
Run: `cd /lambda/nfs/lambda-stanford/jonsf/scratch_v2/OpenJarvis && uv run pytest tests/agents/test_channel_agent.py -v`
Expected: All 17 tests PASS (11 classifier + 6 ChannelAgent).
- [ ] **Step 5: Commit**
```bash
git add src/openjarvis/agents/channel_agent.py tests/agents/test_channel_agent.py
git commit -m "feat: add ChannelAgent with query classification and escalation"
```
---
### Task 3: Integration Test — ChannelAgent + DeepResearchAgent + Real Data
**Files:**
- Create: `tests/agents/test_channel_agent_integration.py`
- [ ] **Step 1: Write integration test**
Create `tests/agents/test_channel_agent_integration.py`:
```python
"""Integration test — ChannelAgent with real KnowledgeStore and mock engine."""
from __future__ import annotations
import json
import time
from pathlib import Path
from unittest.mock import MagicMock
import pytest
from openjarvis.agents.channel_agent import ChannelAgent
from openjarvis.agents.deep_research import DeepResearchAgent
from openjarvis.channels._stubs import BaseChannel, ChannelMessage, ChannelStatus
from openjarvis.connectors._stubs import Document
from openjarvis.connectors.pipeline import IngestionPipeline
from openjarvis.connectors.retriever import TwoStageRetriever
from openjarvis.connectors.store import KnowledgeStore
from openjarvis.tools.knowledge_search import KnowledgeSearchTool
class FakeChannel(BaseChannel):
channel_id = "fake"
def __init__(self) -> None:
self._handlers: list = []
self._sent: list = []
def connect(self) -> None:
pass
def disconnect(self) -> None:
pass
def send(
self,
channel: str,
content: str,
*,
conversation_id: str = "",
metadata: dict | None = None,
) -> bool:
self._sent.append({"content": content, "conv": conversation_id})
return True
def status(self) -> ChannelStatus:
return ChannelStatus.CONNECTED
def list_channels(self) -> list[str]:
return ["test"]
def on_message(self, handler) -> None:
self._handlers.append(handler)
def simulate(self, text: str) -> None:
msg = ChannelMessage(
channel="fake",
sender="user",
content=text,
conversation_id="conv1",
)
for h in self._handlers:
h(msg)
@pytest.fixture
def populated_store(tmp_path: Path) -> KnowledgeStore:
store = KnowledgeStore(
db_path=str(tmp_path / "channel_int.db")
)
pipeline = IngestionPipeline(store=store, max_tokens=256)
pipeline.ingest(
[
Document(
doc_id="gcalendar:evt1",
source="gcalendar",
doc_type="event",
content="Sprint Planning\nWhen: Tomorrow 2pm\nAttendees: Sarah, Mike",
title="Sprint Planning",
),
Document(
doc_id="slack:msg1",
source="slack",
doc_type="message",
content="We need to finalize the API redesign before Friday",
title="#engineering",
author="sarah",
),
Document(
doc_id="gmail:msg1",
source="gmail",
doc_type="email",
content="Budget report attached. Q3 spending up 15%.",
title="Re: Q3 Budget",
author="mike",
),
]
)
return store
def test_quick_query_through_full_stack(
populated_store: KnowledgeStore,
) -> None:
"""Quick query → agent searches → inline response."""
mock_engine = MagicMock()
mock_engine.engine_id = "mock"
mock_engine.generate.return_value = {
"content": "Your next meeting is Sprint Planning tomorrow at 2pm with Sarah and Mike.",
"usage": {"prompt_tokens": 100, "completion_tokens": 50, "total_tokens": 150},
"model": "test",
"finish_reason": "stop",
}
retriever = TwoStageRetriever(store=populated_store)
ks_tool = KnowledgeSearchTool(
store=populated_store, retriever=retriever
)
agent = DeepResearchAgent(
engine=mock_engine,
model="test",
tools=[ks_tool],
max_turns=2,
)
channel = FakeChannel()
ca = ChannelAgent(channel=channel, agent=agent)
channel.simulate("When is my next meeting?")
time.sleep(1.0)
assert len(channel._sent) >= 1
response = channel._sent[0]["content"]
assert "Sprint Planning" in response or "meeting" in response.lower()
# Quick query should NOT have escalation link
assert "openjarvis://" not in response
ca.shutdown()
def test_deep_query_gets_escalation_link(
populated_store: KnowledgeStore,
) -> None:
"""Deep query → agent searches → preview + escalation link."""
search_call = {
"content": "",
"usage": {"prompt_tokens": 100, "completion_tokens": 50, "total_tokens": 150},
"model": "test",
"finish_reason": "tool_calls",
"tool_calls": [
{
"id": "call_1",
"type": "function",
"function": {
"name": "knowledge_search",
"arguments": json.dumps({"query": "budget API"}),
},
}
],
}
final_answer = {
"content": (
"## Research Report\n\n"
+ "Based on analysis of Slack messages and emails, "
+ "the team discussed API redesign and Q3 budget. "
* 10
+ "\n\n**Sources:**\n1. [slack] #engineering\n2. [gmail] Q3 Budget"
),
"usage": {"prompt_tokens": 500, "completion_tokens": 300, "total_tokens": 800},
"model": "test",
"finish_reason": "stop",
}
mock_engine = MagicMock()
mock_engine.engine_id = "mock"
mock_engine.generate.side_effect = [search_call, final_answer]
retriever = TwoStageRetriever(store=populated_store)
ks_tool = KnowledgeSearchTool(
store=populated_store, retriever=retriever
)
agent = DeepResearchAgent(
engine=mock_engine,
model="test",
tools=[ks_tool],
max_turns=3,
)
channel = FakeChannel()
ca = ChannelAgent(channel=channel, agent=agent)
channel.simulate(
"Summarize all discussions about budget and API redesign"
)
time.sleep(1.0)
assert len(channel._sent) >= 1
response = channel._sent[-1]["content"]
# Deep query should have escalation link
assert "openjarvis://" in response
ca.shutdown()
```
- [ ] **Step 2: Run integration test**
Run: `cd /lambda/nfs/lambda-stanford/jonsf/scratch_v2/OpenJarvis && uv run pytest tests/agents/test_channel_agent_integration.py -v`
Expected: All 2 tests PASS.
- [ ] **Step 3: Run full test suite**
Run: `cd /lambda/nfs/lambda-stanford/jonsf/scratch_v2/OpenJarvis && uv run pytest tests/agents/test_channel_agent.py tests/agents/test_channel_agent_integration.py tests/agents/test_deep_research.py tests/connectors/ tests/tools/test_knowledge_search.py -v --tb=short`
Expected: All tests PASS.
- [ ] **Step 4: Run linter**
Run: `cd /lambda/nfs/lambda-stanford/jonsf/scratch_v2/OpenJarvis && uv run ruff check src/openjarvis/agents/channel_agent.py tests/agents/test_channel_agent.py tests/agents/test_channel_agent_integration.py`
Expected: No errors.
- [ ] **Step 5: Commit**
```bash
git add tests/agents/test_channel_agent_integration.py
git commit -m "feat: add integration test for ChannelAgent with DeepResearchAgent"
```
---
## Post-Plan Notes
**What this plan produces:**
- `classify_query()` — automatic quick/deep classification (no user input needed)
- `ChannelAgent` — bridges any `BaseChannel` to the `DeepResearchAgent` with:
- Non-blocking handler (thread pool)
- Quick inline answers for simple lookups
- Preview + `openjarvis://research/{session_id}` escalation link for complex research
- Error handling with user-friendly messages
- Event bus telemetry
- Integration test proving the full stack: channel message → classification → agent → tool call → retrieval → response
**What this does NOT do (future work):**
- Platform-specific formatting (Slack Block Kit, WhatsApp markdown) — can be added per-channel later
- BlueBubbles incoming message support — needs upstream work on the bridge
- Tauri `openjarvis://` URL handler registration — that's Phase 2B (desktop)
- Persistent conversation context across messages — would need session store integration
**Connecting to real channels:** After this plan, wiring to real Slack/WhatsApp is just:
```python
channel = SlackChannel(bot_token="xoxb-...", app_token="xapp-...")
channel.connect()
agent = DeepResearchAgent(engine, model, tools=[ks_tool])
ca = ChannelAgent(channel=channel, agent=agent)
# Now the Slack bot responds to DMs with Deep Research answers
```
docs/superpowers/plans/2026-03-26-deep-research-phase5-polish.md
-596
View File
@@ -1,596 +0,0 @@
# Deep Research Phase 5: Incremental Sync + Attachment Store Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Add incremental delta sync (only fetch new/modified items after initial bulk), content-addressed attachment storage with SHA-256 dedup, a sync trigger API endpoint, and text extraction from PDF/Office attachments.
**Architecture:** The SyncEngine is extended to pass `since=last_sync_time` to connectors on subsequent syncs. A new `AttachmentStore` manages content-addressed blob storage at `~/.openjarvis/blobs/` with SHA-256 dedup and a metadata table. The IngestionPipeline is extended to extract text from attachments and index it alongside parent documents. A `POST /v1/connectors/{id}/sync` endpoint triggers syncs via the API.
**Tech Stack:** Python 3.10+, sqlite3, hashlib (SHA-256), pdfplumber (optional, PDF text extraction), pytest
**Spec:** `docs/superpowers/specs/2026-03-25-deep-research-setup-design.md` — Section 6 (Ingestion Pipeline), Phase 5
**Depends on:** Phase 1 (SyncEngine, IngestionPipeline, KnowledgeStore), Phase 2B-i (API router)
---
## File Structure
```
src/openjarvis/connectors/
├── attachment_store.py # Content-addressed blob store with SHA-256 dedup
├── sync_engine.py # (modify) Wire incremental sync via since param
├── pipeline.py # (modify) Process attachments during ingest
├── gmail.py # (modify) Wire since param for incremental
src/openjarvis/server/
├── connectors_router.py # (modify) Add POST /connectors/{id}/sync endpoint
tests/connectors/
├── test_attachment_store.py # AttachmentStore tests
├── test_incremental_sync.py # Incremental sync tests
```
---
### Task 1: Incremental Sync (Wire `since` Through SyncEngine)
**Files:**
- Modify: `src/openjarvis/connectors/sync_engine.py`
- Create: `tests/connectors/test_incremental_sync.py`
- [ ] **Step 1: Write failing tests**
Create `tests/connectors/test_incremental_sync.py`:
```python
"""Tests for incremental sync — only fetches items after last sync time."""
from __future__ import annotations
from datetime import datetime, timezone
from pathlib import Path
from typing import Iterator, Optional
import pytest
from openjarvis.connectors._stubs import BaseConnector, Document, SyncStatus
from openjarvis.connectors.pipeline import IngestionPipeline
from openjarvis.connectors.store import KnowledgeStore
from openjarvis.connectors.sync_engine import SyncEngine
class TimestampConnector(BaseConnector):
"""Connector that records what `since` value it receives."""
connector_id = "ts_test"
display_name = "Timestamp Test"
auth_type = "filesystem"
def __init__(self, docs: list[Document] | None = None) -> None:
self._docs = docs or []
self._received_since: Optional[datetime] = None
def is_connected(self) -> bool:
return True
def disconnect(self) -> None:
pass
def sync(
self,
*,
since: Optional[datetime] = None,
cursor: Optional[str] = None,
) -> Iterator[Document]:
self._received_since = since
for doc in self._docs:
if since and doc.timestamp < since:
continue
yield doc
def sync_status(self) -> SyncStatus:
return SyncStatus(state="idle")
def _make_doc(
doc_id: str, ts: datetime, content: str = "test"
) -> Document:
return Document(
doc_id=doc_id,
source="ts_test",
doc_type="message",
content=content,
timestamp=ts,
)
@pytest.fixture
def setup(tmp_path: Path):
store = KnowledgeStore(
db_path=str(tmp_path / "inc.db")
)
pipeline = IngestionPipeline(store=store)
engine = SyncEngine(
pipeline=pipeline,
state_db=str(tmp_path / "state.db"),
)
return store, pipeline, engine
def test_first_sync_passes_no_since(setup) -> None:
store, pipeline, engine = setup
old = _make_doc(
"ts:1",
datetime(2024, 1, 1, tzinfo=timezone.utc),
)
conn = TimestampConnector(docs=[old])
engine.sync(conn)
assert conn._received_since is None
def test_second_sync_passes_since(setup) -> None:
store, pipeline, engine = setup
old = _make_doc(
"ts:1",
datetime(2024, 1, 1, tzinfo=timezone.utc),
)
conn1 = TimestampConnector(docs=[old])
engine.sync(conn1)
# Second sync should receive since= from last sync
new = _make_doc(
"ts:2",
datetime(2024, 6, 1, tzinfo=timezone.utc),
)
conn2 = TimestampConnector(docs=[old, new])
conn2.connector_id = "ts_test"
engine.sync(conn2)
assert conn2._received_since is not None
def test_incremental_only_adds_new_items(setup) -> None:
store, pipeline, engine = setup
old = _make_doc(
"ts:1",
datetime(2024, 1, 1, tzinfo=timezone.utc),
content="Old item",
)
conn1 = TimestampConnector(docs=[old])
engine.sync(conn1)
assert store.count() >= 1
count_after_first = store.count()
# Second sync with same + new doc
new = _make_doc(
"ts:2",
datetime(2024, 6, 1, tzinfo=timezone.utc),
content="New item",
)
conn2 = TimestampConnector(docs=[old, new])
conn2.connector_id = "ts_test"
engine.sync(conn2)
# Should have more chunks now
assert store.count() > count_after_first
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `cd /lambda/nfs/lambda-stanford/jonsf/scratch_v2/OpenJarvis && uv run pytest tests/connectors/test_incremental_sync.py -v`
Expected: `test_second_sync_passes_since` FAILS (since is never passed)
- [ ] **Step 3: Modify SyncEngine to pass `since`**
In `src/openjarvis/connectors/sync_engine.py`, modify the `sync()` method. After loading the checkpoint, parse `last_sync` into a datetime and pass it as `since=` to the connector:
Find the line that calls `connector.sync(cursor=prior_cursor)` and change it to also pass `since`:
```python
# Parse last_sync from checkpoint into datetime
since: Optional[datetime] = None
if checkpoint and checkpoint.get("last_sync"):
try:
since = datetime.fromisoformat(checkpoint["last_sync"])
except (ValueError, TypeError):
pass
docs = connector.sync(since=since, cursor=prior_cursor)
```
Add `from datetime import datetime` to imports if not already present.
- [ ] **Step 4: Run tests**
Run: `cd /lambda/nfs/lambda-stanford/jonsf/scratch_v2/OpenJarvis && uv run pytest tests/connectors/test_incremental_sync.py -v`
Expected: All 3 tests PASS.
- [ ] **Step 5: Commit**
```bash
git add src/openjarvis/connectors/sync_engine.py tests/connectors/test_incremental_sync.py
git commit -m "feat: wire incremental sync via since param in SyncEngine"
```
---
### Task 2: Attachment Store (Content-Addressed Blobs)
**Files:**
- Create: `src/openjarvis/connectors/attachment_store.py`
- Create: `tests/connectors/test_attachment_store.py`
- [ ] **Step 1: Write failing tests**
Create `tests/connectors/test_attachment_store.py`:
```python
"""Tests for AttachmentStore — content-addressed blob storage."""
from __future__ import annotations
from pathlib import Path
import pytest
from openjarvis.connectors.attachment_store import AttachmentStore
@pytest.fixture
def store(tmp_path: Path) -> AttachmentStore:
return AttachmentStore(base_dir=str(tmp_path / "blobs"))
def test_store_returns_sha256(store: AttachmentStore) -> None:
sha = store.store(
content=b"Hello, world!",
filename="hello.txt",
mime_type="text/plain",
source_doc_id="gmail:msg1",
)
assert len(sha) == 64 # SHA-256 hex digest
def test_store_creates_blob_file(
store: AttachmentStore, tmp_path: Path
) -> None:
sha = store.store(
content=b"PDF content here",
filename="report.pdf",
mime_type="application/pdf",
source_doc_id="gdrive:doc1",
)
blob_path = Path(store._base_dir) / sha[:2] / sha
assert blob_path.exists()
assert blob_path.read_bytes() == b"PDF content here"
def test_dedup_same_content(store: AttachmentStore) -> None:
content = b"Identical file content"
sha1 = store.store(
content=content,
filename="file1.txt",
mime_type="text/plain",
source_doc_id="gmail:1",
)
sha2 = store.store(
content=content,
filename="file2.txt",
mime_type="text/plain",
source_doc_id="slack:1",
)
assert sha1 == sha2 # Same content → same hash
def test_dedup_tracks_multiple_sources(
store: AttachmentStore,
) -> None:
content = b"Shared file"
store.store(
content=content,
filename="shared.pdf",
mime_type="application/pdf",
source_doc_id="gmail:1",
)
store.store(
content=content,
filename="shared.pdf",
mime_type="application/pdf",
source_doc_id="slack:1",
)
meta = store.get_metadata(
store.store(
content=content,
filename="shared.pdf",
mime_type="application/pdf",
source_doc_id="gdrive:1",
)
)
assert len(meta["source_doc_ids"]) >= 3
def test_get_metadata(store: AttachmentStore) -> None:
sha = store.store(
content=b"test",
filename="test.txt",
mime_type="text/plain",
source_doc_id="gmail:1",
)
meta = store.get_metadata(sha)
assert meta["filename"] == "test.txt"
assert meta["mime_type"] == "text/plain"
assert meta["size_bytes"] == 4
def test_get_content(store: AttachmentStore) -> None:
sha = store.store(
content=b"retrieve me",
filename="data.bin",
mime_type="application/octet-stream",
source_doc_id="test:1",
)
assert store.get_content(sha) == b"retrieve me"
def test_get_content_nonexistent(
store: AttachmentStore,
) -> None:
assert store.get_content("nonexistent") is None
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `cd /lambda/nfs/lambda-stanford/jonsf/scratch_v2/OpenJarvis && uv run pytest tests/connectors/test_attachment_store.py -v`
Expected: FAIL — `ModuleNotFoundError`
- [ ] **Step 3: Implement AttachmentStore**
Create `src/openjarvis/connectors/attachment_store.py`:
```python
"""Content-addressed attachment store with SHA-256 dedup.
Blobs stored at: {base_dir}/{sha256[:2]}/{sha256}
Metadata tracked in SQLite alongside the blobs.
"""
from __future__ import annotations
import hashlib
import json
import logging
import sqlite3
from pathlib import Path
from typing import Any, Dict, List, Optional
from openjarvis.core.config import DEFAULT_CONFIG_DIR
logger = logging.getLogger(__name__)
_DEFAULT_BASE_DIR = str(DEFAULT_CONFIG_DIR / "blobs")
_META_SCHEMA = """\
CREATE TABLE IF NOT EXISTS attachments (
sha256 TEXT PRIMARY KEY,
filename TEXT NOT NULL,
mime_type TEXT NOT NULL DEFAULT '',
size_bytes INTEGER NOT NULL DEFAULT 0,
source_doc_ids TEXT NOT NULL DEFAULT '[]',
created_at TEXT NOT NULL
);
"""
class AttachmentStore:
"""Content-addressed blob store for email attachments, shared files, etc.
Each file is stored once by SHA-256 hash. Multiple source documents
can reference the same blob (dedup across Gmail, Slack, Drive, etc.).
"""
def __init__(self, base_dir: str = "") -> None:
self._base_dir = base_dir or _DEFAULT_BASE_DIR
Path(self._base_dir).mkdir(parents=True, exist_ok=True)
db_path = str(Path(self._base_dir) / "attachments.db")
self._conn = sqlite3.connect(db_path)
self._conn.row_factory = sqlite3.Row
self._conn.executescript(_META_SCHEMA)
def store(
self,
content: bytes,
*,
filename: str,
mime_type: str = "",
source_doc_id: str = "",
) -> str:
"""Store a blob and return its SHA-256 hash.
If the same content already exists, just adds the source_doc_id
to the existing metadata (dedup).
"""
sha = hashlib.sha256(content).hexdigest()
# Write blob file (idempotent)
blob_dir = Path(self._base_dir) / sha[:2]
blob_dir.mkdir(parents=True, exist_ok=True)
blob_path = blob_dir / sha
if not blob_path.exists():
blob_path.write_bytes(content)
# Upsert metadata
row = self._conn.execute(
"SELECT source_doc_ids FROM attachments WHERE sha256 = ?",
(sha,),
).fetchone()
if row is None:
from datetime import datetime, timezone
self._conn.execute(
"""INSERT INTO attachments
(sha256, filename, mime_type, size_bytes,
source_doc_ids, created_at)
VALUES (?, ?, ?, ?, ?, ?)""",
(
sha,
filename,
mime_type,
len(content),
json.dumps([source_doc_id] if source_doc_id else []),
datetime.now(tz=timezone.utc).isoformat(),
),
)
else:
# Add source_doc_id if not already tracked
ids = json.loads(row["source_doc_ids"])
if source_doc_id and source_doc_id not in ids:
ids.append(source_doc_id)
self._conn.execute(
"UPDATE attachments SET source_doc_ids = ? WHERE sha256 = ?",
(json.dumps(ids), sha),
)
self._conn.commit()
return sha
def get_metadata(self, sha: str) -> Optional[Dict[str, Any]]:
"""Get metadata for an attachment by SHA-256."""
row = self._conn.execute(
"SELECT * FROM attachments WHERE sha256 = ?", (sha,)
).fetchone()
if row is None:
return None
return {
"sha256": row["sha256"],
"filename": row["filename"],
"mime_type": row["mime_type"],
"size_bytes": row["size_bytes"],
"source_doc_ids": json.loads(row["source_doc_ids"]),
"created_at": row["created_at"],
}
def get_content(self, sha: str) -> Optional[bytes]:
"""Read blob content by SHA-256. Returns None if not found."""
blob_path = Path(self._base_dir) / sha[:2] / sha
if not blob_path.exists():
return None
return blob_path.read_bytes()
def close(self) -> None:
self._conn.close()
```
- [ ] **Step 4: Run tests**
Run: `cd /lambda/nfs/lambda-stanford/jonsf/scratch_v2/OpenJarvis && uv run pytest tests/connectors/test_attachment_store.py -v`
Expected: All 7 tests PASS.
- [ ] **Step 5: Commit**
```bash
git add src/openjarvis/connectors/attachment_store.py tests/connectors/test_attachment_store.py
git commit -m "feat: add content-addressed AttachmentStore with SHA-256 dedup"
```
---
### Task 3: Sync Trigger API Endpoint
**Files:**
- Modify: `src/openjarvis/server/connectors_router.py`
- Modify: `tests/server/test_connectors_router.py`
- [ ] **Step 1: Add test for POST /sync**
Add to `tests/server/test_connectors_router.py`:
```python
def test_trigger_sync(app, tmp_path: Path) -> None:
"""POST /v1/connectors/obsidian/sync triggers a sync."""
vault = tmp_path / "vault"
vault.mkdir()
(vault / "note.md").write_text("# Test note\n\nContent here.")
# First connect
app.post(
"/v1/connectors/obsidian/connect",
json={"path": str(vault)},
)
# Then trigger sync
resp = app.post("/v1/connectors/obsidian/sync")
assert resp.status_code == 200
data = resp.json()
assert data["chunks_indexed"] >= 1
```
- [ ] **Step 2: Implement POST /sync endpoint**
Add to the router in `src/openjarvis/server/connectors_router.py`:
```python
@router.post("/connectors/{connector_id}/sync")
def trigger_sync(connector_id: str) -> Dict[str, Any]:
"""Trigger an incremental sync for a connector."""
if not ConnectorRegistry.contains(connector_id):
raise HTTPException(
status_code=404,
detail=f"Connector '{connector_id}' not found",
)
inst = _get_or_create(connector_id)
if not inst.is_connected():
raise HTTPException(
status_code=400,
detail=f"Connector '{connector_id}' is not connected",
)
from openjarvis.connectors.pipeline import IngestionPipeline
from openjarvis.connectors.store import KnowledgeStore
from openjarvis.connectors.sync_engine import SyncEngine
store = KnowledgeStore()
pipeline = IngestionPipeline(store=store)
engine = SyncEngine(pipeline=pipeline)
chunks = engine.sync(inst)
return {
"connector_id": connector_id,
"chunks_indexed": chunks,
"status": "complete",
}
```
- [ ] **Step 3: Run tests**
Run: `cd /lambda/nfs/lambda-stanford/jonsf/scratch_v2/OpenJarvis && uv run --extra server pytest tests/server/test_connectors_router.py -v`
Expected: All tests PASS (existing + new).
- [ ] **Step 4: Commit**
```bash
git add src/openjarvis/server/connectors_router.py tests/server/test_connectors_router.py
git commit -m "feat: add POST /v1/connectors/{id}/sync to trigger incremental sync"
```
---
## Post-Plan Notes
**What this plan produces:**
- Incremental sync via `since=last_sync_time` passed to connectors on subsequent syncs
- Content-addressed `AttachmentStore` with SHA-256 dedup and multi-source tracking
- `POST /v1/connectors/{id}/sync` API endpoint to trigger syncs
- Tests for all of the above
**What's NOT in this plan (future):**
- PDF text extraction from attachments (needs pdfplumber wired into pipeline)
- Office doc extraction (python-docx/openpyxl)
- Wiring attachment extraction into the IngestionPipeline
- Populating `doc.attachments` in Gmail/Drive connectors
These are natural follow-ups once the store exists. The current plan establishes the foundation (dedup blob store + incremental sync + sync trigger API).
docs/superpowers/plans/2026-03-26-deep-research-vertical-slice.md
-757
View File
@@ -1,757 +0,0 @@
# Deep Research Vertical Slice Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Get the full Deep Research experience working end-to-end on a MacBook with real data — connect local sources, ingest, retrieve, and run the DeepResearchAgent with Qwen3.5 4B via Ollama producing cited research reports.
**Architecture:** Fix the Apple Notes protobuf extraction test, build a `jarvis deep-research-setup` CLI command that auto-detects local sources (Apple Notes, iMessage, Obsidian), ingests them into a shared KnowledgeStore, and launches an interactive chat session with the DeepResearchAgent. Then wire the missing API router so the desktop wizard UI works too.
**Tech Stack:** Python 3.10+, Click (CLI), SQLite/FTS5, Ollama (Qwen3.5 4B), ColBERT (optional reranking), React/Tauri (wizard smoke test)
---
### Task 1: Fix Apple Notes test
**Files:**
- Modify: `src/openjarvis/connectors/apple_notes.py` (lines 73-103, `_extract_text_from_zdata`)
- Modify: `tests/connectors/test_apple_notes.py` (lines 42-58, test data + lines 127-140, assertion)
The `_extract_text_from_zdata` function currently strips protobuf control bytes but not HTML tags. The test creates fake notes with HTML content. Real Apple Notes uses protobuf. Fix: strip HTML tags too (handles both formats gracefully), and update the test data to use protobuf-like content that exercises the actual code path.
- [ ] **Step 1: Read current files**
Read both files to confirm the exact current state (there are uncommitted changes from the earlier session):
```bash
git diff src/openjarvis/connectors/apple_notes.py
git diff tests/connectors/test_apple_notes.py
```
- [ ] **Step 2: Update `_extract_text_from_zdata` to also strip HTML tags**
In `src/openjarvis/connectors/apple_notes.py`, the function should strip HTML tags after stripping protobuf bytes. This handles both real protobuf data and HTML test data. Edit the function:
```python
def _extract_text_from_zdata(zdata: bytes) -> str:
"""Decompress gzip bytes and extract plain text from the protobuf payload.
Parameters
----------
zdata:
Raw bytes from the ``ZDATA`` column — gzip-compressed protobuf
(``com.apple.notes.ICNote``).
Returns
-------
str
Plain text with protobuf control bytes stripped. Returns an empty
string if decompression fails.
"""
try:
raw = gzip.decompress(zdata)
except Exception: # noqa: BLE001
return ""
text = raw.decode("utf-8", errors="replace")
# Strip HTML tags (older Notes versions or test data may contain HTML)
text = re.sub(r"<[^>]+>", "", text)
# Strip non-printable control bytes and U+FFFD replacement chars that
# come from the protobuf wire format.
cleaned = re.sub(r"[\x00-\x09\x0b\x0c\x0e-\x1f\x7f-\x9f\ufffd]+", " ", text)
# Collapse whitespace runs
cleaned = re.sub(r" {2,}", " ", cleaned)
cleaned = re.sub(r"\n{3,}", "\n\n", cleaned)
cleaned = cleaned.strip()
# Strip short leading protobuf varint artifacts (e.g. "3 3 " or "b b ")
cleaned = re.sub(r"^(?:[a-z0-9] ){1,4}", "", cleaned)
return cleaned.strip()
```
- [ ] **Step 3: Run the Apple Notes tests**
```bash
uv run pytest tests/connectors/test_apple_notes.py -v --tb=short
```
Expected: All 9 tests PASS (including `test_sync_decompresses_content`).
- [ ] **Step 4: Run the iMessage tests too (regression check)**
```bash
uv run pytest tests/connectors/test_imessage.py -v --tb=short
```
Expected: All tests PASS.
- [ ] **Step 5: Commit**
```bash
git add src/openjarvis/connectors/apple_notes.py tests/connectors/test_apple_notes.py
git commit -m "fix: Apple Notes protobuf extraction handles HTML too, fix ZTITLE1 column"
```
---
### Task 2: Build `jarvis deep-research-setup` CLI command
**Files:**
- Create: `src/openjarvis/cli/deep_research_setup_cmd.py`
- Modify: `src/openjarvis/cli/__init__.py` (add command registration)
- Test: `tests/cli/test_deep_research_setup.py`
This is the core of the vertical slice — a single command that auto-detects local sources, ingests data, and drops you into a research chat.
- [ ] **Step 1: Write the test file**
Create `tests/cli/test_deep_research_setup.py`:
```python
"""Tests for the deep-research-setup CLI command."""
from __future__ import annotations
import sqlite3
import tempfile
from pathlib import Path
from unittest.mock import patch
import pytest
from click.testing import CliRunner
from openjarvis.cli import cli
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _create_fake_notes_db(db_path: Path) -> None:
"""Create a minimal Apple Notes SQLite database."""
import gzip
conn = sqlite3.connect(str(db_path))
conn.executescript("""
CREATE TABLE ZICCLOUDSYNCINGOBJECT (
Z_PK INTEGER PRIMARY KEY,
ZTITLE TEXT,
ZTITLE1 TEXT,
ZMODIFICATIONDATE REAL,
ZIDENTIFIER TEXT,
ZNOTE INTEGER
);
CREATE TABLE ZICNOTEDATA (
Z_PK INTEGER PRIMARY KEY,
ZDATA BLOB,
ZNOTE INTEGER
);
""")
content = gzip.compress(b"Test note about meetings")
conn.execute(
"INSERT INTO ZICCLOUDSYNCINGOBJECT VALUES (1, NULL, 'Test Note', 694310400.0, 'n1', 1)"
)
conn.execute("INSERT INTO ZICNOTEDATA VALUES (1, ?, 1)", (content,))
conn.commit()
conn.close()
def _create_fake_imessage_db(db_path: Path) -> None:
"""Create a minimal iMessage SQLite database."""
conn = sqlite3.connect(str(db_path))
conn.executescript("""
CREATE TABLE handle (ROWID INTEGER PRIMARY KEY, id TEXT);
CREATE TABLE chat (ROWID INTEGER PRIMARY KEY, chat_identifier TEXT, display_name TEXT);
CREATE TABLE chat_message_join (chat_id INTEGER, message_id INTEGER);
CREATE TABLE message (
ROWID INTEGER PRIMARY KEY, text TEXT, handle_id INTEGER,
date INTEGER, is_from_me INTEGER
);
""")
conn.execute("INSERT INTO handle VALUES (1, '+15551234567')")
conn.execute("INSERT INTO chat VALUES (1, '+15551234567', 'Test Chat')")
conn.execute("INSERT INTO chat_message_join VALUES (1, 1)")
conn.execute("INSERT INTO message VALUES (1, 'Hello from test', 1, 694310400000000000, 0)")
conn.commit()
conn.close()
# ---------------------------------------------------------------------------
# Tests
# ---------------------------------------------------------------------------
def test_detect_local_sources(tmp_path: Path) -> None:
"""Auto-detection finds Apple Notes and iMessage when DBs exist."""
from openjarvis.cli.deep_research_setup_cmd import detect_local_sources
notes_db = tmp_path / "NoteStore.sqlite"
imessage_db = tmp_path / "chat.db"
_create_fake_notes_db(notes_db)
_create_fake_imessage_db(imessage_db)
sources = detect_local_sources(
notes_db_path=notes_db,
imessage_db_path=imessage_db,
obsidian_vault_path=None,
)
ids = [s["connector_id"] for s in sources]
assert "apple_notes" in ids
assert "imessage" in ids
def test_detect_skips_missing_sources(tmp_path: Path) -> None:
"""Auto-detection skips sources whose files don't exist."""
from openjarvis.cli.deep_research_setup_cmd import detect_local_sources
sources = detect_local_sources(
notes_db_path=tmp_path / "nonexistent.sqlite",
imessage_db_path=tmp_path / "nonexistent.db",
obsidian_vault_path=None,
)
assert len(sources) == 0
def test_detect_includes_obsidian_when_vault_exists(tmp_path: Path) -> None:
"""Auto-detection includes Obsidian when vault path exists."""
from openjarvis.cli.deep_research_setup_cmd import detect_local_sources
vault = tmp_path / "vault"
vault.mkdir()
(vault / "note.md").write_text("# Hello")
sources = detect_local_sources(
notes_db_path=tmp_path / "nonexistent.sqlite",
imessage_db_path=tmp_path / "nonexistent.db",
obsidian_vault_path=vault,
)
ids = [s["connector_id"] for s in sources]
assert "obsidian" in ids
def test_ingest_sources(tmp_path: Path) -> None:
"""ingest_sources connects and ingests documents into KnowledgeStore."""
from openjarvis.cli.deep_research_setup_cmd import detect_local_sources, ingest_sources
from openjarvis.connectors.store import KnowledgeStore
notes_db = tmp_path / "NoteStore.sqlite"
_create_fake_notes_db(notes_db)
sources = detect_local_sources(
notes_db_path=notes_db,
imessage_db_path=tmp_path / "nonexistent.db",
obsidian_vault_path=None,
)
db_path = tmp_path / "knowledge.db"
store = KnowledgeStore(str(db_path))
total = ingest_sources(sources, store)
assert total > 0
assert store.count() > 0
store.close()
```
- [ ] **Step 2: Run tests to verify they fail**
```bash
uv run pytest tests/cli/test_deep_research_setup.py -v --tb=short
```
Expected: FAIL — `ModuleNotFoundError: No module named 'openjarvis.cli.deep_research_setup_cmd'`
- [ ] **Step 3: Create the CLI command module**
Create `src/openjarvis/cli/deep_research_setup_cmd.py`:
```python
"""``jarvis deep-research-setup`` — auto-detect local sources, ingest, and chat.
Walks the user through connecting local data sources (Apple Notes, iMessage,
Obsidian), ingesting them into a shared KnowledgeStore, and launching an
interactive Deep Research chat session with Qwen3.5 via Ollama.
"""
from __future__ import annotations
import sys
from pathlib import Path
from typing import Any, Dict, List, Optional
import click
from rich.console import Console
from rich.table import Table
from openjarvis.connectors.pipeline import IngestionPipeline
from openjarvis.connectors.store import KnowledgeStore
from openjarvis.connectors.sync_engine import SyncEngine
from openjarvis.core.config import DEFAULT_CONFIG_DIR
# ---------------------------------------------------------------------------
# Constants
# ---------------------------------------------------------------------------
_DEFAULT_NOTES_DB = (
Path.home()
/ "Library"
/ "Group Containers"
/ "group.com.apple.notes"
/ "NoteStore.sqlite"
)
_DEFAULT_IMESSAGE_DB = Path.home() / "Library" / "Messages" / "chat.db"
_OLLAMA_MODEL = "qwen3.5:4b"
# ---------------------------------------------------------------------------
# Detection
# ---------------------------------------------------------------------------
def detect_local_sources(
*,
notes_db_path: Optional[Path] = None,
imessage_db_path: Optional[Path] = None,
obsidian_vault_path: Optional[Path] = None,
) -> List[Dict[str, Any]]:
"""Return a list of available local sources with their config.
Each entry is a dict with keys: ``connector_id``, ``display_name``,
``config`` (kwargs for the connector constructor).
"""
sources: List[Dict[str, Any]] = []
notes_path = notes_db_path or _DEFAULT_NOTES_DB
if notes_path.exists():
sources.append({
"connector_id": "apple_notes",
"display_name": "Apple Notes",
"config": {"db_path": str(notes_path)},
})
imessage_path = imessage_db_path or _DEFAULT_IMESSAGE_DB
if imessage_path.exists():
sources.append({
"connector_id": "imessage",
"display_name": "iMessage",
"config": {"db_path": str(imessage_path)},
})
if obsidian_vault_path and obsidian_vault_path.is_dir():
sources.append({
"connector_id": "obsidian",
"display_name": "Obsidian / Markdown",
"config": {"vault_path": str(obsidian_vault_path)},
})
return sources
# ---------------------------------------------------------------------------
# Ingestion
# ---------------------------------------------------------------------------
def _instantiate_connector(connector_id: str, config: Dict[str, Any]) -> Any:
"""Lazily import and instantiate a connector by ID."""
if connector_id == "apple_notes":
from openjarvis.connectors.apple_notes import AppleNotesConnector
return AppleNotesConnector(db_path=config.get("db_path", ""))
elif connector_id == "imessage":
from openjarvis.connectors.imessage import IMessageConnector
return IMessageConnector(db_path=config.get("db_path", ""))
elif connector_id == "obsidian":
from openjarvis.connectors.obsidian import ObsidianConnector
return ObsidianConnector(vault_path=config.get("vault_path", ""))
else:
msg = f"Unknown connector: {connector_id}"
raise ValueError(msg)
def ingest_sources(
sources: List[Dict[str, Any]],
store: KnowledgeStore,
) -> int:
"""Connect and ingest all sources into the KnowledgeStore.
Returns total chunks indexed across all sources.
"""
pipeline = IngestionPipeline(store)
engine = SyncEngine(pipeline)
total = 0
for src in sources:
connector = _instantiate_connector(src["connector_id"], src["config"])
chunks = engine.sync(connector)
total += chunks
return total
# ---------------------------------------------------------------------------
# Chat launch
# ---------------------------------------------------------------------------
def _launch_chat(store: KnowledgeStore, console: Console) -> None:
"""Start an interactive Deep Research chat session."""
from openjarvis.agents.deep_research import DeepResearchAgent
from openjarvis.connectors.retriever import TwoStageRetriever
from openjarvis.engine.ollama import OllamaEngine
from openjarvis.tools.knowledge_search import KnowledgeSearchTool
console.print("\n[bold]Setting up Deep Research agent...[/bold]")
# Engine
engine = OllamaEngine()
if not engine.health():
console.print(
"[red]Ollama is not running.[/red] Start it with: "
"[bold]ollama serve[/bold]"
)
return
models = engine.list_models()
if _OLLAMA_MODEL not in models and f"{_OLLAMA_MODEL}:latest" not in models:
# Check without tag too
base_name = _OLLAMA_MODEL.split(":")[0]
matching = [m for m in models if m.startswith(base_name)]
if not matching:
console.print(
f"[yellow]Model {_OLLAMA_MODEL} not found.[/yellow] "
f"Pull it with: [bold]ollama pull {_OLLAMA_MODEL}[/bold]"
)
return
# Retriever + tool
retriever = TwoStageRetriever(store)
search_tool = KnowledgeSearchTool(retriever=retriever)
# Agent
agent = DeepResearchAgent(
engine=engine,
model=_OLLAMA_MODEL,
tools=[search_tool],
interactive=True,
)
console.print(
f"[green]Ready![/green] Using [bold]{_OLLAMA_MODEL}[/bold] via Ollama.\n"
"Type your research question. Type [bold]/quit[/bold] to exit.\n"
)
# REPL
while True:
try:
query = console.input("[bold blue]research>[/bold blue] ").strip()
except (EOFError, KeyboardInterrupt):
break
if not query:
continue
if query.lower() in ("/quit", "/exit", "quit", "exit"):
break
try:
result = agent.run(query)
console.print(f"\n{result.content}\n")
if result.metadata and result.metadata.get("sources"):
console.print("[dim]Sources:[/dim]")
for s in result.metadata["sources"]:
console.print(f" [dim]- {s}[/dim]")
console.print()
except Exception as exc: # noqa: BLE001
console.print(f"[red]Error: {exc}[/red]\n")
# ---------------------------------------------------------------------------
# CLI command
# ---------------------------------------------------------------------------
@click.command("deep-research-setup")
@click.option(
"--obsidian-vault",
type=click.Path(exists=True, file_okay=False),
default=None,
help="Path to an Obsidian vault directory.",
)
@click.option("--skip-chat", is_flag=True, help="Ingest only, don't launch chat.")
def deep_research_setup(obsidian_vault: Optional[str], skip_chat: bool) -> None:
"""Auto-detect local data sources, ingest, and launch Deep Research chat."""
console = Console()
console.print("\n[bold]Deep Research Setup[/bold]\n")
# 1. Detect
vault_path = Path(obsidian_vault) if obsidian_vault else None
sources = detect_local_sources(obsidian_vault_path=vault_path)
if not sources:
console.print(
"[yellow]No local data sources detected.[/yellow]\n"
"On macOS, ensure Full Disk Access is granted in "
"System Settings > Privacy & Security."
)
sys.exit(1)
# 2. Confirm
table = Table(title="Detected Sources")
table.add_column("Source", style="bold")
table.add_column("Status", style="green")
for src in sources:
table.add_row(src["display_name"], "ready")
console.print(table)
console.print()
if not click.confirm("Ingest these sources?", default=True):
sys.exit(0)
# 3. Ingest
db_path = DEFAULT_CONFIG_DIR / "knowledge.db"
db_path.parent.mkdir(parents=True, exist_ok=True)
store = KnowledgeStore(str(db_path))
console.print("\n[bold]Ingesting...[/bold]")
for src in sources:
connector = _instantiate_connector(src["connector_id"], src["config"])
pipeline = IngestionPipeline(store)
engine = SyncEngine(pipeline)
chunks = engine.sync(connector)
console.print(f" {src['display_name']}: [green]{chunks} chunks[/green]")
console.print(f"\n[bold green]Done![/bold green] {store.count()} total chunks in {db_path}\n")
# 4. Chat
if skip_chat:
return
_launch_chat(store, console)
```
- [ ] **Step 4: Run tests to verify they pass**
```bash
uv run pytest tests/cli/test_deep_research_setup.py -v --tb=short
```
Expected: All 4 tests PASS.
- [ ] **Step 5: Register the command in CLI __init__.py**
In `src/openjarvis/cli/__init__.py`, add the import and registration alongside the existing commands.
Add the import near the other CLI imports:
```python
from openjarvis.cli.deep_research_setup_cmd import deep_research_setup
```
Add the registration near the other `cli.add_command()` calls:
```python
cli.add_command(deep_research_setup, "deep-research-setup")
```
- [ ] **Step 6: Verify the command is discoverable**
```bash
uv run jarvis --help | grep deep-research
```
Expected: `deep-research-setup` appears in the command list.
- [ ] **Step 7: Commit**
```bash
git add src/openjarvis/cli/deep_research_setup_cmd.py tests/cli/test_deep_research_setup.py src/openjarvis/cli/__init__.py
git commit -m "feat: add jarvis deep-research-setup CLI command"
```
---
### Task 3: Wire connectors API router into FastAPI app
**Files:**
- Modify: `src/openjarvis/server/app.py`
The `connectors_router.py` already has all 6 endpoints implemented. It was never registered in the FastAPI app. This is a one-line fix.
- [ ] **Step 1: Read app.py to find where routers are included**
```bash
grep -n "include_router" src/openjarvis/server/app.py
```
Identify the section where `app.include_router(router)` and `app.include_router(dashboard_router)` are called.
- [ ] **Step 2: Add the connectors router import and registration**
Add the import at the top of `app.py` with the other router imports:
```python
from openjarvis.server.connectors_router import create_connectors_router
```
Add the router registration in the same block as the other `app.include_router()` calls:
```python
app.include_router(create_connectors_router())
```
- [ ] **Step 3: Verify the endpoints are now exposed**
```bash
uv run python3 -c "
from openjarvis.server.app import create_app
from openjarvis.engine.ollama import OllamaEngine
engine = OllamaEngine()
app = create_app(engine, 'test')
routes = [r.path for r in app.routes]
connector_routes = [r for r in routes if 'connector' in r]
print(f'Connector routes: {connector_routes}')
assert any('connector' in r for r in routes), 'No connector routes found!'
print('PASS: Connectors router is registered')
"
```
Expected: Lists `/connectors`, `/connectors/{connector_id}`, etc. and prints PASS.
- [ ] **Step 4: Commit**
```bash
git add src/openjarvis/server/app.py
git commit -m "fix: register connectors API router in FastAPI app"
```
---
### Task 4: End-to-end test with real data
**Files:** None (manual testing)
This is the live validation. We pull the Ollama model, run the setup command, and test the Deep Research agent with real personal data.
- [ ] **Step 1: Ensure Ollama is running and pull the model**
```bash
ollama list
ollama pull qwen3.5:4b
```
Expected: Model downloaded. If Ollama isn't running, start it with `ollama serve &`.
- [ ] **Step 2: Run the deep research setup**
```bash
uv run jarvis deep-research-setup
```
Expected output:
```
Deep Research Setup
Detected Sources
┌──────────────┬────────┐
│ Source │ Status │
├──────────────┼────────┤
│ Apple Notes │ ready │
│ iMessage │ ready │
└──────────────┴────────┘
Ingest these sources? [Y/n]: Y
Ingesting...
Apple Notes: ~100 chunks
iMessage: ~52000 chunks
Done! ~52100 total chunks in ~/.openjarvis/knowledge.db
Setting up Deep Research agent...
Ready! Using qwen3.5:4b via Ollama.
Type your research question. Type /quit to exit.
research>
```
If iMessage ingestion is too slow (52K messages), you can Ctrl+C and re-run with a smaller dataset — we can add a `--limit` flag later.
- [ ] **Step 3: Test retrieval quality with 3 queries**
At the `research>` prompt, try:
1. A person's name who appears in both Apple Notes and iMessage
2. A topic from your notes (e.g. "Georgia Tech" which appeared in Apple Notes)
3. A time-bounded query like "what messages did I send last week"
Evaluate: Does the agent use `knowledge_search`? Does it cite sources? Does it make multiple hops?
- [ ] **Step 4: Document results**
Note: query, response quality, number of tool calls, latency, and any errors. This informs whether we need a larger model or retrieval tuning.
---
### Task 5: Smoke test wizard UI
**Files:**
- None (manual testing)
- [ ] **Step 1: Start the FastAPI server**
```bash
uv run jarvis serve --host 127.0.0.1 --port 8000
```
- [ ] **Step 2: Open the frontend dev server**
In a second terminal:
```bash
cd frontend && npm run dev
```
Expected: Vite dev server starts at `http://localhost:5173`.
- [ ] **Step 3: Walk through the wizard**
Open `http://localhost:5173` in a browser. The setup wizard should:
1. Show the source picker with Apple Notes, iMessage, etc.
2. Allow connecting local sources (click → instant green checkmark for local auth types)
3. Show the ingest dashboard with progress polling from `/v1/connectors/{id}/sync`
4. Land on the "Ready" screen
Note any errors — the frontend was built on a remote server and never build-tested until now.
- [ ] **Step 4: Commit any fixes needed**
If frontend or API fixes are required, commit them:
```bash
git add -A
git commit -m "fix: wizard UI smoke test fixes"
```
---
### Task 6: Run full test suite (regression check)
**Files:** None
- [ ] **Step 1: Run all connector + agent tests**
```bash
uv run pytest tests/connectors/ tests/agents/test_deep_research.py tests/agents/test_deep_research_integration.py tests/agents/test_channel_agent.py tests/agents/test_channel_agent_integration.py tests/cli/test_deep_research_setup.py -v --tb=short
```
Expected: All tests PASS (220+ existing + 4 new from Task 2).
- [ ] **Step 2: Run linter**
```bash
uv run ruff check src/openjarvis/cli/deep_research_setup_cmd.py src/openjarvis/server/app.py src/openjarvis/connectors/apple_notes.py
```
Expected: No errors.
- [ ] **Step 3: Commit and push**
```bash
git push origin feat/deep-research-setup
```
docs/superpowers/plans/2026-03-26-outlook-and-token-sources.md
-724
View File
@@ -1,724 +0,0 @@
# Outlook Connector + Token Source Integration Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Generalize Gmail IMAP to support Outlook, and extend `jarvis deep-research-setup` to detect, connect, and ingest token-based sources (Slack, Notion, Granola, Gmail IMAP, Outlook) alongside local sources.
**Architecture:** Add `imap_host` parameter to `GmailIMAPConnector`, create a thin `OutlookConnector` subclass, then extend the CLI command with `detect_token_sources()` and an interactive connect prompt that saves credentials and ingests all sources in one flow.
**Tech Stack:** Python 3.10+, imaplib, Click, Rich, SQLite/FTS5, pytest
---
### Task 1: Generalize GmailIMAPConnector with `imap_host` parameter
**Files:**
- Modify: `src/openjarvis/connectors/gmail_imap.py`
- Modify: `tests/connectors/test_gmail_imap.py`
- [ ] **Step 1: Read the current test file to understand existing test patterns**
```bash
cat tests/connectors/test_gmail_imap.py
```
- [ ] **Step 2: Add `imap_host` parameter to `GmailIMAPConnector.__init__`**
In `src/openjarvis/connectors/gmail_imap.py`, add a class attribute and constructor parameter:
```python
# Add class attribute after line 84 (after display_name):
_default_imap_host = "imap.gmail.com"
# Change __init__ signature to add imap_host:
def __init__(
self,
email_address: str = "",
app_password: str = "",
credentials_path: str = "",
*,
imap_host: str = "",
max_messages: int = 500,
) -> None:
self._email = email_address
self._password = app_password
self._credentials_path = credentials_path or _DEFAULT_CREDENTIALS_PATH
self._imap_host = imap_host or self._default_imap_host
self._max_messages = max_messages
self._items_synced = 0
self._items_total = 0
```
- [ ] **Step 3: Update `sync()` to use `self._imap_host`**
In `src/openjarvis/connectors/gmail_imap.py`, line 147, change:
```python
# Before:
imap = imaplib.IMAP4_SSL("imap.gmail.com")
# After:
imap = imaplib.IMAP4_SSL(self._imap_host)
```
- [ ] **Step 4: Run existing tests to verify nothing breaks**
```bash
uv run pytest tests/connectors/test_gmail_imap.py -v --tb=short
```
Expected: All existing tests PASS (they mock IMAP so the host doesn't matter).
- [ ] **Step 5: Commit**
```bash
git add src/openjarvis/connectors/gmail_imap.py
git commit -m "feat: add imap_host parameter to GmailIMAPConnector"
```
---
### Task 2: Create Outlook connector
**Files:**
- Create: `src/openjarvis/connectors/outlook.py`
- Create: `tests/connectors/test_outlook.py`
- [ ] **Step 1: Write the test file**
Create `tests/connectors/test_outlook.py`:
```python
"""Tests for OutlookConnector — thin subclass of GmailIMAPConnector."""
from __future__ import annotations
from pathlib import Path
import pytest
from openjarvis.core.registry import ConnectorRegistry
def test_outlook_registered() -> None:
"""OutlookConnector is discoverable via ConnectorRegistry."""
from openjarvis.connectors.outlook import OutlookConnector
ConnectorRegistry.register_value("outlook", OutlookConnector)
assert ConnectorRegistry.contains("outlook")
cls = ConnectorRegistry.get("outlook")
assert cls.connector_id == "outlook"
assert cls.display_name == "Outlook / Microsoft 365"
def test_outlook_uses_correct_imap_host() -> None:
"""OutlookConnector defaults to outlook.office365.com."""
from openjarvis.connectors.outlook import OutlookConnector
conn = OutlookConnector()
assert conn._imap_host == "outlook.office365.com"
def test_outlook_auth_url() -> None:
"""auth_url() points to Microsoft security page."""
from openjarvis.connectors.outlook import OutlookConnector
conn = OutlookConnector()
assert "microsoft.com" in conn.auth_url()
def test_outlook_handle_callback(tmp_path: Path) -> None:
"""handle_callback saves email:password credentials."""
from openjarvis.connectors.outlook import OutlookConnector
from openjarvis.connectors.oauth import load_tokens
creds_path = str(tmp_path / "outlook.json")
conn = OutlookConnector(credentials_path=creds_path)
conn.handle_callback("user@outlook.com:mypassword123")
tokens = load_tokens(creds_path)
assert tokens is not None
assert tokens["email"] == "user@outlook.com"
assert tokens["password"] == "mypassword123"
def test_outlook_is_connected(tmp_path: Path) -> None:
"""is_connected returns True when credentials file has email+password."""
from openjarvis.connectors.outlook import OutlookConnector
creds_path = str(tmp_path / "outlook.json")
conn = OutlookConnector(credentials_path=creds_path)
assert conn.is_connected() is False
conn.handle_callback("user@outlook.com:pass")
assert conn.is_connected() is True
def test_outlook_sync_source_is_outlook(tmp_path: Path) -> None:
"""Documents yielded by sync() have source='outlook' and doc_id prefix 'outlook:'."""
from unittest.mock import MagicMock, patch
from openjarvis.connectors.outlook import OutlookConnector
creds_path = str(tmp_path / "outlook.json")
conn = OutlookConnector(credentials_path=creds_path)
conn.handle_callback("user@outlook.com:pass")
# Mock IMAP to return one email
mock_imap = MagicMock()
mock_imap.login.return_value = ("OK", [])
mock_imap.select.return_value = ("OK", [])
mock_imap.search.return_value = ("OK", [b"1"])
raw_email = (
b"From: sender@test.com\r\n"
b"To: user@outlook.com\r\n"
b"Subject: Test Email\r\n"
b"Date: Mon, 01 Jan 2024 00:00:00 +0000\r\n"
b"Message-ID: <test123@test.com>\r\n"
b"\r\n"
b"Hello from Outlook test"
)
mock_imap.fetch.return_value = ("OK", [(b"1", raw_email)])
mock_imap.logout.return_value = ("OK", [])
with patch("openjarvis.connectors.gmail_imap.imaplib") as mock_imaplib:
mock_imaplib.IMAP4_SSL.return_value = mock_imap
mock_imaplib.IMAP4 = type(mock_imap)
docs = list(conn.sync())
assert len(docs) == 1
assert docs[0].source == "outlook"
assert docs[0].doc_id.startswith("outlook:")
assert docs[0].title == "Test Email"
```
- [ ] **Step 2: Run tests to verify they fail**
```bash
uv run pytest tests/connectors/test_outlook.py -v --tb=short
```
Expected: FAIL — `ModuleNotFoundError: No module named 'openjarvis.connectors.outlook'`
- [ ] **Step 3: Create the Outlook connector**
Create `src/openjarvis/connectors/outlook.py`:
```python
"""Outlook / Microsoft 365 connector — reads email via IMAP with app password.
Thin subclass of GmailIMAPConnector that defaults to the Outlook IMAP host
and relabels documents with source='outlook'.
Setup: enable 2FA on your Microsoft account, then generate an app password
at https://account.microsoft.com/security
"""
from __future__ import annotations
from typing import Iterator, Optional
from datetime import datetime
from openjarvis.connectors._stubs import Document
from openjarvis.connectors.gmail_imap import GmailIMAPConnector
from openjarvis.core.config import DEFAULT_CONFIG_DIR
from openjarvis.core.registry import ConnectorRegistry
_DEFAULT_CREDENTIALS_PATH = str(
DEFAULT_CONFIG_DIR / "connectors" / "outlook.json"
)
@ConnectorRegistry.register("outlook")
class OutlookConnector(GmailIMAPConnector):
"""Outlook connector using IMAP + app password.
Inherits all IMAP logic from :class:`GmailIMAPConnector` and overrides
the IMAP host, credential path, auth URL, and document source labels.
"""
connector_id = "outlook"
display_name = "Outlook / Microsoft 365"
_default_imap_host = "outlook.office365.com"
def __init__(
self,
email_address: str = "",
app_password: str = "",
credentials_path: str = "",
*,
max_messages: int = 500,
) -> None:
super().__init__(
email_address,
app_password,
credentials_path or _DEFAULT_CREDENTIALS_PATH,
max_messages=max_messages,
)
def auth_url(self) -> str:
"""Return the Microsoft account security page for app passwords."""
return "https://account.microsoft.com/security"
def sync(
self,
*,
since: Optional[datetime] = None,
cursor: Optional[str] = None,
) -> Iterator[Document]:
"""Sync emails and relabel with source='outlook'."""
for doc in super().sync(since=since, cursor=cursor):
doc.source = "outlook"
doc.doc_id = doc.doc_id.replace("gmail:", "outlook:", 1)
yield doc
```
- [ ] **Step 4: Run tests to verify they pass**
```bash
uv run pytest tests/connectors/test_outlook.py -v --tb=short
```
Expected: All 6 tests PASS.
- [ ] **Step 5: Commit**
```bash
git add src/openjarvis/connectors/outlook.py tests/connectors/test_outlook.py
git commit -m "feat: add Outlook connector as IMAP subclass"
```
---
### Task 3: Extend deep-research-setup with token source detection and interactive connect
**Files:**
- Modify: `src/openjarvis/cli/deep_research_setup_cmd.py`
- Modify: `tests/cli/test_deep_research_setup.py`
- [ ] **Step 1: Write tests for token source detection**
Append to `tests/cli/test_deep_research_setup.py`:
```python
def test_detect_token_sources_finds_connected(tmp_path: Path) -> None:
"""detect_token_sources finds sources with valid credential files."""
from openjarvis.cli.deep_research_setup_cmd import detect_token_sources
connectors_dir = tmp_path / "connectors"
connectors_dir.mkdir()
(connectors_dir / "slack.json").write_text('{"token": "xoxb-test"}')
(connectors_dir / "notion.json").write_text('{"token": "ntn_test"}')
sources = detect_token_sources(connectors_dir=connectors_dir)
ids = [s["connector_id"] for s in sources]
assert "slack" in ids
assert "notion" in ids
def test_detect_token_sources_skips_empty(tmp_path: Path) -> None:
"""detect_token_sources skips files with empty or invalid JSON."""
from openjarvis.cli.deep_research_setup_cmd import detect_token_sources
connectors_dir = tmp_path / "connectors"
connectors_dir.mkdir()
(connectors_dir / "slack.json").write_text("{}")
(connectors_dir / "notion.json").write_text("invalid json")
sources = detect_token_sources(connectors_dir=connectors_dir)
assert len(sources) == 0
def test_detect_token_sources_empty_dir(tmp_path: Path) -> None:
"""detect_token_sources returns empty list when no credential files exist."""
from openjarvis.cli.deep_research_setup_cmd import detect_token_sources
connectors_dir = tmp_path / "connectors"
connectors_dir.mkdir()
sources = detect_token_sources(connectors_dir=connectors_dir)
assert len(sources) == 0
```
- [ ] **Step 2: Run tests to verify they fail**
```bash
uv run pytest tests/cli/test_deep_research_setup.py::test_detect_token_sources_finds_connected -v --tb=short
```
Expected: FAIL — `ImportError: cannot import name 'detect_token_sources'`
- [ ] **Step 3: Add `detect_token_sources` and `_TOKEN_SOURCES` to deep_research_setup_cmd.py**
Add after the `detect_local_sources` function (around line 80):
```python
# ---------------------------------------------------------------------------
# Token-based source detection
# ---------------------------------------------------------------------------
_TOKEN_SOURCES = [
{
"connector_id": "gmail_imap",
"display_name": "Gmail (IMAP)",
"creds_file": "gmail_imap.json",
"prompt_label": "email:app_password",
},
{
"connector_id": "outlook",
"display_name": "Outlook / Microsoft 365",
"creds_file": "outlook.json",
"prompt_label": "email:app_password",
},
{
"connector_id": "slack",
"display_name": "Slack",
"creds_file": "slack.json",
"prompt_label": "Bot token (xoxb-... or xoxe-...)",
},
{
"connector_id": "notion",
"display_name": "Notion",
"creds_file": "notion.json",
"prompt_label": "Integration token (ntn_...)",
},
{
"connector_id": "granola",
"display_name": "Granola",
"creds_file": "granola.json",
"prompt_label": "API key (grn_...)",
},
]
def detect_token_sources(
*,
connectors_dir: Optional[Path] = None,
) -> List[Dict[str, Any]]:
"""Return token-based sources that already have valid credentials.
Scans ``~/.openjarvis/connectors/`` for known credential files.
"""
cdir = connectors_dir or (DEFAULT_CONFIG_DIR / "connectors")
sources: List[Dict[str, Any]] = []
for ts in _TOKEN_SOURCES:
creds_file = cdir / ts["creds_file"]
if not creds_file.exists():
continue
try:
data = json.loads(creds_file.read_text(encoding="utf-8"))
except (json.JSONDecodeError, OSError):
continue
# Must have at least one non-empty value
if not data or not any(v for v in data.values() if v):
continue
sources.append({
"connector_id": ts["connector_id"],
"display_name": ts["display_name"],
"config": {},
})
return sources
```
Also add `import json` at the top of the file (add to existing imports section).
- [ ] **Step 4: Run token detection tests**
```bash
uv run pytest tests/cli/test_deep_research_setup.py -v --tb=short
```
Expected: All 7 tests PASS (4 old + 3 new).
- [ ] **Step 5: Add `_prompt_connect_sources` function and update `_instantiate_connector`**
Add the interactive connect function after `detect_token_sources`:
```python
def _prompt_connect_sources(console: Console) -> List[Dict[str, Any]]:
"""Interactively prompt the user to connect token-based sources."""
connected: List[Dict[str, Any]] = []
cdir = DEFAULT_CONFIG_DIR / "connectors"
cdir.mkdir(parents=True, exist_ok=True)
while True:
# Show unconnected sources
unconnected = [
ts for ts in _TOKEN_SOURCES
if not (cdir / ts["creds_file"]).exists()
]
if not unconnected:
console.print("[dim]All token sources already connected.[/dim]")
break
if not click.confirm("Connect additional sources?", default=False):
break
names = [ts["connector_id"] for ts in unconnected]
labels = [f"{ts['display_name']} ({ts['connector_id']})" for ts in unconnected]
console.print("Available:")
for label in labels:
console.print(f" {label}")
choice = click.prompt(
"Which source?",
type=click.Choice(names, case_sensitive=False),
)
ts = next(t for t in unconnected if t["connector_id"] == choice)
token = click.prompt(f"Paste your {ts['prompt_label']}")
# Save credentials via connector's handle_callback
connector = _instantiate_connector(choice, {})
connector.handle_callback(token.strip())
console.print(f" [green]{ts['display_name']}: connected![/green]")
connected.append({
"connector_id": choice,
"display_name": ts["display_name"],
"config": {},
})
return connected
```
Update `_instantiate_connector` to handle all connector types:
```python
def _instantiate_connector(connector_id: str, config: Dict[str, Any]) -> Any:
"""Lazily import and instantiate a connector by ID."""
if connector_id == "apple_notes":
from openjarvis.connectors.apple_notes import AppleNotesConnector
return AppleNotesConnector(db_path=config.get("db_path", ""))
elif connector_id == "imessage":
from openjarvis.connectors.imessage import IMessageConnector
return IMessageConnector(db_path=config.get("db_path", ""))
elif connector_id == "obsidian":
from openjarvis.connectors.obsidian import ObsidianConnector
return ObsidianConnector(vault_path=config.get("vault_path", ""))
elif connector_id == "gmail_imap":
from openjarvis.connectors.gmail_imap import GmailIMAPConnector
return GmailIMAPConnector()
elif connector_id == "outlook":
from openjarvis.connectors.outlook import OutlookConnector
return OutlookConnector()
elif connector_id == "slack":
from openjarvis.connectors.slack_connector import SlackConnector
return SlackConnector()
elif connector_id == "notion":
from openjarvis.connectors.notion import NotionConnector
return NotionConnector()
elif connector_id == "granola":
from openjarvis.connectors.granola import GranolaConnector
return GranolaConnector()
else:
msg = f"Unknown connector: {connector_id}"
raise ValueError(msg)
```
- [ ] **Step 6: Update the `deep_research_setup` Click command to use token sources**
Replace the body of the `deep_research_setup` function with:
```python
@click.command("deep-research-setup")
@click.option(
"--obsidian-vault",
type=click.Path(exists=True, file_okay=False),
default=None,
help="Path to an Obsidian vault directory.",
)
@click.option("--skip-chat", is_flag=True, help="Ingest only, don't launch chat.")
def deep_research_setup(obsidian_vault: Optional[str], skip_chat: bool) -> None:
"""Auto-detect local data sources, ingest, and launch Deep Research chat."""
console = Console()
console.print("\n[bold]Deep Research Setup[/bold]\n")
# 1. Detect local sources
vault_path = Path(obsidian_vault) if obsidian_vault else None
local_sources = detect_local_sources(obsidian_vault_path=vault_path)
# 2. Detect already-connected token sources
token_sources = detect_token_sources()
all_sources = local_sources + token_sources
# 3. Show what we found
if all_sources:
table = Table(title="Detected Sources")
table.add_column("Source", style="bold")
table.add_column("Type", style="dim")
table.add_column("Status", style="green")
for src in local_sources:
table.add_row(src["display_name"], "local", "ready")
for src in token_sources:
table.add_row(src["display_name"], "token", "connected")
console.print(table)
console.print()
# 4. Offer to connect new token sources
newly_connected = _prompt_connect_sources(console)
all_sources.extend(newly_connected)
if not all_sources:
console.print(
"[yellow]No data sources detected or connected.[/yellow]\n"
"On macOS, ensure Full Disk Access is granted in "
"System Settings > Privacy & Security."
)
sys.exit(1)
# 5. Confirm and ingest
if not click.confirm("Ingest these sources?", default=True):
sys.exit(0)
db_path = DEFAULT_CONFIG_DIR / "knowledge.db"
db_path.parent.mkdir(parents=True, exist_ok=True)
store = KnowledgeStore(str(db_path))
console.print("\n[bold]Ingesting...[/bold]")
for src in all_sources:
try:
connector = _instantiate_connector(
src["connector_id"], src["config"],
)
pipeline = IngestionPipeline(store)
engine = SyncEngine(pipeline)
chunks = engine.sync(connector)
console.print(
f" {src['display_name']}: [green]{chunks} chunks[/green]"
)
except Exception as exc: # noqa: BLE001
console.print(
f" {src['display_name']}: [red]error: {exc}[/red]"
)
total = store.count()
console.print(
f"\n[bold green]Done![/bold green] {total} total chunks in {db_path}\n"
)
# 6. Chat
if skip_chat:
return
_launch_chat(store, console)
```
- [ ] **Step 7: Run all tests**
```bash
uv run pytest tests/cli/test_deep_research_setup.py tests/connectors/test_outlook.py -v --tb=short
```
Expected: All tests PASS.
- [ ] **Step 8: Lint**
```bash
uv run ruff check src/openjarvis/cli/deep_research_setup_cmd.py src/openjarvis/connectors/outlook.py src/openjarvis/connectors/gmail_imap.py
```
Expected: No errors.
- [ ] **Step 9: Commit**
```bash
git add src/openjarvis/cli/deep_research_setup_cmd.py tests/cli/test_deep_research_setup.py
git commit -m "feat: extend deep-research-setup with token source detection and interactive connect"
```
---
### Task 4: Live test — connect all sources and ingest
**Files:** None (manual testing)
- [ ] **Step 1: Connect Slack**
```bash
uv run jarvis connect slack
```
Paste: your Slack bot token (`xoxb-...` or `xoxe-...`)
- [ ] **Step 2: Connect Notion**
```bash
uv run jarvis connect notion
```
Paste: your Notion integration token (`ntn_...`)
- [ ] **Step 3: Connect Granola**
```bash
uv run jarvis connect granola
```
Paste: your Granola API key (`grn_...`)
- [ ] **Step 4: Connect Gmail IMAP**
```bash
uv run jarvis connect gmail_imap
```
Paste: `your-email@gmail.com:qpde kebj evhy zljc`
(Replace `your-email@gmail.com` with your actual Gmail address.)
- [ ] **Step 5: Run deep-research-setup and verify all sources ingest**
```bash
uv run jarvis deep-research-setup --skip-chat
```
Expected output:
```
Deep Research Setup
Detected Sources
┌───────────────────────┬───────┬───────────┐
│ Source │ Type │ Status │
├───────────────────────┼───────┼───────────┤
│ Apple Notes │ local │ ready │
│ iMessage │ local │ ready │
│ Gmail (IMAP) │ token │ connected │
│ Slack │ token │ connected │
│ Notion │ token │ connected │
│ Granola │ token │ connected │
└───────────────────────┴───────┴───────────┘
Ingest these sources? [Y/n]: Y
Ingesting...
Apple Notes: ~100 chunks
iMessage: ~52000 chunks
Gmail (IMAP): N chunks
Slack: N chunks
Notion: N chunks
Granola: N chunks
Done! N total chunks in ~/.openjarvis/knowledge.db
```
- [ ] **Step 6: Test the research agent with cross-source queries**
```bash
uv run jarvis deep-research-setup
```
At the `research>` prompt, try:
1. A query that should span emails + messages
2. A query about meeting notes (Granola data)
3. A query about a Notion page
- [ ] **Step 7: Push**
```bash
git push origin feat/deep-research-setup
```
docs/superpowers/plans/2026-03-27-channel-gateway-deep-research.md
-722
View File
@@ -1,722 +0,0 @@
# Channel Gateway → DeepResearch Wiring Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Wire the ChannelBridge (from PR #78, now merged) to route incoming messages from Slack, iMessage/BlueBubbles, WhatsApp, and SMS to the DeepResearchAgent, and add an AppleScript-based iMessage daemon for direct iPhone-to-agent messaging without external services.
**Architecture:** Modify `ChannelBridge.handle_incoming()` to use DeepResearchAgent instead of generic `JarvisSystem.ask()`. Add an `IMessageDaemon` that polls `chat.db` for new messages and routes them through the bridge. Add `jarvis channels` CLI commands for setup and lifecycle.
**Tech Stack:** Python 3.10+, sqlite3, subprocess (AppleScript), Click, asyncio, pytest
---
### Task 1: Wire ChannelBridge to DeepResearchAgent
**Files:**
- Modify: `src/openjarvis/server/channel_bridge.py`
- Create: `tests/server/test_channel_bridge_deep_research.py`
- [ ] **Step 1: Read `channel_bridge.py` to find `_handle_chat` method**
The `_handle_chat` method currently calls `self._system.ask()`. We need it to use DeepResearchAgent when available.
- [ ] **Step 2: Write the test**
Create `tests/server/test_channel_bridge_deep_research.py`:
```python
"""Test ChannelBridge routes to DeepResearchAgent."""
from __future__ import annotations
from unittest.mock import MagicMock, patch
import pytest
def test_handle_chat_uses_deep_research_agent() -> None:
"""When a DeepResearch agent is configured, route through it."""
from openjarvis.server.channel_bridge import ChannelBridge
from openjarvis.server.session_store import SessionStore
mock_agent = MagicMock()
mock_agent.run.return_value = MagicMock(
content="Found 3 results about Spain.",
)
bridge = ChannelBridge(
channels={},
session_store=SessionStore(db_path=":memory:"),
bus=MagicMock(),
deep_research_agent=mock_agent,
)
result = bridge.handle_incoming(
sender_id="+15551234567",
content="When was my last trip to Spain?",
channel_type="twilio",
)
assert "Spain" in result
mock_agent.run.assert_called_once()
def test_handle_chat_falls_back_to_system() -> None:
"""When no DeepResearch agent, fall back to system.ask()."""
from openjarvis.server.channel_bridge import ChannelBridge
from openjarvis.server.session_store import SessionStore
mock_system = MagicMock()
mock_system.ask.return_value = "Generic response"
bridge = ChannelBridge(
channels={},
session_store=SessionStore(db_path=":memory:"),
bus=MagicMock(),
system=mock_system,
)
result = bridge.handle_incoming(
sender_id="+15551234567",
content="Hello",
channel_type="twilio",
)
assert result == "Generic response"
mock_system.ask.assert_called_once()
```
- [ ] **Step 3: Modify `ChannelBridge.__init__` to accept `deep_research_agent`**
In `channel_bridge.py`, add `deep_research_agent` parameter to `__init__`:
```python
def __init__(
self,
channels: Dict[str, BaseChannel],
session_store: SessionStore,
bus: EventBus,
system: Any = None,
agent_manager: Any = None,
deep_research_agent: Any = None,
) -> None:
self._channels = channels
self._session_store = session_store
self._bus = bus
self._system = system
self._agent_manager = agent_manager
self._deep_research_agent = deep_research_agent
self._notification_timestamps: Dict[str, float] = {}
self._subscribe_notifications()
```
- [ ] **Step 4: Modify `_handle_chat` to use DeepResearchAgent first**
Find the `_handle_chat` method. Replace the body to try DeepResearchAgent first, fall back to system:
```python
def _handle_chat(
self,
sender_id: str,
content: str,
channel_type: str,
max_length: int = _DEFAULT_MAX_LENGTH,
) -> str:
"""Route chat to DeepResearchAgent or JarvisSystem."""
# Try DeepResearchAgent first
if self._deep_research_agent is not None:
try:
result = self._deep_research_agent.run(content)
response = result.content or "No results found."
except Exception as exc:
logger.error("DeepResearch agent failed: %s", exc)
response = f"Research error: {exc}"
elif self._system is not None:
response = self._system.ask(content)
else:
response = "No agent or system configured."
# Truncate if needed
if len(response) > max_length:
self._session_store.set_overflow(sender_id, response)
response = response[:max_length] + "\n\n(truncated — send /more for the rest)"
return response
```
- [ ] **Step 5: Run tests**
```bash
uv run pytest tests/server/test_channel_bridge_deep_research.py tests/server/test_channel_bridge.py -v --tb=short
```
Expected: All PASS (new tests + existing tests).
- [ ] **Step 6: Commit**
```bash
git add src/openjarvis/server/channel_bridge.py tests/server/test_channel_bridge_deep_research.py
git commit -m "feat: wire ChannelBridge to route messages to DeepResearchAgent"
```
---
### Task 2: Create iMessage AppleScript daemon
**Files:**
- Create: `src/openjarvis/channels/imessage_daemon.py`
- Create: `tests/channels/test_imessage_daemon.py`
- [ ] **Step 1: Write the test**
Create `tests/channels/test_imessage_daemon.py`:
```python
"""Tests for iMessage AppleScript daemon."""
from __future__ import annotations
import sqlite3
from pathlib import Path
from unittest.mock import MagicMock, patch
import pytest
def _create_fake_chat_db(db_path: Path) -> None:
"""Create a minimal chat.db with one message."""
conn = sqlite3.connect(str(db_path))
conn.executescript("""
CREATE TABLE handle (ROWID INTEGER PRIMARY KEY, id TEXT);
CREATE TABLE chat (
ROWID INTEGER PRIMARY KEY,
chat_identifier TEXT, display_name TEXT
);
CREATE TABLE chat_message_join (chat_id INTEGER, message_id INTEGER);
CREATE TABLE message (
ROWID INTEGER PRIMARY KEY, text TEXT, handle_id INTEGER,
date INTEGER, is_from_me INTEGER
);
""")
conn.execute("INSERT INTO handle VALUES (1, '+15551234567')")
conn.execute("INSERT INTO chat VALUES (1, '+15551234567', 'Test Chat')")
conn.execute("INSERT INTO chat_message_join VALUES (1, 1)")
conn.execute(
"INSERT INTO message VALUES (1, 'Hello agent', 1, 700000000000000000, 0)"
)
conn.commit()
conn.close()
def test_poll_new_messages(tmp_path: Path) -> None:
"""Daemon detects new messages after last_rowid."""
from openjarvis.channels.imessage_daemon import poll_new_messages
db_path = tmp_path / "chat.db"
_create_fake_chat_db(db_path)
messages = poll_new_messages(
db_path=str(db_path),
last_rowid=0,
chat_identifier="+15551234567",
)
assert len(messages) == 1
assert messages[0]["text"] == "Hello agent"
assert messages[0]["rowid"] == 1
def test_poll_skips_old_messages(tmp_path: Path) -> None:
"""Daemon skips messages with ROWID <= last_rowid."""
from openjarvis.channels.imessage_daemon import poll_new_messages
db_path = tmp_path / "chat.db"
_create_fake_chat_db(db_path)
messages = poll_new_messages(
db_path=str(db_path),
last_rowid=1,
chat_identifier="+15551234567",
)
assert len(messages) == 0
def test_poll_filters_by_chat(tmp_path: Path) -> None:
"""Daemon only returns messages from the designated chat."""
from openjarvis.channels.imessage_daemon import poll_new_messages
db_path = tmp_path / "chat.db"
_create_fake_chat_db(db_path)
messages = poll_new_messages(
db_path=str(db_path),
last_rowid=0,
chat_identifier="+15559999999",
)
assert len(messages) == 0
def test_poll_skips_own_messages(tmp_path: Path) -> None:
"""Daemon ignores messages sent by the user (is_from_me=1)."""
from openjarvis.channels.imessage_daemon import poll_new_messages
db_path = tmp_path / "chat.db"
conn = sqlite3.connect(str(db_path))
conn.executescript("""
CREATE TABLE handle (ROWID INTEGER PRIMARY KEY, id TEXT);
CREATE TABLE chat (
ROWID INTEGER PRIMARY KEY,
chat_identifier TEXT, display_name TEXT
);
CREATE TABLE chat_message_join (chat_id INTEGER, message_id INTEGER);
CREATE TABLE message (
ROWID INTEGER PRIMARY KEY, text TEXT, handle_id INTEGER,
date INTEGER, is_from_me INTEGER
);
""")
conn.execute("INSERT INTO handle VALUES (1, '+15551234567')")
conn.execute("INSERT INTO chat VALUES (1, '+15551234567', 'Test')")
conn.execute("INSERT INTO chat_message_join VALUES (1, 1)")
conn.execute(
"INSERT INTO message VALUES (1, 'My own msg', 1, 700000000000000000, 1)"
)
conn.commit()
conn.close()
messages = poll_new_messages(
db_path=str(db_path),
last_rowid=0,
chat_identifier="+15551234567",
)
assert len(messages) == 0
```
- [ ] **Step 2: Run tests to verify they fail**
```bash
uv run pytest tests/channels/test_imessage_daemon.py -v --tb=short
```
Expected: FAIL — `ModuleNotFoundError`
- [ ] **Step 3: Create the daemon module**
Create `src/openjarvis/channels/imessage_daemon.py`:
```python
"""iMessage daemon — polls chat.db and routes to DeepResearchAgent.
Monitors a designated iMessage conversation for new messages, routes
them to the agent, and sends responses back via AppleScript.
Requires macOS with Full Disk Access for chat.db reading and
Accessibility permission for AppleScript Messages control.
"""
from __future__ import annotations
import logging
import os
import signal
import sqlite3
import subprocess
import time
from pathlib import Path
from typing import Any, Dict, List, Optional
logger = logging.getLogger(__name__)
_DEFAULT_DB_PATH = str(
Path.home() / "Library" / "Messages" / "chat.db"
)
_POLL_INTERVAL = 5 # seconds
_PID_FILE = str(Path.home() / ".openjarvis" / "imessage-agent.pid")
# -------------------------------------------------------------------
# Polling
# -------------------------------------------------------------------
def poll_new_messages(
*,
db_path: str = _DEFAULT_DB_PATH,
last_rowid: int = 0,
chat_identifier: str = "",
) -> List[Dict[str, Any]]:
"""Return new incoming messages since last_rowid.
Only returns messages where ``is_from_me = 0`` (incoming) and
that belong to the designated chat.
"""
try:
conn = sqlite3.connect(f"file:{db_path}?mode=ro", uri=True)
conn.row_factory = sqlite3.Row
except sqlite3.OperationalError:
return []
try:
rows = conn.execute(
"SELECT m.ROWID as rowid, m.text, m.date, c.chat_identifier "
"FROM message m "
"JOIN chat_message_join cmj ON cmj.message_id = m.ROWID "
"JOIN chat c ON c.ROWID = cmj.chat_id "
"WHERE m.ROWID > ? AND m.is_from_me = 0 AND m.text IS NOT NULL "
"AND c.chat_identifier = ? "
"ORDER BY m.ROWID ASC",
(last_rowid, chat_identifier),
).fetchall()
return [dict(row) for row in rows]
finally:
conn.close()
# -------------------------------------------------------------------
# AppleScript sending
# -------------------------------------------------------------------
def send_imessage(chat_identifier: str, message: str) -> bool:
"""Send an iMessage via AppleScript.
Returns True on success, False on failure.
"""
# Escape double quotes and backslashes for AppleScript
escaped = message.replace("\\", "\\\\").replace('"', '\\"')
script = (
f'tell application "Messages"\n'
f' set targetChat to a reference to chat id "{chat_identifier}"\n'
f' send "{escaped}" to targetChat\n'
f"end tell"
)
try:
subprocess.run(
["osascript", "-e", script],
capture_output=True,
text=True,
timeout=30,
)
return True
except (subprocess.TimeoutExpired, FileNotFoundError):
logger.error("Failed to send iMessage via AppleScript")
return False
# -------------------------------------------------------------------
# Daemon loop
# -------------------------------------------------------------------
def run_daemon(
*,
chat_identifier: str,
db_path: str = _DEFAULT_DB_PATH,
handler: Any = None,
poll_interval: float = _POLL_INTERVAL,
max_iterations: int = 0,
) -> None:
"""Run the iMessage polling daemon.
Parameters
----------
chat_identifier:
The chat to monitor (phone number or email).
handler:
Callable that takes a message string and returns a response string.
If None, messages are logged but not responded to.
poll_interval:
Seconds between polls.
max_iterations:
Stop after this many iterations (0 = run forever).
"""
# Write PID file
pid_path = Path(_PID_FILE)
pid_path.parent.mkdir(parents=True, exist_ok=True)
pid_path.write_text(str(os.getpid()))
# Find starting ROWID
last_rowid = _get_max_rowid(db_path)
logger.info(
"iMessage daemon started — monitoring %s from ROWID %d",
chat_identifier,
last_rowid,
)
running = True
def _stop(signum: int, frame: Any) -> None:
nonlocal running
running = False
signal.signal(signal.SIGTERM, _stop)
signal.signal(signal.SIGINT, _stop)
iterations = 0
while running:
messages = poll_new_messages(
db_path=db_path,
last_rowid=last_rowid,
chat_identifier=chat_identifier,
)
for msg in messages:
last_rowid = msg["rowid"]
text = msg["text"]
logger.info("Received: %s", text[:100])
if handler is not None:
try:
response = handler(text)
if response:
send_imessage(chat_identifier, response)
except Exception:
logger.exception("Handler failed for message %d", msg["rowid"])
iterations += 1
if max_iterations and iterations >= max_iterations:
break
time.sleep(poll_interval)
# Cleanup PID file
if pid_path.exists():
pid_path.unlink()
logger.info("iMessage daemon stopped")
def _get_max_rowid(db_path: str) -> int:
"""Get the current max ROWID from chat.db."""
try:
conn = sqlite3.connect(f"file:{db_path}?mode=ro", uri=True)
row = conn.execute("SELECT MAX(ROWID) FROM message").fetchone()
conn.close()
return row[0] or 0
except sqlite3.OperationalError:
return 0
def is_running() -> bool:
"""Check if the daemon is currently running."""
pid_path = Path(_PID_FILE)
if not pid_path.exists():
return False
try:
pid = int(pid_path.read_text().strip())
os.kill(pid, 0)
return True
except (ValueError, ProcessLookupError, PermissionError):
pid_path.unlink(missing_ok=True)
return False
def stop_daemon() -> bool:
"""Stop the running daemon. Returns True if stopped."""
pid_path = Path(_PID_FILE)
if not pid_path.exists():
return False
try:
pid = int(pid_path.read_text().strip())
os.kill(pid, signal.SIGTERM)
pid_path.unlink(missing_ok=True)
return True
except (ValueError, ProcessLookupError, PermissionError):
pid_path.unlink(missing_ok=True)
return False
```
- [ ] **Step 4: Run tests**
```bash
uv run pytest tests/channels/test_imessage_daemon.py -v --tb=short
```
Expected: 4/4 PASS.
- [ ] **Step 5: Lint + commit**
```bash
uv run ruff check src/openjarvis/channels/imessage_daemon.py
git add src/openjarvis/channels/imessage_daemon.py tests/channels/test_imessage_daemon.py
git commit -m "feat: add iMessage AppleScript daemon for iPhone-to-agent messaging"
```
---
### Task 3: Add `jarvis channels` CLI commands
**Files:**
- Create: `src/openjarvis/cli/channels_cmd.py`
- Modify: `src/openjarvis/cli/__init__.py`
- [ ] **Step 1: Create the CLI command module**
Create `src/openjarvis/cli/channels_cmd.py`:
```python
"""``jarvis channels`` — manage channel connections for the DeepResearch agent."""
from __future__ import annotations
import sys
from typing import Optional
import click
from rich.console import Console
from rich.table import Table
@click.group("channels")
def channels() -> None:
"""Manage messaging channels (iMessage, Slack, WhatsApp, SMS)."""
@channels.command("status")
def channels_status() -> None:
"""Show status of all configured channels."""
from openjarvis.channels.imessage_daemon import is_running
console = Console()
table = Table(title="Channel Status")
table.add_column("Channel", style="bold")
table.add_column("Status")
table.add_column("Details", style="dim")
# iMessage daemon
if is_running():
table.add_row("iMessage", "[green]running[/green]", "Polling chat.db")
else:
table.add_row("iMessage", "[dim]stopped[/dim]", "jarvis channels imessage-start")
console.print(table)
@channels.command("imessage-start")
@click.argument("chat_identifier")
@click.option("--background/--foreground", default=True, help="Run in background.")
def imessage_start(chat_identifier: str, background: bool) -> None:
"""Start the iMessage daemon for a chat.
CHAT_IDENTIFIER is the phone number or email to monitor
(e.g. +15551234567 or group chat name).
"""
from openjarvis.channels.imessage_daemon import is_running, run_daemon
console = Console()
if is_running():
console.print("[yellow]iMessage daemon is already running.[/yellow]")
return
if background:
import subprocess
proc = subprocess.Popen(
[
sys.executable, "-m", "openjarvis.channels.imessage_daemon",
"--chat", chat_identifier,
],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
start_new_session=True,
)
console.print(
f"[green]iMessage daemon started[/green] (PID {proc.pid})\n"
f"Monitoring: {chat_identifier}\n"
f"Text this contact from your iPhone to chat with the agent."
)
else:
console.print(f"[green]Starting iMessage daemon[/green] — monitoring {chat_identifier}")
console.print("Press Ctrl+C to stop.\n")
from openjarvis.agents.deep_research import DeepResearchAgent
from openjarvis.connectors.retriever import TwoStageRetriever
from openjarvis.connectors.store import KnowledgeStore
from openjarvis.engine.ollama import OllamaEngine
from openjarvis.tools.knowledge_search import KnowledgeSearchTool
from openjarvis.tools.knowledge_sql import KnowledgeSQLTool
from openjarvis.tools.scan_chunks import ScanChunksTool
from openjarvis.tools.think import ThinkTool
engine = OllamaEngine()
store = KnowledgeStore()
retriever = TwoStageRetriever(store)
tools = [
KnowledgeSearchTool(retriever=retriever),
KnowledgeSQLTool(store=store),
ScanChunksTool(store=store, engine=engine, model="qwen3.5:4b"),
ThinkTool(),
]
agent = DeepResearchAgent(engine=engine, model="qwen3.5:4b", tools=tools)
def handler(text: str) -> str:
result = agent.run(text)
return result.content or "No results found."
run_daemon(
chat_identifier=chat_identifier,
handler=handler,
)
@channels.command("imessage-stop")
def imessage_stop() -> None:
"""Stop the iMessage daemon."""
from openjarvis.channels.imessage_daemon import stop_daemon
console = Console()
if stop_daemon():
console.print("[green]iMessage daemon stopped.[/green]")
else:
console.print("[dim]iMessage daemon is not running.[/dim]")
```
- [ ] **Step 2: Register in CLI __init__.py**
In `src/openjarvis/cli/__init__.py`, add:
```python
from openjarvis.cli.channels_cmd import channels
cli.add_command(channels, "channels")
```
- [ ] **Step 3: Verify**
```bash
uv run jarvis channels --help
uv run jarvis channels status
```
- [ ] **Step 4: Lint + commit**
```bash
uv run ruff check src/openjarvis/cli/channels_cmd.py
git add src/openjarvis/cli/channels_cmd.py src/openjarvis/cli/__init__.py
git commit -m "feat: add jarvis channels CLI commands for iMessage daemon lifecycle"
```
---
### Task 4: Run full test suite + push
**Files:** None
- [ ] **Step 1: Run all tests**
```bash
uv run pytest tests/connectors/ tests/agents/test_deep_research.py tests/agents/test_deep_research_integration.py tests/agents/test_channel_agent.py tests/agents/test_channel_agent_integration.py tests/cli/test_deep_research_setup.py tests/tools/test_knowledge_sql.py tests/tools/test_scan_chunks.py tests/server/test_channel_bridge.py tests/server/test_channel_bridge_deep_research.py tests/server/test_session_store.py tests/server/test_webhook_routes.py tests/server/test_auth_middleware.py tests/server/test_deep_research_tools_wiring.py tests/channels/test_twilio_sms.py tests/channels/test_imessage_daemon.py --ignore=tests/connectors/test_embedding_store.py -k "not cached_embeddings and not caches_new_embeddings" -q --tb=short
```
Expected: All PASS.
- [ ] **Step 2: Lint**
```bash
uv run ruff check src/openjarvis/channels/imessage_daemon.py src/openjarvis/server/channel_bridge.py src/openjarvis/cli/channels_cmd.py
```
- [ ] **Step 3: Push to both branches**
```bash
git push origin feat/deep-research-setup
git push origin feat/deep-research-setup:feat/mobile-channel-gateway --force-with-lease
```
docs/superpowers/plans/2026-03-27-channels-and-messaging-tabs-v2.md
-672
View File
@@ -1,672 +0,0 @@
# Channels + Messaging Tabs Implementation Plan (v2)
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Replace the existing Channels tab with two tabs — **Channels** (data sources with chunk counts and inline "+ Add" setup) and **Messaging** (clear "Text this number from your iPhone" instructions) — on every agent detail page.
**Architecture:** Refactor the existing `ChannelsTab` component in `AgentsPage.tsx` into two separate components. The Channels tab calls existing connector API endpoints + a new chunk-count endpoint. The Messaging tab reuses the existing channel binding API. The `StepByStepPanel` from `SourceConnectFlow.tsx` is extracted for reuse in the Channels tab inline setup.
**Tech Stack:** React 19, TypeScript, existing FastAPI endpoints
---
### Task 1: Add connector chunk counts to backend API
**Files:**
- Modify: `src/openjarvis/server/connectors_router.py`
The Channels tab needs to show chunk counts per source. Add a `chunks` field to the connector summary by querying the KnowledgeStore.
- [ ] **Step 1: Modify `_connector_summary` to include chunk count**
In `src/openjarvis/server/connectors_router.py`, find the `_connector_summary` function (line ~102). Add a chunk count from the KnowledgeStore:
```python
def _connector_summary(connector_id: str, instance: Any) -> Dict[str, Any]:
"""Build the dict returned by GET /connectors."""
chunks = 0
try:
from openjarvis.connectors.store import KnowledgeStore
store = KnowledgeStore()
rows = store._conn.execute(
"SELECT COUNT(*) FROM knowledge_chunks WHERE source = ?",
(connector_id,),
).fetchone()
chunks = rows[0] if rows else 0
except Exception:
pass
return {
"connector_id": connector_id,
"display_name": getattr(instance, "display_name", connector_id),
"auth_type": getattr(instance, "auth_type", "unknown"),
"connected": instance.is_connected(),
"chunks": chunks,
}
```
- [ ] **Step 2: Verify endpoint returns chunks**
```bash
uv run python3 -c "
import httpx
r = httpx.get('http://127.0.0.1:8222/connectors', timeout=5)
for c in r.json().get('connectors', [])[:5]:
print(f'{c[\"connector_id\"]}: connected={c[\"connected\"]}, chunks={c.get(\"chunks\", \"?\")}')
"
```
- [ ] **Step 3: Lint + commit**
```bash
uv run ruff check src/openjarvis/server/connectors_router.py
git add src/openjarvis/server/connectors_router.py
git commit -m "feat: include chunk counts in connector list API response"
```
---
### Task 2: Rewrite Channels tab as data sources view
**Files:**
- Modify: `frontend/src/pages/AgentsPage.tsx`
- Modify: `frontend/src/lib/connectors-api.ts`
Replace the existing `ChannelsTab` (messaging channels) with a new `ChannelsTab` (data sources).
- [ ] **Step 1: Update ConnectorInfo type in connectors-api.ts to include chunks**
In `frontend/src/lib/connectors-api.ts`, find the `ConnectorInfo` type (or wherever it's defined) and add `chunks`:
If there's an interface in `types/connectors.ts`:
```typescript
export interface ConnectorInfo {
connector_id: string;
display_name: string;
auth_type: 'oauth' | 'local' | 'bridge' | 'filesystem';
connected: boolean;
auth_url?: string;
mcp_tools?: string[];
chunks?: number; // ADD THIS
}
```
- [ ] **Step 2: Replace the ChannelsTab component**
In `AgentsPage.tsx`, find the existing `AVAILABLE_CHANNELS` array and `ChannelsTab` component (around lines 1000-1170). Replace the entire block with a new `ChannelsTab` that shows data sources:
```typescript
import { SOURCE_CATALOG, ConnectorMeta, ConnectRequest } from '../types/connectors';
import { listConnectors, connectSource } from '../lib/connectors-api';
function ChannelsTab({ agentId }: { agentId: string }) {
const [connectors, setConnectors] = useState<
Array<{ connector_id: string; display_name: string; connected: boolean; chunks: number }>
>([]);
const [expandedId, setExpandedId] = useState<string | null>(null);
const [loading, setLoading] = useState(false);
const loadConnectors = useCallback(() => {
listConnectors()
.then((list) =>
setConnectors(
list.map((c) => ({
connector_id: c.connector_id,
display_name: c.display_name,
connected: c.connected,
chunks: (c as any).chunks || 0,
})),
),
)
.catch(() => {});
}, []);
useEffect(() => { loadConnectors(); }, [loadConnectors]);
const handleConnect = async (id: string, req: ConnectRequest) => {
setLoading(true);
try {
await connectSource(id, req);
setExpandedId(null);
loadConnectors();
} catch {
// error handling
} finally {
setLoading(false);
}
};
const connected = connectors.filter((c) => c.connected);
const notConnected = connectors.filter((c) => !c.connected);
// Merge with SOURCE_CATALOG for icons/descriptions
const getMeta = (id: string) =>
SOURCE_CATALOG.find((s) => s.connector_id === id);
const iconMap: Record<string, string> = {
gmail: '✉️', gmail_imap: '✉️', slack: '#',
imessage: '💬', gdrive: '📁', notion: '📄',
obsidian: '📁', granola: '🎙️', gcalendar: '📅',
gcontacts: '📇', outlook: '✉️', apple_notes: '🍎',
dropbox: '📦', whatsapp: '📱',
};
return (
<div style={{ padding: 16 }}>
<div style={{
color: 'var(--color-text-secondary)',
fontSize: 12, marginBottom: 12,
}}>
Data sources your agent can search
</div>
{/* Connected sources grid */}
{connected.length > 0 && (
<div style={{
display: 'grid',
gridTemplateColumns: '1fr 1fr',
gap: 8, marginBottom: 12,
}}>
{connected.map((c) => (
<div
key={c.connector_id}
style={{
background: 'var(--color-bg-secondary)',
border: '1px solid #2a5a3a',
borderRadius: 6, padding: '10px 12px',
display: 'flex', alignItems: 'center', gap: 8,
}}
>
<span>{iconMap[c.connector_id] || '🔗'}</span>
<div style={{ flex: 1 }}>
<div style={{ fontSize: 12, fontWeight: 500 }}>
{c.display_name}
</div>
<div style={{ fontSize: 10, color: '#4ade80' }}>
{c.chunks.toLocaleString()} chunks
</div>
</div>
<span style={{ color: '#4ade80', fontSize: 12 }}>✓</span>
</div>
))}
</div>
)}
{/* Not connected grid */}
{notConnected.length > 0 && (
<div style={{
display: 'grid',
gridTemplateColumns: '1fr 1fr',
gap: 8,
}}>
{notConnected.map((c) => {
const meta = getMeta(c.connector_id);
const isExpanded = expandedId === c.connector_id;
return (
<div
key={c.connector_id}
style={{
background: 'var(--color-bg-secondary)',
border: '1px dashed var(--color-border)',
borderRadius: 6, overflow: 'hidden',
opacity: isExpanded ? 1 : 0.6,
gridColumn: isExpanded ? '1 / -1' : undefined,
}}
>
<div
style={{
padding: '10px 12px', display: 'flex',
alignItems: 'center', gap: 8,
cursor: 'pointer',
}}
onClick={() =>
setExpandedId(isExpanded ? null : c.connector_id)
}
>
<span>{iconMap[c.connector_id] || '🔗'}</span>
<div style={{ flex: 1 }}>
<div style={{ fontSize: 12, fontWeight: 500,
color: 'var(--color-text-secondary)' }}>
{c.display_name}
</div>
<div style={{ fontSize: 10,
color: 'var(--color-text-secondary)' }}>
Not connected
</div>
</div>
<span style={{
color: '#7c3aed', fontSize: 11, fontWeight: 500,
}}>
{isExpanded ? '✕ Close' : '+ Add'}
</span>
</div>
{/* Inline setup panel */}
{isExpanded && meta?.steps && (
<div style={{
borderTop: '1px solid var(--color-border)',
padding: 12,
}}>
{meta.steps.map((step, i) => (
<div
key={i}
style={{
background: 'var(--color-bg)',
border: '1px solid var(--color-border)',
borderRadius: 6, padding: 10,
marginBottom: 8,
}}
>
<div style={{
color: '#7c3aed', fontSize: 10,
fontWeight: 600, marginBottom: 3,
}}>
STEP {i + 1}
</div>
<div style={{
fontSize: 12, marginBottom: step.url ? 4 : 0,
}}>
{step.label}
</div>
{step.url && (
<a
href={step.url}
target="_blank"
rel="noopener noreferrer"
style={{
color: '#60a5fa', fontSize: 11,
textDecoration: 'underline',
}}
>
{step.urlLabel || 'Open'} →
</a>
)}
</div>
))}
{meta.inputFields && (
<InlineConnectForm
fields={meta.inputFields}
loading={loading}
onSubmit={(req) =>
handleConnect(c.connector_id, req)
}
/>
)}
<div style={{
fontSize: 10, color: 'var(--color-text-secondary)',
textAlign: 'center', marginTop: 8,
}}>
🔒 Read-only access · No data leaves your device
</div>
</div>
)}
</div>
);
})}
</div>
)}
</div>
);
}
function InlineConnectForm({
fields,
loading,
onSubmit,
}: {
fields: Array<{ name: string; placeholder: string; type?: string }>;
loading: boolean;
onSubmit: (req: ConnectRequest) => void;
}) {
const [inputs, setInputs] = useState<Record<string, string>>({});
const update = (name: string, value: string) =>
setInputs((p) => ({ ...p, [name]: value }));
const allFilled = fields.every((f) => inputs[f.name]?.trim());
const submit = () => {
const req: ConnectRequest = {};
for (const f of fields) {
if (f.name === 'email') req.email = inputs.email;
else if (f.name === 'password') req.password = inputs.password;
else if (f.name === 'token') req.token = inputs.token;
else if (f.name === 'path') req.path = inputs.path;
}
if (req.email && req.password) {
req.token = `${req.email}:${req.password}`;
req.code = req.token;
}
if (req.token && !req.code) req.code = req.token;
onSubmit(req);
};
return (
<div>
{fields.map((f) => (
<input
key={f.name}
value={inputs[f.name] || ''}
onChange={(e) => update(f.name, e.target.value)}
placeholder={f.placeholder}
type={f.type || 'text'}
style={{
width: '100%', padding: '7px 10px',
background: 'var(--color-bg)',
border: '1px solid var(--color-border)',
borderRadius: 4, color: 'var(--color-text)',
fontSize: 12, marginBottom: 6,
boxSizing: 'border-box',
}}
/>
))}
<button
onClick={submit}
disabled={loading || !allFilled}
style={{
width: '100%', padding: 8,
background: loading || !allFilled ? '#444' : '#7c3aed',
color: 'white', border: 'none',
borderRadius: 6, fontSize: 12, cursor: 'pointer',
}}
>
{loading ? 'Connecting...' : 'Connect'}
</button>
</div>
);
}
```
- [ ] **Step 3: Update DETAIL_TABS to rename and add Messaging**
Find `DETAIL_TABS` (line ~1553). Update to:
```typescript
const DETAIL_TABS = [
{ id: 'overview', label: 'Overview', icon: Activity },
{ id: 'interact', label: 'Interact', icon: MessageSquare },
{ id: 'channels', label: 'Channels', icon: Database },
{ id: 'messaging', label: 'Messaging', icon: Wifi },
{ id: 'tasks', label: 'Tasks', icon: ListTodo },
{ id: 'memory', label: 'Memory', icon: Brain },
{ id: 'learning', label: 'Learning', icon: Settings },
{ id: 'logs', label: 'Logs', icon: FileText },
] as const;
```
Add `Database` to lucide-react imports. Update detailTab type:
```typescript
const [detailTab, setDetailTab] = useState<
'overview' | 'interact' | 'channels' | 'messaging' | 'tasks' | 'memory' | 'learning' | 'logs'
>('overview');
```
- [ ] **Step 4: Add MessagingTab component and wire both tabs**
Add the `MessagingTab` component (reuses the old `AVAILABLE_CHANNELS` data but with clear UX):
```typescript
const MESSAGING_CHANNELS = [
{
type: 'imessage', name: 'iMessage', icon: '💬',
description: 'Text from your iPhone, iPad, or Mac',
activeTemplate: (id: string) => `Text ${id} from your iPhone`,
setupLabel: 'Phone number or email for the agent to use',
placeholder: '+15551234567',
},
{
type: 'slack', name: 'Slack', icon: '#',
description: 'Message from any Slack workspace',
activeTemplate: (id: string) => `DM @jarvis in ${id}`,
setupLabel: 'Slack bot token (xoxb-...)',
placeholder: 'xoxb-...',
},
{
type: 'whatsapp', name: 'WhatsApp', icon: '📱',
description: 'Message via WhatsApp',
activeTemplate: (id: string) => `Message ${id} on WhatsApp`,
setupLabel: 'WhatsApp access token',
placeholder: 'Access token',
},
{
type: 'twilio', name: 'SMS (Twilio)', icon: '📨',
description: 'Text from any phone via Twilio',
activeTemplate: (id: string) => `Text ${id} from any phone`,
setupLabel: 'Twilio phone number',
placeholder: '+15551234567',
},
];
function MessagingTab({ agentId }: { agentId: string }) {
const [bindings, setBindings] = useState<ChannelBinding[]>([]);
const [setupType, setSetupType] = useState<string | null>(null);
const [inputValue, setInputValue] = useState('');
const [loading, setLoading] = useState(false);
const loadBindings = useCallback(() => {
fetchAgentChannels(agentId).then(setBindings).catch(() => setBindings([]));
}, [agentId]);
useEffect(() => { loadBindings(); }, [loadBindings]);
const handleSetup = async (channelType: string) => {
if (!inputValue.trim()) return;
setLoading(true);
try {
await bindAgentChannel(agentId, channelType, {
identifier: inputValue.trim(),
});
setSetupType(null);
setInputValue('');
loadBindings();
} catch { /* */ } finally { setLoading(false); }
};
const handleRemove = async (bindingId: string) => {
try {
await unbindAgentChannel(agentId, bindingId);
loadBindings();
} catch { /* */ }
};
return (
<div style={{ padding: 16 }}>
<div style={{
color: 'var(--color-text-secondary)',
fontSize: 12, marginBottom: 12,
}}>
Talk to your agent from your phone or other platforms
</div>
{MESSAGING_CHANNELS.map((ch) => {
const binding = bindings.find((b) => b.channel_type === ch.type);
const identifier = binding?.config?.identifier as string || binding?.session_id || '';
const isSetup = setupType === ch.type;
return (
<div
key={ch.type}
style={{
background: 'var(--color-bg-secondary)',
border: binding
? '1px solid #2a5a3a'
: '1px dashed var(--color-border)',
borderRadius: 8, marginBottom: 8,
opacity: binding ? 1 : 0.6,
overflow: 'hidden',
}}
>
<div style={{
display: 'flex', alignItems: 'center',
padding: '12px 14px',
}}>
<span style={{ fontSize: 16, marginRight: 10 }}>{ch.icon}</span>
<div style={{ flex: 1 }}>
<div style={{ fontWeight: 600, fontSize: 13 }}>{ch.name}</div>
<div style={{ fontSize: 11,
color: binding ? '#4ade80' : 'var(--color-text-secondary)',
}}>
{binding
? ch.activeTemplate(identifier)
: 'Not set up'}
</div>
</div>
{binding ? (
<div style={{ display: 'flex', alignItems: 'center', gap: 8 }}>
<span style={{
background: '#2a5a3a', color: '#4ade80',
padding: '2px 8px', borderRadius: 10,
fontSize: 10,
}}>Active</span>
<button
onClick={() => handleRemove(binding.id)}
style={{
fontSize: 10, padding: '2px 8px',
background: 'transparent',
color: 'var(--color-text-secondary)',
border: '1px solid var(--color-border)',
borderRadius: 4, cursor: 'pointer',
}}
>Remove</button>
</div>
) : (
<button
onClick={() => {
setSetupType(isSetup ? null : ch.type);
setInputValue('');
}}
style={{
fontSize: 10, padding: '3px 12px',
background: '#7c3aed', color: 'white',
border: 'none', borderRadius: 5,
cursor: 'pointer',
}}
>
{isSetup ? 'Cancel' : 'Set Up'}
</button>
)}
</div>
{isSetup && (
<div style={{
borderTop: '1px solid var(--color-border)',
padding: '10px 14px',
background: 'var(--color-bg)',
}}>
<div style={{
fontSize: 11, marginBottom: 6,
color: 'var(--color-text-secondary)',
}}>{ch.setupLabel}</div>
<div style={{ display: 'flex', gap: 6 }}>
<input
value={inputValue}
onChange={(e) => setInputValue(e.target.value)}
placeholder={ch.placeholder}
style={{
flex: 1, padding: '6px 10px',
background: 'var(--color-bg-secondary)',
border: '1px solid var(--color-border)',
borderRadius: 4, color: 'var(--color-text)',
fontSize: 12,
}}
onKeyDown={(e) => {
if (e.key === 'Enter') handleSetup(ch.type);
}}
/>
<button
onClick={() => handleSetup(ch.type)}
disabled={loading || !inputValue.trim()}
style={{
fontSize: 11, padding: '6px 14px',
background: '#7c3aed', color: 'white',
border: 'none', borderRadius: 4,
cursor: 'pointer',
opacity: loading || !inputValue.trim() ? 0.5 : 1,
}}
>
{loading ? '...' : 'Connect'}
</button>
</div>
</div>
)}
</div>
);
})}
</div>
);
}
```
Wire both tabs in the content rendering section:
```typescript
{detailTab === 'channels' && (
<ChannelsTab agentId={selectedAgent.id} />
)}
{detailTab === 'messaging' && (
<MessagingTab agentId={selectedAgent.id} />
)}
```
- [ ] **Step 5: Add required imports**
At the top of `AgentsPage.tsx`:
```typescript
import { Database } from 'lucide-react';
import { SOURCE_CATALOG } from '../types/connectors';
import type { ConnectRequest } from '../types/connectors';
import { listConnectors, connectSource } from '../lib/connectors-api';
```
- [ ] **Step 6: TypeScript check + build**
```bash
cd frontend && npx tsc --noEmit 2>&1 | head -20
cd frontend && npm run build 2>&1 | tail -10
```
- [ ] **Step 7: Commit**
```bash
git add frontend/src/pages/AgentsPage.tsx frontend/src/lib/connectors-api.ts frontend/src/types/connectors.ts
git commit -m "feat: replace Channels tab with data sources + add Messaging tab"
```
---
### Task 3: E2E test both tabs
**Files:** None (manual testing)
- [ ] **Step 1: Restart server to pick up new static files**
```bash
# Kill existing server, rebuild frontend, restart
kill $(pgrep -f "jarvis serve") 2>/dev/null
cd frontend && npm run build
uv run jarvis serve --port 8222 --model qwen3.5:9b &
```
- [ ] **Step 2: Test Channels tab**
1. Open http://127.0.0.1:8222 → Agents → open agent
2. Click "Channels" tab
3. Verify connected sources show with green border + chunk counts
4. Verify unconnected sources show with dashed border + "+ Add"
5. Click "+ Add" on an unconnected source → verify step-by-step instructions appear inline
- [ ] **Step 3: Test Messaging tab**
1. Click "Messaging" tab
2. Verify active channels show with "Text +1... from your iPhone" instructions
3. Verify inactive channels show "Set Up" button
4. Click "Set Up" → verify input field appears with clear label
- [ ] **Step 4: Push**
```bash
git push origin feat/deep-research-setup
```
docs/superpowers/plans/2026-03-27-channels-tab-and-connector-setup.md
-1006
View File
File diff suppressed because it is too large Load Diff
docs/superpowers/plans/2026-03-27-personal-deep-research-template.md
-356
View File
@@ -1,356 +0,0 @@
# Personal DeepResearch Template + Server Wiring Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Register "Personal Deep Research" as a default agent template, wire DeepResearch tools in the server streaming endpoint, and add a `jarvis research` CLI alias — so users can create and chat with the agent from the desktop app, browser, or CLI.
**Architecture:** A new TOML template file auto-discovered by `AgentManager.list_templates()`. The server's `_stream_managed_agent()` is extended to build the 4 DeepResearch tools (knowledge_search, knowledge_sql, scan_chunks, think) from `~/.openjarvis/knowledge.db` when `agent_type == "deep_research"`. A CLI alias `jarvis research` wraps the existing `deep-research-setup` command.
**Tech Stack:** Python 3.10+, TOML, FastAPI, Click, pytest
---
### Task 1: Create the agent template TOML
**Files:**
- Create: `src/openjarvis/agents/templates/personal_deep_research.toml`
- [ ] **Step 1: Create the template file**
Create `src/openjarvis/agents/templates/personal_deep_research.toml`:
```toml
[template]
id = "personal_deep_research"
name = "Personal Deep Research"
description = "Search across your emails, messages, meeting notes, and documents with multi-hop retrieval and cited reports. Uses BM25 keyword search, SQL aggregation, and LM-powered semantic scanning."
agent_type = "deep_research"
schedule_type = "manual"
tools = ["knowledge_search", "knowledge_sql", "scan_chunks", "think"]
max_turns = 8
temperature = 0.3
max_tokens = 4096
```
- [ ] **Step 2: Verify template is discovered**
```bash
uv run python3 -c "
from openjarvis.agents.manager import AgentManager
templates = AgentManager.list_templates()
ids = [t['id'] for t in templates]
print(f'Templates: {ids}')
assert 'personal_deep_research' in ids, 'Template not found!'
t = next(t for t in templates if t['id'] == 'personal_deep_research')
print(f'Name: {t[\"name\"]}')
print(f'Agent type: {t[\"agent_type\"]}')
print(f'Tools: {t.get(\"tools\", [])}')
print('PASS')
"
```
Expected: Template found with correct fields.
- [ ] **Step 3: Commit**
```bash
git add src/openjarvis/agents/templates/personal_deep_research.toml
git commit -m "feat: add Personal Deep Research agent template"
```
---
### Task 2: Wire DeepResearch tools in server streaming endpoint
**Files:**
- Modify: `src/openjarvis/server/agent_manager_routes.py` (lines 263-266)
- Create: `tests/server/test_deep_research_tools_wiring.py`
- [ ] **Step 1: Write the test**
Create `tests/server/test_deep_research_tools_wiring.py`:
```python
"""Test that _stream_managed_agent wires DeepResearch tools correctly."""
from __future__ import annotations
import tempfile
from pathlib import Path
from unittest.mock import MagicMock, patch
import pytest
def test_deep_research_agent_gets_tools(tmp_path: Path) -> None:
"""When agent_type is deep_research, agent receives 4 tools."""
from openjarvis.connectors.store import KnowledgeStore
# Create a knowledge store with some data
db_path = tmp_path / "knowledge.db"
store = KnowledgeStore(str(db_path))
store.store("test content", source="test", doc_type="note")
from openjarvis.server.agent_manager_routes import _build_deep_research_tools
tools = _build_deep_research_tools(
engine=MagicMock(),
model="test-model",
knowledge_db_path=str(db_path),
)
tool_ids = [t.tool_id for t in tools]
assert "knowledge_search" in tool_ids
assert "knowledge_sql" in tool_ids
assert "scan_chunks" in tool_ids
assert "think" in tool_ids
assert len(tools) == 4
store.close()
def test_deep_research_tools_returns_empty_when_no_db() -> None:
"""When knowledge.db doesn't exist, returns empty list."""
from openjarvis.server.agent_manager_routes import _build_deep_research_tools
tools = _build_deep_research_tools(
engine=MagicMock(),
model="test-model",
knowledge_db_path="/nonexistent/path/knowledge.db",
)
assert tools == []
```
- [ ] **Step 2: Run test to verify it fails**
```bash
uv run pytest tests/server/test_deep_research_tools_wiring.py -v --tb=short
```
Expected: FAIL — `ImportError: cannot import name '_build_deep_research_tools'`
- [ ] **Step 3: Add `_build_deep_research_tools` function to agent_manager_routes.py**
Add this function near the top of the file (after the imports, before the router definitions):
```python
def _build_deep_research_tools(
engine: Any,
model: str,
knowledge_db_path: str = "",
) -> list:
"""Build the 4 DeepResearch tools from a KnowledgeStore.
Returns an empty list if the knowledge DB does not exist.
"""
from pathlib import Path
if not knowledge_db_path:
from openjarvis.core.config import DEFAULT_CONFIG_DIR
knowledge_db_path = str(DEFAULT_CONFIG_DIR / "knowledge.db")
if not Path(knowledge_db_path).exists():
return []
from openjarvis.connectors.retriever import TwoStageRetriever
from openjarvis.connectors.store import KnowledgeStore
from openjarvis.tools.knowledge_search import KnowledgeSearchTool
from openjarvis.tools.knowledge_sql import KnowledgeSQLTool
from openjarvis.tools.scan_chunks import ScanChunksTool
from openjarvis.tools.think import ThinkTool
store = KnowledgeStore(knowledge_db_path)
retriever = TwoStageRetriever(store)
return [
KnowledgeSearchTool(retriever=retriever),
KnowledgeSQLTool(store=store),
ScanChunksTool(store=store, engine=engine, model=model),
ThinkTool(),
]
```
- [ ] **Step 4: Wire the tools into `_stream_managed_agent`**
In `_stream_managed_agent()`, add tool wiring between the config extraction (line ~263) and the agent instantiation (line ~265). Insert after `if config.get("max_turns") is not None:` block:
```python
# Build DeepResearch tools when applicable
if agent_type == "deep_research":
tools = _build_deep_research_tools(engine=engine, model=model)
if tools:
agent_kwargs["tools"] = tools
```
- [ ] **Step 5: Ensure tests/server/__init__.py exists**
```bash
touch tests/server/__init__.py
```
- [ ] **Step 6: Run tests**
```bash
uv run pytest tests/server/test_deep_research_tools_wiring.py -v --tb=short
```
Expected: 2/2 PASS.
- [ ] **Step 7: Commit**
```bash
git add src/openjarvis/server/agent_manager_routes.py tests/server/__init__.py tests/server/test_deep_research_tools_wiring.py
git commit -m "feat: wire DeepResearch tools in managed agent streaming endpoint"
```
---
### Task 3: Add `jarvis research` CLI alias
**Files:**
- Modify: `src/openjarvis/cli/__init__.py`
- [ ] **Step 1: Add the alias**
In `src/openjarvis/cli/__init__.py`, find the line that registers `deep-research-setup` (should look like `cli.add_command(deep_research_setup, "deep-research-setup")`). Add the alias right after:
```python
cli.add_command(deep_research_setup, "research")
```
This registers the same command under the shorter name `research`.
- [ ] **Step 2: Verify**
```bash
uv run jarvis research --help | head -5
```
Expected: Shows the same help as `jarvis deep-research-setup --help`.
- [ ] **Step 3: Commit**
```bash
git add src/openjarvis/cli/__init__.py
git commit -m "feat: add 'jarvis research' CLI alias for deep-research-setup"
```
---
### Task 4: End-to-end test — template in Agents tab via API
**Files:** None (manual testing via API)
- [ ] **Step 1: Start the server**
```bash
uv run jarvis serve --host 127.0.0.1 --port 8000 &
sleep 3
```
- [ ] **Step 2: Verify template appears**
```bash
curl -s http://localhost:8000/v1/templates | python3 -c "
import sys, json
data = json.load(sys.stdin)
ids = [t['id'] for t in data['templates']]
print(f'Templates: {ids}')
assert 'personal_deep_research' in ids
print('PASS: template found')
"
```
- [ ] **Step 3: Create agent from template**
```bash
curl -s -X POST http://localhost:8000/v1/managed-agents \
-H 'Content-Type: application/json' \
-d '{"name": "My Research Agent", "template_id": "personal_deep_research"}' \
| python3 -c "
import sys, json
agent = json.load(sys.stdin)
print(f'Agent created: {agent[\"id\"]}')
print(f'Type: {agent[\"agent_type\"]}')
print(f'Status: {agent[\"status\"]}')
print(f'Config tools: {agent.get(\"config\", {}).get(\"tools\", [])}')
"
```
Expected: Agent created with `agent_type: deep_research`.
- [ ] **Step 4: Send a message and verify streaming**
```bash
AGENT_ID=$(curl -s http://localhost:8000/v1/managed-agents | python3 -c "
import sys, json
agents = json.load(sys.stdin)['agents']
dr = [a for a in agents if a['agent_type'] == 'deep_research']
print(dr[0]['id'] if dr else '')
")
curl -s -X POST "http://localhost:8000/v1/managed-agents/${AGENT_ID}/messages" \
-H 'Content-Type: application/json' \
-d '{"content": "Who are the 5 people I message the most?", "stream": true}' \
--no-buffer | head -20
```
Expected: SSE chunks streaming with agent response including tool call results.
- [ ] **Step 5: Stop server and push**
```bash
kill %1
git push origin feat/deep-research-setup
```
---
### Task 5: Update PR #78 title and description
**Files:** None (GitHub CLI)
- [ ] **Step 1: Update PR title and body**
```bash
gh pr edit 78 --repo open-jarvis/OpenJarvis \
--title "feat: Personal Deep Research — connectors, retrieval, agent, channels, desktop" \
--body "$(cat <<'PREOF'
## Summary
Full Deep Research experience: connect personal data sources → ingest → multi-hop retrieval → cited research reports via desktop app, browser, CLI, or iMessage.
### What's built
- **13 connectors**: Gmail IMAP, Outlook, Obsidian, Apple Notes, iMessage, Slack, Notion, Granola, Google Drive*, Calendar*, Contacts*, Dropbox*, WhatsApp*
- **Ingestion pipeline**: SemanticChunker, KnowledgeStore (FTS5/BM25), incremental sync, attachment store
- **4 agent tools**: knowledge_search (BM25), knowledge_sql (SQL aggregation), scan_chunks (LM semantic grep), think (reasoning)
- **DeepResearchAgent**: multi-hop with query expansion, forced synthesis, cited reports
- **Agent template**: "Personal Deep Research" in Agents tab — create and chat via desktop/browser
- **CLI**: `jarvis deep-research-setup` / `jarvis research` — auto-detect sources, ingest, chat
- **API**: 6 connector endpoints + managed agent streaming
- **Channel gateway**: ChannelBridge, webhook routes for Twilio/BlueBubbles/WhatsApp
- **220+ tests**
*\* = mocked tests only*
### Live-verified with real data
- 55,302 chunks (Apple Notes + iMessage + Gmail + Slack + Notion + Granola)
- DeepResearchAgent with Qwen3.5 35B MoE producing cited reports
- Cross-source queries: found Spain trip dates from iMessages, ranked top contacts via SQL, identified recurring meetings from Granola
## Test plan
- [ ] `uv run pytest tests/ -v` — all tests pass
- [ ] `uv run jarvis research` — end-to-end CLI flow
- [ ] Create agent from template in Agents tab
- [ ] Stream chat response with tool calls
- [ ] Verify read-only: no writes to any connector
🤖 Generated with [Claude Code](https://claude.com/claude-code)
PREOF
)"
```
- [ ] **Step 2: Push our branch to the PR branch**
```bash
git push origin feat/deep-research-setup:feat/mobile-channel-gateway --force-with-lease
```
docs/superpowers/specs/2026-03-25-deep-research-setup-design.md
-468
View File
@@ -1,468 +0,0 @@
# Deep Research Setup Experience — Design Spec
**Date:** 2026-03-25
**Status:** Draft
**Scope:** End-to-end design for the OpenJarvis "killer install + setup" experience — from first launch to Deep Research over your personal data, accessible via desktop app, CLI, and messaging channels.
---
## 1. Vision
An Apple-like onboarding experience that connects users to their personal data (email, messaging, documents, calendar, contacts, notes), indexes it privately on-device, and provides a Deep Research agent that can navigate the full corpus with multi-hop retrieval — all accessible from the desktop app, CLI, or messaging channels like iMessage, WhatsApp, and Slack.
## 2. Architecture Overview
Seven layers from data sources to user interfaces:
```
Data Sources (12 services)
↓ OAuth / API / Local DB access
Connector Layer (native bulk sync + MCP real-time tools)
↓ Normalized Document objects
Ingestion Pipeline (normalize → deduplicate → chunk → index)
↓
On-Device Storage (SQLite/FTS5 + ColBERTv2 + attachment store)
↓
Multi-Layer Retrieval (BM25 recall → ColBERTv2 rerank → agent multi-hop)
↓
Deep Research Agent (ToolUsingAgent with knowledge_search, MCP tools, think, web_search)
↓
User Interfaces (Desktop App, CLI, iMessage, WhatsApp, Slack)
```
### Design Principles
- **Local-first:** All data stored on-device in SQLite. No cloud dependency for storage or retrieval.
- **Hybrid connectors:** Native bulk sync for initial ingest, MCP tools for real-time agent queries.
- **Two-stage retrieval:** BM25 for fast recall, ColBERTv2 for precise reranking. Agent adds multi-hop.
- **Progressive onboarding:** Pick sources → guided OAuth for each → bulk ingest → ready to research.
- **Escalation model:** Quick answers in chat channels, full reports in desktop app.
## 3. Data Sources
Twelve data sources across three categories:
| Category | Sources | Auth Type |
|----------|---------|-----------|
| **Communication** | Gmail, Slack, iMessage, WhatsApp | OAuth, OAuth, Local DB, Bridge (QR) |
| **Documents** | Google Drive, Dropbox, Notion | OAuth, OAuth, OAuth |
| **PIM** | Google Calendar, Google Contacts, Apple Notes, Obsidian/Markdown | OAuth, OAuth, Local DB, Filesystem |
## 4. Setup Wizard (Desktop-First)
The primary onboarding path is the Tauri desktop app. The CLI gets a simpler `jarvis connect` command that performs the same operations text-based. The desktop wizard is the "wow" experience — polished visual design with real product logos, clean progress bars, and premium feel throughout.
### Step 1: Welcome + Hardware Detection
Auto-detects CPU, RAM, GPU. Shows friendly capability messaging (e.g., "MacBook Pro M4 Max, 128GB RAM — you can run powerful local models"). Background: Ollama starts, recommended model begins downloading.
**Builds on:** Existing desktop boot sequence (Ollama → model → server) and `jarvis init` hardware detection.
**New:** Friendlier UI with hardware capability messaging, sets user expectations for what's possible on their machine.
### Step 2: Engine Setup
If Ollama is already installed, skip silently. If not, one-click install or "Use cloud instead" option. Model download with a polished progress bar. Option to add cloud API keys for hybrid local+cloud mode.
**Builds on:** Existing `jarvis init` engine selection and model download logic.
**New:** Automated Ollama install, cloud key entry UI, visual progress.
### Step 3: Pick Your Sources
A grid of source cards with product logos, grouped by category (Communication / Documents / PIM). User toggles on/off. "Connect what you want now — you can always add more later."
**Entirely new.** This step and everything after it is new functionality.
### Step 4: Connect Each Source (Guided)
For each selected source, one at a time, with a sidebar checklist showing connected/remaining status. "Skip" option for each. Four connection patterns:
**OAuth services** (Gmail, Drive, Slack, Calendar, Contacts, Dropbox, Notion): Show what permissions are requested and why → "Open in browser" button → OAuth consent screen → redirect back → green checkmark. OpenJarvis registers its own OAuth apps — users never create developer credentials.
**Local access** (iMessage, Apple Notes): Request macOS Full Disk Access permission → explain why in plain language → link to System Preferences → verify access was granted.
**Bridge services** (WhatsApp): Display QR code in the app → user scans with their phone → connection confirmed.
**Local files** (Obsidian/Markdown): File picker → select vault directory → confirm.
### Step 5: Ingest & Index
Bulk sync begins for all connected sources in parallel. Live dashboard shows per-source progress with item counts and progress bars:
```
✉ Gmail 12,847 / ~45,000 emails ████████░░░░░░░ 28%
💬 Slack 3,201 / ~8,500 messages ██████████░░░░░ 38%
📁 Google Drive 156 / ~420 docs █████████░░░░░░ 37%
📱 iMessage ✓ Complete 24,102 messages indexed
```
**Key UX:** User doesn't have to wait. They can start chatting with partial data — "Your data is still syncing, but you can start asking questions now." Ingest is resumable: close the app, come back later, it picks up where it left off.
### Step 6: Ready — First Query
Drops user into the Deep Research chat. Suggested starter queries based on their connected sources:
- "What were the key decisions from last week's team threads?"
- "Find the proposal doc Sarah shared about the Q3 roadmap"
- "Summarize my unread emails from today"
## 5. Connector Layer
### New Abstraction: `BaseConnector`
Lives in `src/openjarvis/connectors/` — separate from channels. Channels are for real-time communication (send/receive). Connectors are for data ingest (bulk download + index history).
```python
class BaseConnector(ABC):
# Identity
connector_id: str # "gmail", "slack", etc.
display_name: str # "Gmail", "Slack", etc.
icon_url: str # Product logo for UI
auth_type: str # "oauth" | "local" | "bridge" | "filesystem"
# Auth lifecycle
def auth_url() -> str # OAuth: generate consent URL
def handle_callback(code: str) -> None # OAuth: exchange code for tokens
def is_connected() -> bool
def disconnect() -> None
# Bulk sync (native)
def sync(since: datetime | None) -> Iterator[Document]
def sync_status() -> SyncStatus
# MCP real-time (agent-callable)
def mcp_tools() -> list[ToolSpec]
```
Registered via `@ConnectorRegistry.register("gmail")` — new registry alongside the existing ones.
### SyncStatus
```python
@dataclass
class SyncStatus:
state: str # "idle", "syncing", "paused", "error"
items_synced: int # items completed so far
items_total: int # estimated total (0 if unknown)
last_sync: datetime | None
cursor: str | None # opaque checkpoint for resume
error: str | None # last error message if state == "error"
```
### Universal Document Schema
All connectors normalize their data into a common dataclass:
```python
@dataclass
class Document:
doc_id: str # globally unique (connector:source_id)
source: str # "gmail", "slack", "gdrive", etc.
doc_type: str # "email", "message", "document", "event", "contact", "note"
content: str # plain text body
title: str # subject line, filename, event title
author: str # sender / creator
participants: list[str] # recipients, channel members, attendees
timestamp: datetime # when created/sent
thread_id: str | None # conversation grouping
url: str | None # deep link back to original source
attachments: list[Attachment]
metadata: dict # source-specific (labels, channel name, folder, etc.)
```
Six `doc_type` values: `email`, `message`, `document`, `event`, `contact`, `note`.
### Per-Connector Details
**Gmail** — Gmail API (REST). Scope: `gmail.readonly`. Bulk sync via `messages.list` → `messages.get`, paginated, resumable via `historyId`. MCP tools: `gmail_search_emails`, `gmail_get_thread`, `gmail_list_unread`.
**Slack** — Slack Web API. Scopes: `channels:history`, `channels:read`, `users:read`. Bulk sync via `conversations.history` per channel, cursor-paginated. MCP tools: `slack_search_messages`, `slack_get_thread`, `slack_list_channels`.
**iMessage** — Direct SQLite read of `~/Library/Messages/chat.db`. Requires macOS Full Disk Access. Bulk sync queries message + handle tables. MCP tools: `imessage_search_messages`, `imessage_get_conversation`.
**WhatsApp** — Baileys Node.js bridge (existing). Auth via QR code scan. Bulk sync via `fetchMessageHistory` through Baileys protocol. MCP tools: `whatsapp_search_messages`, `whatsapp_get_chat`.
**Google Drive** — Drive API v3. Scope: `drive.readonly`. Bulk sync via `files.list` → export/download. Google Docs/Sheets/Slides exported as text/markdown. MCP tools: `gdrive_search_files`, `gdrive_get_document`, `gdrive_list_recent`.
**Dropbox** — Dropbox API v2. Scopes: `files.metadata.read`, `files.content.read`. Bulk sync via `list_folder` recursive → download text-extractable files. MCP tools: `dropbox_search_files`, `dropbox_get_file`, `dropbox_list_recent`.
**Notion** — Notion API. Read content integration scope. Bulk sync via `search` → retrieve blocks → render to markdown. MCP tools: `notion_search_pages`, `notion_get_page`.
**Google Calendar** — Calendar API v3. Scope: `calendar.readonly`. Bulk sync via `events.list` across calendars, paginated. MCP tools: `calendar_get_events_today`, `calendar_search_events`, `calendar_next_meeting`.
**Google Contacts** — People API. Scope: `contacts.readonly`. Bulk sync via `people.connections.list`, paginated. MCP tools: `contacts_find`, `contacts_get_info`.
**Apple Notes** — Direct SQLite read of `~/Library/Group Containers/group.com.apple.notes/NoteStore.sqlite`. Requires Full Disk Access. Bulk sync decodes `ZICNOTEDATA` (gzipped HTML → plain text). MCP tools: `notes_search`, `notes_get_note`.
**Obsidian/Markdown** — Direct filesystem read of a user-selected vault directory. Bulk sync walks `.md` files, parses YAML frontmatter + wikilinks. MCP tools: `obsidian_search_notes`, `obsidian_get_note`, `obsidian_list_recent`.
### SyncEngine
A shared `SyncEngine` orchestrates all connectors:
- **Checkpoint/Resume:** Stores last sync cursor per connector in SQLite. Interrupted syncs resume from the checkpoint.
- **Rate Limiting:** Per-connector rate limits with exponential backoff. Respects API quotas (e.g., Gmail: 250 quota units/sec).
- **Incremental Sync:** After initial bulk sync, periodic delta sync fetches only new/modified items. Configurable interval.
- **Parallel Ingest:** Multiple connectors sync concurrently. Documents stream into the ingestion pipeline as they arrive.
## 6. Ingestion Pipeline
Four stages from raw documents to retrieval-ready index:
### Stage 1: Normalize
Connectors yield `Document` objects. HTML emails converted to plain text via `html2text`. Google Docs exported as markdown. Notion blocks rendered to markdown. Apple Notes decompressed from gzipped HTML to plain text. Attachments extracted, hashed (SHA-256), stored in content-addressed blob store.
### Stage 2: Deduplicate
Cross-source dedup by `doc_id` (connector:source_id). If the same message appears in a Slack export and an email notification, keep both but link them via metadata. Attachments deduped by SHA-256 — a file shared across Gmail, Slack, and Drive is stored once.
### Stage 3: Chunk
Uses the existing `chunking.py` infrastructure with type-aware strategies. **Chunk size defaults to the active reranker's max context:** 512 tokens when ColBERTv2 is available (its WordPiece max), 256 tokens when falling back to FAISS with `all-MiniLM-L6-v2`.
**Semantic splitting rules (never split mid-sentence):**
- **Emails/Messages:** Keep whole if under the chunk limit. For long threads, split on message/reply boundaries — never mid-message. If a single message exceeds the limit, fall back to sentence-boundary splitting.
- **Documents:** Split on section boundaries (`## Heading`) first, then paragraph boundaries within sections. If a single paragraph exceeds the limit, split on sentence boundaries. Each chunk inherits its section heading as metadata.
- **Events:** Single chunk per event (title + attendees + description + location). No splitting needed.
- **Contacts:** Single chunk per contact (name + email + phone + org + notes). Structured for entity lookup.
- **Notes:** Same rules as documents — section boundaries first, then paragraphs, then sentences.
**Metadata carries context:** Each chunk inherits the parent document's title, author, timestamp, source, and its section heading (if any), so retrieval results are never orphaned fragments with no context about where they came from.
### Stage 4: Index (Dual-Write)
Each chunk is written to both indexes simultaneously:
**SQLite + FTS5** — durable store with BM25 ranking. Rust-backed (existing infrastructure). Stores: chunk content, doc_id, source, doc_type, title, author, participants, timestamp, thread_id, url, metadata JSON, chunk_index.
**ColBERTv2 Index** — token-level embeddings for semantic reranking. **New: persisted to disk** (current implementation is in-memory). Memory-mapped tensor file with chunk_id→offset index. Append-only for incremental indexing. Lazy loading — only map pages touched by current query.
### SQLite Schema
Extends the existing `documents` table with source-aware columns:
```sql
CREATE TABLE documents (
id TEXT PRIMARY KEY, -- chunk_id
doc_id TEXT NOT NULL, -- parent document (connector:source_id)
content TEXT NOT NULL, -- chunk text
source TEXT NOT NULL, -- "gmail", "slack", "gdrive", ...
doc_type TEXT NOT NULL, -- "email", "message", "document", ...
title TEXT, -- subject / filename / event title
author TEXT, -- sender / creator
participants TEXT, -- JSON array
timestamp TEXT NOT NULL, -- ISO 8601
thread_id TEXT, -- conversation grouping
url TEXT, -- deep link to original
metadata TEXT, -- JSON (labels, channel, folder, etc.)
chunk_index INTEGER, -- position within parent doc
created_at TEXT NOT NULL -- when indexed
);
CREATE VIRTUAL TABLE documents_fts USING fts5(
content, title, author,
content=documents, content_rowid=rowid,
tokenize='porter unicode61'
);
CREATE INDEX idx_source ON documents(source);
CREATE INDEX idx_doc_type ON documents(doc_type);
CREATE INDEX idx_author ON documents(author);
CREATE INDEX idx_timestamp ON documents(timestamp);
CREATE INDEX idx_thread ON documents(thread_id);
CREATE INDEX idx_doc_id ON documents(doc_id);
```
### ColBERTv2 Persistence (New)
The current ColBERT backend is in-memory only. For a personal knowledge base with 100K+ chunks, disk persistence is required.
- **On-disk format:** Memory-mapped tensor file + chunk_id→offset index. Lazy loading. Append-only for incremental indexing.
- **Fallback:** Systems without ColBERT dependencies (no torch) fall back to BM25 + FAISS (single-vector with `all-MiniLM-L6-v2`). The retrieval API is identical — callers don't know which reranker is active. When falling back to FAISS, chunk size adjusts to 256 tokens to match the embedding model's context.
### Attachment Store
Content-addressed blob store at `~/.openjarvis/blobs/{sha256[:2]}/{sha256}`. A metadata table tracks: hash, filename, mime_type, size_bytes, and source doc_ids.
Text extraction: PDFs via pdfplumber (existing). Office docs via python-docx/openpyxl. Images skipped for now (future: OCR). Extracted text gets chunked and indexed alongside the parent document.
## 7. Multi-Layer Retrieval
Two-stage retrieval as the base primitive, with agent-level multi-hop on top:
### Stage 1: BM25 Recall
FTS5 MATCH query with Porter stemming against the `documents_fts` virtual table. Returns top-100 candidates ranked by BM25. Supports pre-filtering by source, doc_type, author, and timestamp range via SQL WHERE clauses before the FTS5 MATCH.
### Stage 2: ColBERTv2 Rerank
The top-100 BM25 candidates are reranked using ColBERTv2 late interaction scoring (MaxSim). Returns top-K results (default K=10). This is where semantic understanding kicks in — ColBERT's token-level matching catches results that keyword search alone would miss or misrank.
### Stage 3: Agent Multi-Hop
The Deep Research agent can invoke the retrieval primitive (stages 1+2) multiple times across turns, refining its subqueries based on what it's found. Configurable max turns (default: 5 hops). The multi-hop logic lives at the agent level, not the retrieval level — clean separation of concerns.
## 8. Deep Research Agent
### Agent Configuration
```python
@AgentRegistry.register("deep_research")
class DeepResearchAgent(ToolUsingAgent):
agent_id = "deep_research"
# Default model: largest Qwen3.5 that fits locally, or cloud model if configured
# Default max_turns: 5 (multi-hop research turns)
```
### Tool Belt
**`knowledge_search`** (core retrieval tool): BM25→ColBERT with filters: `query`, `source`, `doc_type`, `author`, `since`/`until`, `top_k`. Returns chunks with content, source, author, timestamp, url (deep link), relevance score.
**MCP connector tools** (real-time data access): Each connected source registers MCP tools. Agent can call these for fresh data not yet in the index — `gmail_search_emails`, `gmail_get_thread`, `slack_search_messages`, `gdrive_get_document`, `calendar_get_events`, `contacts_find`, etc.
**`think`** (reasoning scratchpad): Existing tool. The agent uses this between retrieval hops to synthesize findings and plan the next subquery. Critical for multi-hop research.
**`web_search`** (external supplement): Existing tool (DuckDuckGo or Tavily). When personal data references external context (a news article, a product, a company), the agent supplements with public web search.
### Multi-Hop Research Loop
Example for "What was the context around the Kubernetes migration decision?":
```
Turn 1: think → "I need to find discussions about K8s migration across email and Slack"
knowledge_search(query="Kubernetes migration", top_k=10) → 8 results
Turn 2: think → "Found a Slack thread from March. Sarah and Mike debated cost. Get full thread + related docs"
knowledge_search(query="Kubernetes cost analysis", author="sarah", doc_type="document") → 3 results
slack_get_thread(thread_ts="1709234567.123") → full thread
Turn 3: think → "Sarah's doc references a meeting on March 5th. Check calendar + find the proposal"
calendar_get_events(query="Kubernetes", since="2024-03-01") → 1 event
knowledge_search(query="Kubernetes proposal", doc_type="document") → 2 results
Synthesize: Comprehensive report with cross-platform citations + deep links
```
### Output Format
Structured research report with:
- Narrative synthesis of findings
- Inline citations as clickable deep links back to the original source (Slack thread, Gmail message, Drive doc, Calendar event)
- Source list at the bottom with platform, title, author, and date
### Conversational Follow-up
After the initial report, the user can drill deeper or pivot:
- "Show me just Mike's cost concerns in detail" → new retrieval, focused summary
- "Draft a summary of this for my manager" → uses conversation context, no new retrieval needed
- "What happened after the decision?" → new time-scoped retrieval
## 9. Channel Plugins
### ChannelAgent
A lightweight layer between existing channel transports and the Deep Research Agent. Classifies incoming queries automatically and routes them — the user never specifies "quick" or "deep."
**Classification is automatic** based on query analysis:
- **Quick signals:** Single entity lookup, starts with "when"/"where"/"find", short query (<20 words), mentions a specific person/file/event.
- **Deep signals:** "summarize", "research", "what was the context", time ranges ("last month"), multi-entity queries.
- **Adaptive:** If the agent starts down the quick path but realizes mid-retrieval it needs more hops, it escalates on the fly.
### Quick Path (Answer Inline)
Simple lookups answered in a single retrieval hop that fit in a chat message:
```
User: When's my next meeting with Sarah?
Jarvis: Tomorrow at 2pm — "Q3 Planning Sync" with Sarah Chen, Mike Ross, and you.
Google Meet link: meet.google.com/abc-defg-hij
```
### Deep Path (Escalate to Desktop)
Complex queries requiring multi-hop retrieval or long-form output:
```
User: What was the full context behind the decision to switch to Kubernetes?
Jarvis: That's a complex question spanning multiple sources — I found relevant threads
in Slack, emails from Sarah and Mike, and a proposal doc in Drive.
I'm preparing a full research report for you.
📄 Open full report in OpenJarvis → openjarvis://research/{session_id}
```
Escalation flow:
1. Agent kicks off multi-hop research in the background (doesn't block the chat)
2. Sends brief preview + escalation link in the chat channel
3. When report is ready, sends follow-up notification: "Your report is ready"
4. Deep link (`openjarvis://research/{session_id}`) opens desktop app directly to the research session — Tauri registers as URL handler
### Per-Channel Implementation
**iMessage:** BlueBubbles bridge. Currently send-only — needs incoming message support added. Plain text + URL only. Escalation via `openjarvis://` deep link.
**WhatsApp:** Baileys bridge (existing, already bidirectional). Supports WhatsApp markdown (*bold*, _italic_). Escalation via deep link (WhatsApp renders URLs as tappable).
**Slack:** Socket Mode (existing, already bidirectional). Can use Block Kit for richer formatting — bullet lists, bold, links, buttons. DM the Jarvis bot or @mention in a channel.
### Shared Auth
For WhatsApp and Slack, the data connector and channel plugin share the same authentication. Connecting Slack as a data source automatically enables the Slack channel plugin. One OAuth flow, two capabilities. Channel plugins are configured from a "Connected Apps" settings page after initial setup.
## 10. CLI: `jarvis connect`
The CLI equivalent of the desktop setup wizard. Simpler, text-based, but same underlying logic:
```bash
jarvis connect # Interactive: pick sources, walk through auth
jarvis connect gmail # Connect a specific source
jarvis connect --list # Show connected sources and sync status
jarvis connect --sync # Trigger incremental sync for all sources
jarvis connect --disconnect gmail # Disconnect a source
```
OAuth flows open the system browser for consent, then listen on a localhost callback URL. Local access (iMessage, Apple Notes) checks permissions and provides instructions. Filesystem sources (Obsidian) accept a path argument.
## 11. Implementation Phases
### Phase 1: Connector Foundation + Gmail + Obsidian
- `BaseConnector` ABC and `ConnectorRegistry`
- `Document` schema and `SyncEngine` (checkpoint/resume, rate limiting)
- Gmail connector (OAuth flow + bulk sync + MCP tools)
- Obsidian/Markdown connector (filesystem, simplest possible)
- Extended SQLite schema with source-aware columns
- `knowledge_search` tool with filtered BM25 retrieval
- CLI `jarvis connect` command
- Tests for all of the above
### Phase 2: Desktop Wizard + More Connectors
- Desktop setup wizard UI (Steps 1-6) with product logos, progress bars, polished design
- Slack connector
- Google Drive connector
- Google Calendar + Contacts connectors
- iMessage connector (macOS local DB access)
- Parallel ingest dashboard in desktop UI
### Phase 3: ColBERTv2 Persistence + Deep Research Agent
- ColBERTv2 on-disk persistence (memory-mapped tensors)
- Two-stage retrieval (BM25 recall → ColBERT rerank)
- `DeepResearchAgent` with multi-hop loop
- Research report output with inline citations and deep links
- Conversational follow-up support
- Desktop app research UI (report rendering, source panel)
### Phase 4: Channel Plugins + Remaining Connectors
- `ChannelAgent` with automatic quick/deep classification
- iMessage channel plugin (add incoming message support to BlueBubbles)
- WhatsApp channel plugin (leverage existing Baileys bridge)
- Slack channel plugin (leverage existing Socket Mode)
- `openjarvis://` URL handler registration in Tauri
- Remaining connectors: WhatsApp, Dropbox, Notion, Apple Notes
### Phase 5: Polish + Incremental Sync
- Incremental delta sync for all connectors
- Attachment store with content-addressed dedup and text extraction
- Desktop "Connected Apps" settings page (manage sources post-setup)
- Sync status dashboard (last sync time, item counts, errors)
- Error handling and retry UX across all connectors
docs/superpowers/specs/2026-03-26-deep-research-agent-v2-design.md
-117
View File
@@ -1,117 +0,0 @@
# Deep Research Agent v2 — Better Tools + Prompts
## Goal
Improve the DeepResearchAgent so it can answer aggregation queries ("who do I talk to most"), semantic/fuzzy queries ("which VCs have I spoken with"), and factual queries ("when was my trip to Spain") by adding SQL and LM-scan tools alongside better prompts.
## Evaluation Criteria
Re-run these 4 queries and compare to v1 (which returned empty or "no data found" for all 4):
1. "When was my most recent trip to Spain?"
2. "Which VCs have I spoken with since 2023?"
3. "Who are the 10 people I have spoken with the most over text?"
4. "What meetings take up most of my time based on my calendar and meeting logs?"
Success = the agent produces substantive, cited answers for at least 3 of 4.
## Changes
### 1. New tool: `knowledge_sql`
Read-only SQL queries against the `knowledge_chunks` table. The agent writes SELECT statements to aggregate, count, filter, and rank data.
```python
class KnowledgeSQLTool(BaseTool):
tool_id = "knowledge_sql"
def execute(self, query: str) -> ToolResult:
# Reject non-SELECT queries
# Execute against KnowledgeStore's SQLite connection
# Return formatted rows (max 50 rows)
```
Exposed schema (included in the tool description so the model knows the columns):
```
knowledge_chunks(id, content, source, doc_type, doc_id, title, author,
participants, timestamp, thread_id, url, metadata, chunk_index)
```
### 2. New tool: `scan_chunks`
Semantic grep — pulls chunks by filter, batches through the LM with a question.
```python
class ScanChunksTool(BaseTool):
tool_id = "scan_chunks"
def execute(self, question: str, source: str = "", doc_type: str = "",
since: str = "", until: str = "", max_chunks: int = 200,
batch_size: int = 20) -> ToolResult:
# 1. Pull chunks matching filters from KnowledgeStore
# 2. Batch into groups of batch_size
# 3. For each batch, call engine.generate():
# "Extract info relevant to: {question}\n\nChunks:\n{batch_text}"
# 4. Return aggregated findings
```
This is the "LM combs through tokens" capability — catches semantic matches that BM25 keyword search misses.
### 3. Wire `think` tool
The `think` tool already exists at `src/openjarvis/tools/think.py`. Wire it into the DeepResearchAgent's tool list in `deep_research_setup_cmd.py`.
### 4. System prompt rewrite
Replace the current generic prompt with one that teaches query strategies:
```
/no_think
You are a deep research agent with access to a personal knowledge base
containing emails, messages, meeting notes, documents, and notes.
## Your Tools
- **knowledge_search**: BM25 keyword search. Best for finding specific topics,
names, or phrases. Use filters (source, author, since, until) to narrow results.
- **knowledge_sql**: Run SQL queries against the knowledge_chunks table.
Schema: knowledge_chunks(id, content, source, doc_type, doc_id, title,
author, participants, timestamp, thread_id, url, metadata, chunk_index)
Best for: counting, ranking, aggregation, time-range queries.
Example: SELECT author, COUNT(*) as n FROM knowledge_chunks WHERE source='imessage' GROUP BY author ORDER BY n DESC LIMIT 10
- **scan_chunks**: Semantic search — feeds chunks to an LM that reads the actual
text looking for relevant information. Use when keyword search misses semantic
matches (e.g. searching for "VCs" when text says "fundraising round").
Slower but catches what BM25 misses.
- **think**: Reasoning scratchpad. Use between searches to plan your next query,
evaluate findings, and identify gaps.
## Strategy
1. Start with **think** to plan your approach — which tools suit this query?
2. For "who/what/how many" queries → start with **knowledge_sql**
3. For specific topics → start with **knowledge_search**
4. If keyword search returns nothing useful → try **scan_chunks** with broader filters
5. Cross-reference across sources — search emails, then messages, then notes
6. After gathering evidence → write a cited narrative report
## Citation Format
Cite sources as: [source] title -- author
Include a Sources section at the end.
```
### 5. Loop guard tuning
Increase the `max_turns` default from 5 to 8, and configure the loop guard (if present) with a higher poll budget to accommodate the larger tool set.
## What Does NOT Change
- Agent class structure (DeepResearchAgent)
- Retriever pipeline (BM25 + optional ColBERT)
- Ingestion pipeline
- Connectors
- CLI command
docs/superpowers/specs/2026-03-26-deep-research-vertical-slice-design.md
-101
View File
@@ -1,101 +0,0 @@
# Deep Research Vertical Slice — End-to-End on Laptop
## Goal
Get the full Deep Research experience working end-to-end on a MacBook with real personal data: connect local sources → ingest → retrieve → research agent → cited report. Then wire the API router so the desktop wizard UI works too.
## Scope
**In scope:**
- Fix Apple Notes protobuf text extraction + broken test
- `jarvis deep-research-setup` CLI command that auto-detects and ingests local sources
- Ingest 3 local sources into shared KnowledgeStore (~/.openjarvis/knowledge.db):
- Apple Notes (42 notes, ~100 chunks)
- iMessage (52K messages, ~52K chunks)
- Obsidian vault (size varies)
- Run TwoStageRetriever queries (BM25 + optional ColBERT rerank) against combined corpus
- Run DeepResearchAgent with Qwen3.5 4B via Ollama — multi-hop research producing cited reports
- Wire connectors_router into FastAPI app (one-line fix)
- Smoke test wizard UI via browser
**Out of scope:**
- OAuth connectors (Gmail, Drive, Calendar, Contacts, Slack, Notion, Dropbox)
- Channel plugins (iMessage/WhatsApp/Slack bot)
- Sync scheduling, performance optimization
- ColBERT disk persistence tuning
## Architecture
### Data Flow
```
Local Sources (no OAuth)
Apple Notes → NoteStore.sqlite → protobuf → plain text
iMessage → chat.db → messages with contact/conversation metadata
Obsidian → filesystem → .md files
↓
Connectors (AppleNotesConnector, IMessageConnector, ObsidianConnector)
yield Document(doc_id, source, doc_type, content, title, author, timestamp, ...)
↓
IngestionPipeline
Dedup by doc_id → SemanticChunker (512 token chunks) → KnowledgeStore
↓
KnowledgeStore (~/.openjarvis/knowledge.db)
SQLite + FTS5 (BM25), ~50K-55K chunks
↓
TwoStageRetriever
Stage 1: BM25 recall (top 100) with source/type/date filters
Stage 2: ColBERT rerank (top 10) — optional, falls back to BM25-only
↓
DeepResearchAgent (Qwen3.5 4B via Ollama)
Multi-hop loop (up to 5 turns):
think → knowledge_search → refine → knowledge_search → ... → synthesize
Output: cited research report with source attribution
```
### CLI Command: `jarvis deep-research-setup`
Interactive CLI command that:
1. Auto-detects available local sources (checks if NoteStore.sqlite, chat.db exist; prompts for Obsidian vault path or skips if none)
2. Confirms with user which sources to connect
3. Syncs each source via SyncEngine + IngestionPipeline into ~/.openjarvis/knowledge.db
4. Prints summary (sources, document count, chunk count)
5. Drops user into `jarvis chat` with DeepResearchAgent configured
### LLM Configuration
- Engine: Ollama
- Model: qwen3.5:4b (pulled via `ollama pull qwen3.5:4b`)
- Agent: DeepResearchAgent with knowledge_search + think tools
- Expected latency: fast token generation on Apple Silicon, main cost is ColBERT model loading on first query
### API Router Fix
The `connectors_router.py` file exists with all endpoints but was never registered in the FastAPI app. Fix: add `app.include_router(create_connectors_router())` in `app.py` or `api_routes.py`.
## Test Plan
### Retrieval Quality Checks
1. Name search spanning Apple Notes + iMessage (person who appears in both)
2. Topical search across Obsidian vault
3. Time-bounded query ("what was I working on last week")
### Agent Quality Checks
1. Cross-source research query requiring multi-hop
2. Check that citations point back to real source documents
3. Check that agent uses multiple search refinements (not just one query)
### Wizard Smoke Test
1. `jarvis serve` → open localhost in browser
2. Walk through wizard: pick sources → connect → ingest → ready screen
3. Verify sync status polling shows progress
4. Verify "Ready" screen appears with suggestion queries
## Risks
| Risk | Impact | Mitigation |
|------|--------|------------|
| 4B model too small for multi-hop reasoning | Agent produces shallow one-hop reports | Swap to larger Ollama model or cloud engine |
| 52K iMessage chunks slow to ingest | Long initial setup time | Show progress, allow incremental use |
| ColBERT model loading slow on first query | Bad first-query UX | Fall back to BM25-only, lazy load ColBERT |
| Apple Notes protobuf extraction still garbled for some notes | Missing or corrupted content | Graceful degradation, skip notes with no extractable text |
docs/superpowers/specs/2026-03-26-outlook-connector-and-token-sources-design.md
-123
View File
@@ -1,123 +0,0 @@
# Outlook Connector + Token-Based Source Integration
## Goal
Generalize the Gmail IMAP connector to support Outlook/Microsoft 365, and extend `jarvis deep-research-setup` to detect and connect token-based sources (Slack, Notion, Granola, Gmail IMAP, Outlook) alongside the existing local sources.
## Scope
**In scope:**
- Generalize `gmail_imap.py` to accept an `imap_host` parameter (default `imap.gmail.com`)
- Create `outlook.py` as a thin subclass with `outlook.office365.com` default
- Add `detect_token_sources()` to `deep_research_setup_cmd.py` — scans `~/.openjarvis/connectors/*.json` for already-connected token-based sources
- Add interactive prompts in `deep-research-setup` to connect new token-based sources (Slack, Notion, Granola, Gmail IMAP, Outlook)
- Extend `_instantiate_connector()` to handle all connector types
- Connect and ingest all sources in one unified flow
**Out of scope:**
- Google OAuth token exchange (Drive, Calendar, Contacts still blocked)
- Microsoft Graph API
- Channel plugins (Slack bot, etc.)
## Architecture
### IMAP Generalization
`GmailIMAPConnector` gains one new parameter:
```python
class GmailIMAPConnector(BaseConnector):
_default_imap_host = "imap.gmail.com"
def __init__(self, ..., imap_host: str = "") -> None:
self._imap_host = imap_host or self._default_imap_host
```
The `sync()` method changes from `imaplib.IMAP4_SSL("imap.gmail.com")` to `imaplib.IMAP4_SSL(self._imap_host)`.
### Outlook Connector
A thin subclass in `outlook.py`:
```python
@ConnectorRegistry.register("outlook")
class OutlookConnector(GmailIMAPConnector):
connector_id = "outlook"
display_name = "Outlook / Microsoft 365"
_default_imap_host = "outlook.office365.com"
def __init__(self, email_address="", app_password="", credentials_path="", *, max_messages=500):
super().__init__(email_address, app_password,
credentials_path or str(DEFAULT_CONFIG_DIR / "connectors" / "outlook.json"),
max_messages=max_messages)
def auth_url(self) -> str:
return "https://account.microsoft.com/security"
def sync(self, *, since=None, cursor=None) -> Iterator[Document]:
# Delegate to parent but override source/doc_id prefix
for doc in super().sync(since=since, cursor=cursor):
doc.source = "outlook"
doc.doc_id = doc.doc_id.replace("gmail:", "outlook:", 1)
yield doc
```
### Token Source Detection in deep-research-setup
New function `detect_token_sources()`:
```python
_TOKEN_SOURCES = [
{"connector_id": "gmail_imap", "display_name": "Gmail (IMAP)", "creds_file": "gmail_imap.json"},
{"connector_id": "outlook", "display_name": "Outlook", "creds_file": "outlook.json"},
{"connector_id": "slack", "display_name": "Slack", "creds_file": "slack.json"},
{"connector_id": "notion", "display_name": "Notion", "creds_file": "notion.json"},
{"connector_id": "granola", "display_name": "Granola", "creds_file": "granola.json"},
]
```
For each: check if `~/.openjarvis/connectors/{creds_file}` exists with valid content. If so, include it as an already-connected source.
### Interactive Connect Flow
For sources not yet connected, `deep-research-setup` offers to connect them:
```
Detected Sources:
Apple Notes ready (local)
iMessage ready (local)
Available to Connect:
Gmail (IMAP) not connected
Outlook not connected
Slack not connected
Notion not connected
Granola not connected
Connect additional sources? [y/N]: y
Which source? [gmail_imap/outlook/slack/notion/granola]: slack
Paste your Slack token (xoxb-... or xoxe-...): xoxe-1-...
Slack: connected!
Connect another? [y/N]: y
...
```
Each connector's `handle_callback()` saves the credential. Then all connected sources are ingested together.
### Connector Credential Formats
| Connector | Credential Format | `handle_callback` Input |
|-----------|------------------|------------------------|
| Gmail IMAP | `{"email": "...", "password": "..."}` | `email:password` |
| Outlook | `{"email": "...", "password": "..."}` | `email:password` |
| Slack | `{"token": "..."}` | Raw token |
| Notion | `{"token": "..."}` | Raw token |
| Granola | `{"token": "..."}` | Raw token |
## Test Plan
- Test `OutlookConnector` inherits IMAP behavior with fake IMAP server (same pattern as gmail_imap tests)
- Test `detect_token_sources()` finds connected sources from credential files
- Test `detect_token_sources()` skips sources with missing/empty credential files
- Test interactive connect saves credentials correctly
docs/superpowers/specs/2026-03-27-channels-and-messaging-tabs-design.md
-85
View File
@@ -1,85 +0,0 @@
# Channels + Messaging Tabs — Unified Agent Setup
## Goal
Replace the current disconnected setup experience with two new tabs on every agent detail page: **Channels** (data sources the agent can search) and **Messaging** (ways to talk to the agent from your phone). Clicking "+ Add" on an unconnected source opens step-by-step setup inline.
## Tab Structure
```
Overview | Interact | Channels | Messaging | Tasks | Memory | Learning | Logs
```
### Channels Tab
Shows all data source connectors with their connection status and chunk counts.
**Connected sources** — green border, checkmark, chunk count:
- Gmail (1,076 chunks) ✓
- iMessage (52,956 chunks) ✓
- Granola (1,144 chunks) ✓
- etc.
**Not connected** — dashed border, "+ Add" button:
- Google Drive — Not connected [+ Add]
- Calendar — Not connected [+ Add]
- Outlook — Not connected [+ Add]
Clicking "+ Add" expands inline with the StepByStepPanel (per-connector instructions, links to settings pages, input fields for credentials). Same flow currently in the setup wizard but embedded in the tab.
**Data shown per connected source:**
- Icon + name
- Chunk count (from KnowledgeStore)
- Connection status
- Disconnect option (on hover/click)
### Messaging Tab
Shows ways to reach the agent from your phone or other platforms.
**Active channels** — green border, clear instructions:
- iMessage: "Text +1 (408) 981-9553 from your iPhone" [Active]
- Slack: "DM @jarvis in your workspace" [Active]
**Not set up** — dashed border, "Set Up" button:
- WhatsApp — Not set up [Set Up]
- SMS (Twilio) — Not set up [Set Up]
The key UX principle: **tell the user what to do, don't ask them for config.** Active channels show the action ("Text this number"), not the plumbing ("Enter phone number to monitor").
### What Changes vs Current
| Before | After |
|--------|-------|
| Separate "Channels" tab with confusing "phone number to monitor" prompt | **Messaging** tab with clear "Text this number" instructions |
| Setup wizard only during onboarding, hidden after | **Channels** tab shows all sources with inline "+ Add" |
| No way to see connected data sources on agent page | **Channels** tab shows all with chunk counts |
| Connector setup disconnected from agent | Unified on the agent page |
### What Does NOT Change
- Overview tab (stats, config, instruction)
- Interact tab (chat)
- Tasks, Memory, Learning, Logs tabs
- Backend API endpoints (already exist)
- Connector sync logic
- iMessage daemon / ChannelBridge / webhook routes
### Backend Requirements
One new endpoint needed:
`GET /v1/connectors/status` — returns all connectors with connection status AND chunk counts from KnowledgeStore. The Channels tab calls this to populate the grid.
Everything else (bind/unbind channel, connect/disconnect source, sync status) already exists.
## Test Plan
- Channels tab shows connected sources with chunk counts
- Channels tab shows unconnected sources with "+ Add"
- Clicking "+ Add" shows step-by-step instructions inline
- Completing setup adds the source and starts ingestion
- Messaging tab shows active channels with clear action text
- Messaging tab shows "Set Up" for inactive channels
- Setting up iMessage starts the daemon
- Both tabs appear on ALL agents, not just DeepResearch
docs/superpowers/specs/2026-03-27-channels-tab-and-connector-setup-design.md
-120
View File
@@ -1,120 +0,0 @@
# Channels Tab + Seamless Connector Setup
## Goal
Add a Channels tab to every agent for bidirectional messaging (iMessage, Slack, WhatsApp, SMS), and build per-connector step-by-step setup guides in the wizard with one-click OAuth where available.
## Sub-project 1: Channels Tab on Every Agent
### What it does
A new "Channels" tab on the agent detail page showing messaging interfaces to reach the agent. Users can connect/disconnect channels from the UI. Appears on ALL agents.
### Frontend
Add a 7th tab "Channels" to the agent detail view in `AgentsPage.tsx`. Content:
- Header: "Talk to this agent from" + "+ Add Channel" button
- Channel list: iMessage, Slack, WhatsApp, SMS (Twilio)
- Each channel shows: icon, name, description, status badge (Active/Not connected)
- Connected channels expand to show: monitored contact/workspace, instructions, Stop/Settings buttons
- Disconnected channels show a "Connect" button
**"+ Add Channel" / "Connect" flow:**
- Opens a modal/inline form specific to the channel type:
- **iMessage:** prompt for phone number or contact to monitor → starts daemon
- **Slack:** prompt for bot token → registers webhook
- **WhatsApp:** prompt for access token + phone number ID → registers webhook
- **SMS:** prompt for Twilio account SID + auth token + phone number → registers webhook
### Backend
- `POST /v1/managed-agents/{id}/channels` — bind a channel (creates binding + starts daemon/webhook)
- `DELETE /v1/managed-agents/{id}/channels/{binding_id}` — unbind (stops daemon/webhook)
- `AgentManager.add_channel_binding()` already exists — wire it to the new endpoint
- For iMessage: the endpoint starts/stops the `imessage_daemon` via the existing `run_daemon`/`stop_daemon` functions
- For Slack/WhatsApp/SMS: the endpoint updates `ChannelBridge` channel config
### No new daemon code
`imessage_daemon.py`, `ChannelBridge`, `webhook_routes.py`, and `channels_cmd.py` already exist. This sub-project wires them to the frontend UI.
---
## Sub-project 2: Seamless Connector Setup
### What it does
Replace the current generic "paste token" connect flow with per-connector step-by-step guides. Two patterns:
**Pattern A — Step-by-step guide (for token/password auth):**
Numbered steps with direct links to settings pages, then paste credentials.
**Pattern B — One-click OAuth + manual fallback:**
Primary "Authorize" button opens the service in browser. Manual token paste as secondary option.
### Per-connector flows
| Connector | Pattern | UX |
|-----------|---------|-----|
| Gmail IMAP | A | Steps: enable 2FA → generate app password → paste email:password |
| Outlook | A | Steps: enable 2FA → generate app password → paste email:password |
| Slack | B | "Open Slack App Settings" button → paste bot token (xoxb-...) |
| Notion | B | "Open Notion Integrations" button → paste token (ntn_...) → remind to share pages |
| Granola | A | Steps: open Granola app → Settings → API → paste key (grn_...) |
| Google Drive | B | "Authorize" opens OAuth consent URL → localhost callback captures code → exchanges for tokens |
| Google Calendar | B | Same as Drive (shared OAuth client, different scope) |
| Google Contacts | B | Same as Drive (shared OAuth client, different scope) |
| Obsidian | Filesystem | Path input field |
| Apple Notes | Auto | Check Full Disk Access, auto-connect |
| iMessage | Auto | Check Full Disk Access, auto-connect |
### Google OAuth proper flow (new)
The current Gmail OAuth connector saves the raw auth code without exchanging it. Fix:
1. Add `OAuthCallbackServer` — a tiny localhost HTTP server at `http://localhost:8789/callback` that:
- Waits for Google to redirect with `?code=...`
- Exchanges the code for access_token + refresh_token via `POST https://oauth2.googleapis.com/token`
- Stores tokens in `~/.openjarvis/connectors/{connector}.json`
- Returns a "Success! You can close this tab" page to the browser
2. Update `oauth.py` with `exchange_google_token(code, client_id, client_secret, redirect_uri)` function
3. Update `handle_callback()` in Gmail, Drive, Calendar, Contacts connectors to use the exchange flow
4. Google OAuth client ID/secret stored in `~/.openjarvis/config.toml` under `[connectors.google]` — user provides their own from Google Cloud Console
### Frontend changes
In `SourceConnectFlow.tsx`, replace the generic OAuth panel with connector-specific panels:
- Each connector gets its own instructions component
- Step-by-step connectors show numbered cards with links
- OAuth connectors show the "Authorize" button + manual fallback input
- All connectors show a "Read-only access · No data leaves your device" badge
### What does NOT change
- SourcePicker (already lists all connectors)
- IngestDashboard (already shows progress)
- ReadyScreen (already shows suggestions)
- Backend connector sync logic (already works)
- KnowledgeStore / ingestion pipeline
## Test Plan
### Channels Tab
- Channels tab appears on all agents (not just DeepResearch)
- Binding iMessage channel starts daemon and shows "Active" status
- Unbinding stops daemon
- Channel status updates in real-time
- Multiple agents can bind different channels
### Connector Setup
- Gmail IMAP: step-by-step flow → paste credentials → connected
- Notion: one-click opens integration page → paste token → connected
- Slack: paste bot token → connected
- Google Drive: OAuth consent URL opens → localhost callback exchanges token → connected
- Setup wizard completes with all connected sources ingested
- Read-only guarantee maintained across all connectors
docs/superpowers/specs/2026-03-27-personal-deep-research-template-design.md
-162
View File
@@ -1,162 +0,0 @@
# Personal DeepResearch Agent Template — Desktop + Browser + CLI
## Goal
Register "Personal Deep Research" as a default agent template so users can create it from the Agents tab in the desktop/browser app, chat with it via streaming SSE, and use it from the CLI. All read-only — no writes to any connector.
## Scope
**In scope:**
- New template TOML: `personal_deep_research.toml`
- Wire DeepResearch tools in `_stream_managed_agent()` when `agent_type == "deep_research"`
- Ensure KnowledgeStore at `~/.openjarvis/knowledge.db` is loaded when the agent runs
- Verify end-to-end: create from template in Agents tab → send message → get streamed cited report
**Out of scope:**
- Google OAuth (Sub-project 2)
- Channel listeners (Sub-project 3)
- New frontend components (existing Agents tab + chat UI work as-is)
## Architecture
### Template File
`src/openjarvis/agents/templates/personal_deep_research.toml`:
```toml
[template]
id = "personal_deep_research"
name = "Personal Deep Research"
description = "Search across your emails, messages, meeting notes, and documents with multi-hop retrieval and cited reports. Uses BM25 keyword search, SQL aggregation, and LM-powered semantic scanning."
agent_type = "deep_research"
schedule_type = "manual"
tools = ["knowledge_search", "knowledge_sql", "scan_chunks", "think"]
max_turns = 8
temperature = 0.3
max_tokens = 4096
```
This is auto-discovered by `AgentManager.list_templates()` and appears in the Agents tab wizard alongside the existing templates (code_reviewer, inbox_triager, research_monitor).
### Server-Side Tool Wiring
In `_stream_managed_agent()` (agent_manager_routes.py), when the agent type is `deep_research`, build the tools from the shared KnowledgeStore:
```python
if agent_record["agent_type"] == "deep_research":
from openjarvis.connectors.store import KnowledgeStore
from openjarvis.connectors.retriever import TwoStageRetriever
from openjarvis.tools.knowledge_search import KnowledgeSearchTool
from openjarvis.tools.knowledge_sql import KnowledgeSQLTool
from openjarvis.tools.scan_chunks import ScanChunksTool
from openjarvis.tools.think import ThinkTool
store = KnowledgeStore() # defaults to ~/.openjarvis/knowledge.db
retriever = TwoStageRetriever(store)
model = agent_record.get("config", {}).get("model") or engine_model
tools = [
KnowledgeSearchTool(retriever=retriever),
KnowledgeSQLTool(store=store),
ScanChunksTool(store=store, engine=engine, model=model),
ThinkTool(),
]
```
These tools are passed to `DeepResearchAgent(engine=engine, model=model, tools=tools)` when instantiating the agent for this request.
### Flow
```
User clicks "Personal Deep Research" template in Agents tab
→ POST /v1/managed-agents {template_id: "personal_deep_research", name: "My Research"}
→ AgentManager.create_from_template() → stores agent record in SQLite
User types question in chat
→ POST /v1/managed-agents/{id}/messages {content: "...", stream: true}
→ _stream_managed_agent() detects agent_type == "deep_research"
→ Builds 4 tools from KnowledgeStore
→ Instantiates DeepResearchAgent with tools
→ agent.run(query) → multi-hop tool calls
→ Streams response as SSE word-by-word
→ Frontend renders streamed markdown with citations
```
### CLI Alias
Add `jarvis research` as a convenience alias that runs `jarvis deep-research-setup`. Same behavior, shorter command.
### Multi-Channel Messaging — iMessage, Slack, WhatsApp, SMS
Users can message their DeepResearch agent from their phone via iMessage, Slack, WhatsApp, or SMS. This builds on the `ChannelBridge` + webhook architecture from PR #78 (open-jarvis/OpenJarvis#78).
**Three options for iMessage (user picks during setup):**
1. **AppleScript daemon (free, no external services)** — Background daemon on the Mac polls `chat.db` for new messages in a designated conversation, routes to agent, responds via AppleScript. Requires Mac to be running.
2. **BlueBubbles (free, self-hosted)** — Install BlueBubbles on the Mac, which provides a REST API + webhook for iMessage. PR #78 already has webhook handler at `/webhooks/bluebubbles`. More reliable than AppleScript, supports media.
3. **Sendblue (paid cloud API)** — Cloud iMessage API, no Mac required for sending. Works from any server. Costs money but no local dependencies. We'd add a Sendblue webhook handler alongside the existing BlueBubbles one.
**Architecture (from PR #78):**
```
Phone sends message
→ Webhook hits /webhooks/{twilio|bluebubbles|whatsapp}
→ ChannelBridge.handle_incoming(sender, content, channel_type)
→ SessionStore tracks per-sender conversation
→ Routes to DeepResearchAgent.run(query)
→ Response sent back via channel adapter (same channel)
```
For the AppleScript daemon (option 1), the flow is:
```
iPhone sends iMessage → Mac Messages.app → chat.db
→ iMessageDaemon polls chat.db (read-only)
→ ChannelBridge.handle_incoming(sender, content, "imessage")
→ DeepResearchAgent.run(query)
→ AppleScript sends response via Messages.app
→ Response appears on iPhone
```
**What we build (from PR #78 + new):**
- Merge/rebase PR #78's `ChannelBridge`, `SessionStore`, `webhook_routes`, `auth_middleware`
- Add AppleScript-based iMessage daemon as an alternative to BlueBubbles
- Add `jarvis channels setup` CLI command to configure channel(s)
- Add `jarvis channels start` / `stop` / `status` for daemon lifecycle
- Wire `ChannelBridge` to route to DeepResearchAgent instead of generic `JarvisSystem.ask()`
**Finding the agent address (shown in CLI + Desktop + Browser):**
| Channel | How user finds the address |
|---------|---------------------------|
| iMessage (AppleScript) | `jarvis channels status` → "iMessage: listening on [chat name]" |
| iMessage (BlueBubbles) | `jarvis channels status` → "iMessage: via BlueBubbles at [url]" |
| Slack | Agent detail → Channels tab → shows Slack workspace + channel |
| WhatsApp | Agent detail → Channels tab → shows WhatsApp number |
| SMS (Twilio) | Agent detail → Channels tab → shows Twilio phone number |
**Read-only guarantee:** All channel adapters only READ incoming messages and SEND responses. No channel adapter modifies, deletes, or marks messages as read. The iMessage daemon reads chat.db in read-only mode (`?mode=ro`). Responses go through Messages.app (AppleScript) or BlueBubbles API, not by writing to chat.db.
## What Does NOT Change
- Frontend Agents tab UI (already supports templates)
- Frontend chat components (already support SSE streaming)
- Agent CRUD endpoints (already work)
- DeepResearchAgent class (already has all 4 tools + prompt)
- KnowledgeStore / ingestion pipeline (already populated)
- iMessage connector (read-only data source — separate from the iMessage agent channel)
## Test Plan
- Template appears in `GET /v1/templates` response
- Creating from template succeeds via `POST /v1/managed-agents`
- Sending a message with `stream: true` returns SSE chunks
- Agent uses knowledge_search/knowledge_sql/scan_chunks/think tools
- Response includes citations
- `jarvis research` CLI alias works
- iMessage daemon detects new messages in designated chat
- iMessage daemon sends response via AppleScript
- `jarvis channels status` shows correct state per channel
- ChannelBridge routes incoming webhook to DeepResearchAgent
- Webhook endpoints return correct HTTP responses (TwiML for Twilio, 200 for BlueBubbles/WhatsApp)
- SessionStore tracks per-sender conversations across channels
frontend/src/components/Chat/AudioPlayer.tsx
+119
View File
@@ -0,0 +1,119 @@
import { useRef, useState, useEffect, useCallback } from 'react';
import { Play, Pause, Volume2 } from 'lucide-react';
interface AudioPlayerProps {
src: string;
}
function formatTime(seconds: number): string {
const m = Math.floor(seconds / 60);
const s = Math.floor(seconds % 60);
return `${m}:${s.toString().padStart(2, '0')}`;
}
export function AudioPlayer({ src }: AudioPlayerProps) {
const audioRef = useRef<HTMLAudioElement>(null);
const [playing, setPlaying] = useState(false);
const [currentTime, setCurrentTime] = useState(0);
const [duration, setDuration] = useState(0);
const toggle = useCallback(() => {
const el = audioRef.current;
if (!el) return;
if (playing) {
el.pause();
} else {
el.play();
}
setPlaying(!playing);
}, [playing]);
useEffect(() => {
const el = audioRef.current;
if (!el) return;
const onTime = () => setCurrentTime(el.currentTime);
const onMeta = () => setDuration(el.duration);
const onEnded = () => {
setPlaying(false);
setCurrentTime(0);
};
el.addEventListener('timeupdate', onTime);
el.addEventListener('loadedmetadata', onMeta);
el.addEventListener('ended', onEnded);
return () => {
el.removeEventListener('timeupdate', onTime);
el.removeEventListener('loadedmetadata', onMeta);
el.removeEventListener('ended', onEnded);
};
}, []);
const progress = duration > 0 ? (currentTime / duration) * 100 : 0;
const seek = (e: React.MouseEvent<HTMLDivElement>) => {
const el = audioRef.current;
if (!el || !duration) return;
const rect = e.currentTarget.getBoundingClientRect();
const pct = (e.clientX - rect.left) / rect.width;
el.currentTime = pct * duration;
};
return (
<div
className="flex items-center gap-3 px-4 py-3 rounded-xl mb-3"
style={{
background: 'var(--color-surface, #f5f5f5)',
border: '1px solid var(--color-border, #e0e0e0)',
}}
>
<audio ref={audioRef} src={src} preload="metadata" />
<button
onClick={toggle}
className="flex items-center justify-center w-9 h-9 rounded-full transition-colors shrink-0"
style={{
background: 'var(--color-accent, #6366f1)',
color: 'white',
cursor: 'pointer',
}}
>
{playing ? <Pause size={16} /> : <Play size={16} className="ml-0.5" />}
</button>
<div className="flex flex-col gap-1.5 flex-1 min-w-0">
<div className="flex items-center gap-2">
<Volume2 size={14} style={{ color: 'var(--color-text-tertiary)' }} />
<span
className="text-xs font-medium"
style={{ color: 'var(--color-text-secondary)' }}
>
Morning Digest
</span>
</div>
<div
className="h-1.5 rounded-full cursor-pointer"
style={{ background: 'var(--color-bg-tertiary, #e0e0e0)' }}
onClick={seek}
>
<div
className="h-full rounded-full transition-all"
style={{
width: `${progress}%`,
background: 'var(--color-accent, #6366f1)',
}}
/>
</div>
<div
className="flex justify-between text-xs"
style={{ color: 'var(--color-text-tertiary)' }}
>
<span>{formatTime(currentTime)}</span>
<span>{duration > 0 ? formatTime(duration) : '--:--'}</span>
</div>
</div>
</div>
);
}
frontend/src/components/Chat/InputArea.tsx
+16 -1
View File
@@ -2,7 +2,7 @@ import { useState, useRef, useCallback, useEffect } from 'react';
import { Send, Square, Paperclip } from 'lucide-react';
import { useAppStore, generateId } from '../../lib/store';
import { streamChat } from '../../lib/sse';
import { fetchSavings } from '../../lib/api';
import { fetchSavings, getBase } from '../../lib/api';
import { MicButton } from './MicButton';
import { useSpeech } from '../../hooks/useSpeech';
import type { ChatMessage, ToolCallInfo, TokenUsage, MessageTelemetry } from '../../types';
@@ -257,12 +257,27 @@ export function InputArea() {
complexity_tier: complexity?.tier,
suggested_max_tokens: complexity?.suggested_max_tokens,
};
// Check if the response has digest audio available
let audioMeta: { url: string } | undefined;
try {
const digestRes = await fetch(`${getBase()}/api/digest`);
if (digestRes.ok) {
const digest = await digestRes.json();
if (digest.audio_available) {
audioMeta = { url: `${getBase()}/api/digest/audio` };
}
}
} catch {
// Not a digest response or server unavailable — skip
}
updateLastAssistant(
convId,
accumulatedContent,
toolCalls.length > 0 ? toolCalls : undefined,
usage,
telemetry,
audioMeta,
);
if (timerRef.current) {
clearInterval(timerRef.current);
frontend/src/components/Chat/MessageBubble.tsx
+4
View File
@@ -6,6 +6,7 @@ import remarkGfm from 'remark-gfm';
import remarkMath from 'remark-math';
import 'katex/dist/katex.min.css';
import { Copy, Check } from 'lucide-react';
import { AudioPlayer } from './AudioPlayer';
import { ToolCallCard } from './ToolCallCard';
import { XRayFooter } from './XRayFooter';
import type { ChatMessage } from '../../types';
@@ -131,6 +132,9 @@ export function MessageBubble({ message }: Props) {
</div>
)}
{/* Audio player (e.g. morning digest) */}
{message.audio?.url && <AudioPlayer src={message.audio.url} />}
{/* Assistant message */}
{cleanContent && (
<div className="prose max-w-none">
frontend/src/components/setup/SourceConnectFlow.tsx
+52 -68
View File
@@ -8,7 +8,8 @@ import {
Loader2,
} from 'lucide-react';
import { SOURCE_CATALOG } from '../../types/connectors';
import { connectSource } from '../../lib/connectors-api';
import { connectSource, getConnector } from '../../lib/connectors-api';
import { getBase } from '../../lib/api';
import type { ConnectRequest, ConnectorMeta } from '../../types/connectors';
// ---------------------------------------------------------------------------
@@ -138,89 +139,71 @@ function FilesystemPanel({
function OAuthPanel({
displayName,
authUrl,
connectorId,
onConnect,
onSkip,
isConnecting,
}: {
displayName: string;
authUrl?: string;
connectorId: string;
onConnect: (req: ConnectRequest) => void;
onSkip: () => void;
isConnecting: boolean;
}) {
const [token, setToken] = useState('');
const [phase, setPhase] = useState<'start' | 'paste'>('start');
const [waiting, setWaiting] = useState(false);
const openBrowser = () => {
if (authUrl) {
window.open(authUrl, '_blank');
}
setPhase('paste');
const startOAuth = () => {
// Open the server's OAuth start endpoint which redirects to the provider
const oauthUrl = `${getBase()}/v1/connectors/${encodeURIComponent(connectorId)}/oauth/start`;
window.open(oauthUrl, '_blank', 'width=600,height=700');
setWaiting(true);
// Poll for connection status
const interval = setInterval(async () => {
try {
const info = await getConnector(connectorId);
if (info.connected) {
clearInterval(interval);
setWaiting(false);
onConnect({});
}
} catch {
// ignore polling errors
}
}, 2000);
// Stop polling after 3 minutes
setTimeout(() => {
clearInterval(interval);
setWaiting(false);
}, 180000);
};
return (
<div className="flex flex-col gap-4">
{phase === 'start' ? (
<>
<p className="text-sm" style={{ color: 'var(--color-text-secondary)' }}>
Authorize OpenJarvis to access your {displayName} account.
</p>
<button
onClick={openBrowser}
className="flex items-center gap-2 px-4 py-2 rounded-lg text-sm font-medium self-start transition-all"
style={{
background: 'var(--color-accent)',
color: 'white',
}}
>
<ExternalLink size={14} />
Open in browser
</button>
</>
<p className="text-sm" style={{ color: 'var(--color-text-secondary)' }}>
{waiting
? `Waiting for ${displayName} authorization... Complete it in the browser window.`
: `Connect your ${displayName} account with one click.`}
</p>
{waiting ? (
<div className="flex items-center gap-2 text-sm" style={{ color: 'var(--color-accent)' }}>
<Loader2 size={16} className="animate-spin" />
Waiting for authorization...
</div>
) : (
<>
<p className="text-sm" style={{ color: 'var(--color-text-secondary)' }}>
After authorizing, paste the token or code below.
</p>
<textarea
value={token}
onChange={(e) => setToken(e.target.value)}
rows={3}
placeholder="Paste auth token or code here..."
className="w-full px-3 py-2 rounded-lg text-sm outline-none resize-none font-mono"
style={{
background: 'var(--color-surface)',
border: '1px solid var(--color-border)',
color: 'var(--color-text)',
}}
/>
<div className="flex gap-2">
<button
onClick={() => onConnect({ token, code: token })}
disabled={!token.trim() || isConnecting}
className="px-4 py-2 rounded-lg text-sm font-medium flex items-center gap-2 transition-all"
style={{
background: token.trim() ? 'var(--color-accent)' : 'var(--color-bg-tertiary)',
color: token.trim() ? 'white' : 'var(--color-text-tertiary)',
cursor: token.trim() && !isConnecting ? 'pointer' : 'not-allowed',
}}
>
{isConnecting && <Loader2 size={14} className="animate-spin" />}
Confirm
</button>
<button
onClick={() => setPhase('start')}
className="px-4 py-2 rounded-lg text-sm font-medium transition-all"
style={{
background: 'var(--color-surface)',
border: '1px solid var(--color-border)',
color: 'var(--color-text-secondary)',
}}
>
Back
</button>
</div>
</>
<button
onClick={startOAuth}
className="flex items-center gap-2 px-4 py-2 rounded-lg text-sm font-medium self-start transition-all"
style={{
background: 'var(--color-accent)',
color: 'white',
}}
>
<ExternalLink size={14} />
Connect {displayName}
</button>
)}
<button
onClick={onSkip}
@@ -589,6 +572,7 @@ export function SourceConnectFlow({
<OAuthPanel
displayName={activeCard.display_name}
authUrl={undefined}
connectorId={activeEntry.id}
onConnect={(req) => handleConnect(activeEntry.id, req)}
onSkip={() => handleSkip(activeEntry.id)}
isConnecting={activeEntry.state === 'connecting'}
frontend/src/lib/store.ts
+3
View File
@@ -149,6 +149,7 @@ interface AppState {
toolCalls?: ToolCallInfo[],
usage?: TokenUsage,
telemetry?: MessageTelemetry,
audio?: { url: string },
) => void;
setStreamState: (state: Partial<StreamState>) => void;
resetStream: () => void;
@@ -337,6 +338,7 @@ export const useAppStore = create<AppState>((set, get) => {
toolCalls?: ToolCallInfo[],
usage?: TokenUsage,
telemetry?: MessageTelemetry,
audio?: { url: string },
) => {
const store = loadConversations();
const conv = store.conversations[conversationId];
@@ -347,6 +349,7 @@ export const useAppStore = create<AppState>((set, get) => {
if (toolCalls) lastMsg.toolCalls = toolCalls;
if (usage) lastMsg.usage = usage;
if (telemetry) lastMsg.telemetry = telemetry;
if (audio) lastMsg.audio = audio;
conv.updatedAt = Date.now();
saveConversations(store);
set({ messages: [...conv.messages] });
frontend/src/types/index.ts
+1
View File
@@ -69,6 +69,7 @@ export interface ChatMessage {
toolCalls?: ToolCallInfo[];
usage?: TokenUsage;
telemetry?: MessageTelemetry;
audio?: { url: string };
}
export interface Conversation {
scripts/oauth_all.py
+341
View File
@@ -0,0 +1,341 @@
#!/usr/bin/env python3
"""Run OAuth flows for Google, Strava, and Spotify in sequence.
Usage:
uv run python scripts/oauth_all.py [--google] [--strava] [--spotify]
uv run python scripts/oauth_all.py # runs all three
"""
from __future__ import annotations
import argparse
import json
import os
import webbrowser
from http.server import BaseHTTPRequestHandler, HTTPServer
from pathlib import Path
from typing import Any, Dict, List, Optional
from urllib.parse import parse_qs, urlencode, urlparse
import httpx
CONFIG_DIR = Path.home() / ".openjarvis" / "connectors"
CONFIG_DIR.mkdir(parents=True, exist_ok=True)
CALLBACK_PORT = 8789
REDIRECT_URI = f"http://localhost:{CALLBACK_PORT}/callback"
# ── Credentials (read from stored files, then env vars) ──────────────────────
def _load_creds(filename: str) -> Dict[str, str]:
"""Load client_id/client_secret from a stored credential file."""
path = CONFIG_DIR / filename
if path.exists():
try:
data = json.loads(path.read_text(encoding="utf-8"))
return {
"client_id": data.get("client_id", ""),
"client_secret": data.get("client_secret", ""),
}
except (json.JSONDecodeError, OSError):
pass
return {"client_id": "", "client_secret": ""}
_google = _load_creds("google.json")
GOOGLE_CLIENT_ID = os.environ.get("OPENJARVIS_GOOGLE_CLIENT_ID", "") or _google["client_id"]
GOOGLE_CLIENT_SECRET = os.environ.get("OPENJARVIS_GOOGLE_CLIENT_SECRET", "") or _google["client_secret"]
_strava = _load_creds("strava.json")
STRAVA_CLIENT_ID = os.environ.get("OPENJARVIS_STRAVA_CLIENT_ID", "") or _strava["client_id"]
STRAVA_CLIENT_SECRET = os.environ.get("OPENJARVIS_STRAVA_CLIENT_SECRET", "") or _strava["client_secret"]
_spotify = _load_creds("spotify.json")
SPOTIFY_CLIENT_ID = os.environ.get("OPENJARVIS_SPOTIFY_CLIENT_ID", "") or _spotify["client_id"]
SPOTIFY_CLIENT_SECRET = os.environ.get("OPENJARVIS_SPOTIFY_CLIENT_SECRET", "") or _spotify["client_secret"]
# ── Generic OAuth helpers ────────────────────────────────────────────────────
def _wait_for_code(port: int = CALLBACK_PORT, timeout: int = 120) -> str:
"""Start a localhost server and wait for the OAuth callback with ?code=."""
auth_code: List[str] = []
error: List[str] = []
class Handler(BaseHTTPRequestHandler):
def do_GET(self) -> None:
params = parse_qs(urlparse(self.path).query)
if "code" in params:
auth_code.append(params["code"][0])
self.send_response(200)
self.send_header("Content-Type", "text/html")
self.end_headers()
self.wfile.write(
b"<html><body><h2>Success!</h2>"
b"<p>You can close this tab.</p></body></html>"
)
else:
error.append(params.get("error", ["unknown"])[0])
self.send_response(400)
self.send_header("Content-Type", "text/html")
self.end_headers()
self.wfile.write(b"<html><body><h2>Failed</h2></body></html>")
def log_message(self, *args: Any) -> None:
pass
server = HTTPServer(("127.0.0.1", port), Handler)
server.timeout = timeout
while not auth_code and not error:
server.handle_request()
server.server_close()
if error:
raise RuntimeError(f"OAuth error: {error[0]}")
if not auth_code:
raise RuntimeError("OAuth timed out")
return auth_code[0]
def _save(path: Path, data: Dict[str, Any]) -> None:
path.write_text(json.dumps(data, indent=2), encoding="utf-8")
path.chmod(0o600)
print(f" Saved tokens to {path}")
# ── Google ───────────────────────────────────────────────────────────────────
def do_google() -> None:
print("\n=== Google OAuth (Drive, Calendar, Contacts, Gmail, Tasks) ===")
scopes = [
"openid", "email", "profile",
"https://www.googleapis.com/auth/drive.readonly",
"https://www.googleapis.com/auth/calendar.readonly",
"https://www.googleapis.com/auth/contacts.readonly",
"https://www.googleapis.com/auth/gmail.readonly",
"https://www.googleapis.com/auth/tasks.readonly",
]
url = (
"https://accounts.google.com/o/oauth2/v2/auth?"
+ urlencode({
"client_id": GOOGLE_CLIENT_ID,
"redirect_uri": REDIRECT_URI,
"response_type": "code",
"scope": " ".join(scopes),
"access_type": "offline",
"prompt": "consent",
})
)
print(f" Opening browser...")
webbrowser.open(url)
code = _wait_for_code()
resp = httpx.post(
"https://oauth2.googleapis.com/token",
data={
"code": code,
"client_id": GOOGLE_CLIENT_ID,
"client_secret": GOOGLE_CLIENT_SECRET,
"redirect_uri": REDIRECT_URI,
"grant_type": "authorization_code",
},
timeout=30.0,
)
resp.raise_for_status()
tokens = resp.json()
payload = {
"access_token": tokens["access_token"],
"refresh_token": tokens.get("refresh_token", ""),
"token_type": tokens.get("token_type", "Bearer"),
"expires_in": tokens.get("expires_in", 3600),
"client_id": GOOGLE_CLIENT_ID,
"client_secret": GOOGLE_CLIENT_SECRET,
}
# Save to shared + all connector-specific files
for name in ["google", "gdrive", "gcalendar", "gcontacts", "gmail", "google_tasks"]:
_save(CONFIG_DIR / f"{name}.json", payload)
print(" Google OAuth complete!")
# ── Strava ───────────────────────────────────────────────────────────────────
def do_strava() -> None:
print("\n=== Strava OAuth ===")
url = (
"https://www.strava.com/oauth/authorize?"
+ urlencode({
"client_id": STRAVA_CLIENT_ID,
"redirect_uri": REDIRECT_URI,
"response_type": "code",
"scope": "activity:read_all",
})
)
print(f" Opening browser...")
webbrowser.open(url)
code = _wait_for_code()
resp = httpx.post(
"https://www.strava.com/oauth/token",
data={
"client_id": STRAVA_CLIENT_ID,
"client_secret": STRAVA_CLIENT_SECRET,
"code": code,
"grant_type": "authorization_code",
},
timeout=30.0,
)
resp.raise_for_status()
tokens = resp.json()
payload = {
"access_token": tokens["access_token"],
"refresh_token": tokens.get("refresh_token", ""),
"client_id": STRAVA_CLIENT_ID,
"client_secret": STRAVA_CLIENT_SECRET,
}
_save(CONFIG_DIR / "strava.json", payload)
print(" Strava OAuth complete!")
# ── Spotify ──────────────────────────────────────────────────────────────────
def _make_self_signed_cert() -> tuple[str, str]:
"""Generate a temporary self-signed cert for localhost HTTPS."""
import ssl
import subprocess
import tempfile
cert_dir = Path(tempfile.mkdtemp())
key_path = str(cert_dir / "key.pem")
cert_path = str(cert_dir / "cert.pem")
subprocess.run(
[
"openssl", "req", "-x509", "-newkey", "rsa:2048",
"-keyout", key_path, "-out", cert_path,
"-days", "1", "-nodes",
"-subj", "/CN=localhost",
],
capture_output=True, check=True,
)
return cert_path, key_path
def _wait_for_code_https(port: int = 8888, timeout: int = 120) -> str:
"""Like _wait_for_code but serves over HTTPS with a self-signed cert."""
import ssl
cert_path, key_path = _make_self_signed_cert()
auth_code: List[str] = []
error: List[str] = []
class Handler(BaseHTTPRequestHandler):
def do_GET(self) -> None:
params = parse_qs(urlparse(self.path).query)
if "code" in params:
auth_code.append(params["code"][0])
self.send_response(200)
self.send_header("Content-Type", "text/html")
self.end_headers()
self.wfile.write(
b"<html><body><h2>Success!</h2>"
b"<p>You can close this tab.</p></body></html>"
)
else:
error.append(params.get("error", ["unknown"])[0])
self.send_response(400)
self.send_header("Content-Type", "text/html")
self.end_headers()
self.wfile.write(b"<html><body><h2>Failed</h2></body></html>")
def log_message(self, *args: Any) -> None:
pass
server = HTTPServer(("127.0.0.1", port), Handler)
ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
ctx.load_cert_chain(cert_path, key_path)
server.socket = ctx.wrap_socket(server.socket, server_side=True)
server.timeout = timeout
while not auth_code and not error:
server.handle_request()
server.server_close()
if error:
raise RuntimeError(f"OAuth error: {error[0]}")
if not auth_code:
raise RuntimeError("OAuth timed out")
return auth_code[0]
def do_spotify() -> None:
print("\n=== Spotify OAuth ===")
spotify_port = 8888
spotify_redirect = f"http://127.0.0.1:{spotify_port}/callback"
url = (
"https://accounts.spotify.com/authorize?"
+ urlencode({
"client_id": SPOTIFY_CLIENT_ID,
"redirect_uri": spotify_redirect,
"response_type": "code",
"scope": "user-read-recently-played",
})
)
print(f" Opening browser...")
webbrowser.open(url)
code = _wait_for_code(port=spotify_port)
import base64
auth_header = base64.b64encode(
f"{SPOTIFY_CLIENT_ID}:{SPOTIFY_CLIENT_SECRET}".encode()
).decode()
resp = httpx.post(
"https://accounts.spotify.com/api/token",
headers={"Authorization": f"Basic {auth_header}"},
data={
"code": code,
"redirect_uri": spotify_redirect,
"grant_type": "authorization_code",
},
timeout=30.0,
)
resp.raise_for_status()
tokens = resp.json()
payload = {
"access_token": tokens["access_token"],
"refresh_token": tokens.get("refresh_token", ""),
"client_id": SPOTIFY_CLIENT_ID,
"client_secret": SPOTIFY_CLIENT_SECRET,
}
_save(CONFIG_DIR / "spotify.json", payload)
print(" Spotify OAuth complete!")
# ── Main ─────────────────────────────────────────────────────────────────────
if __name__ == "__main__":
parser = argparse.ArgumentParser(description="Run OAuth flows for OpenJarvis connectors")
parser.add_argument("--google", action="store_true", help="Only Google")
parser.add_argument("--strava", action="store_true", help="Only Strava")
parser.add_argument("--spotify", action="store_true", help="Only Spotify")
args = parser.parse_args()
run_all = not (args.google or args.strava or args.spotify)
if run_all or args.google:
do_google()
if run_all or args.strava:
do_strava()
if run_all or args.spotify:
do_spotify()
print("\nDone! All OAuth flows complete.")
src/openjarvis/agents/__init__.py
+5
View File
@@ -74,6 +74,11 @@ try:
except ImportError:
pass
try:
import openjarvis.agents.morning_digest # noqa: F401
except ImportError:
pass
# Registry alias: "react" -> NativeReActAgent (for backward compat)
try:
from openjarvis.core.registry import AgentRegistry
src/openjarvis/agents/digest_store.py
+153
View File
@@ -0,0 +1,153 @@
"""DigestStore — SQLite-backed storage for pre-computed digest artifacts."""
from __future__ import annotations
import json
import sqlite3
from dataclasses import dataclass
from datetime import datetime
from pathlib import Path
from typing import Dict, List, Optional
@dataclass
class DigestArtifact:
"""A pre-computed morning digest ready for delivery."""
text: str
audio_path: Path
sections: Dict[str, str]
sources_used: List[str]
generated_at: datetime
model_used: str
voice_used: str
quality_score: float = 0.0
evaluator_feedback: str = ""
class DigestStore:
"""SQLite store for digest artifacts."""
def __init__(self, db_path: str = "") -> None:
if not db_path:
db_path = str(Path.home() / ".openjarvis" / "digest.db")
self._db_path = db_path
self._conn = sqlite3.connect(db_path, check_same_thread=False)
self._conn.execute("PRAGMA journal_mode=WAL")
self._conn.execute(
"""
CREATE TABLE IF NOT EXISTS digests (
id INTEGER PRIMARY KEY AUTOINCREMENT,
text TEXT NOT NULL,
audio_path TEXT NOT NULL,
sections TEXT NOT NULL,
sources_used TEXT NOT NULL,
generated_at TEXT NOT NULL,
model_used TEXT NOT NULL,
voice_used TEXT NOT NULL,
quality_score REAL NOT NULL DEFAULT 0.0,
evaluator_feedback TEXT NOT NULL DEFAULT ''
)
"""
)
self._migrate()
self._conn.commit()
def _migrate(self) -> None:
"""Add columns introduced after the initial schema."""
existing = {
row[1]
for row in self._conn.execute("PRAGMA table_info(digests)").fetchall()
}
if "quality_score" not in existing:
self._conn.execute(
"ALTER TABLE digests ADD COLUMN quality_score REAL NOT NULL DEFAULT 0.0"
)
if "evaluator_feedback" not in existing:
self._conn.execute(
"ALTER TABLE digests"
" ADD COLUMN evaluator_feedback TEXT NOT NULL DEFAULT ''"
)
def save(self, artifact: DigestArtifact) -> None:
"""Save a digest artifact."""
self._conn.execute(
"""
INSERT INTO digests
(text, audio_path, sections, sources_used,
generated_at, model_used, voice_used,
quality_score, evaluator_feedback)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
(
artifact.text,
str(artifact.audio_path),
json.dumps(artifact.sections),
json.dumps(artifact.sources_used),
artifact.generated_at.isoformat(),
artifact.model_used,
artifact.voice_used,
artifact.quality_score,
artifact.evaluator_feedback,
),
)
self._conn.commit()
def _row_to_artifact(self, row: tuple) -> DigestArtifact:
return DigestArtifact(
text=row[0],
audio_path=Path(row[1]),
sections=json.loads(row[2]),
sources_used=json.loads(row[3]),
generated_at=datetime.fromisoformat(row[4]),
model_used=row[5],
voice_used=row[6],
quality_score=row[7] if len(row) > 7 else 0.0,
evaluator_feedback=row[8] if len(row) > 8 else "",
)
def get_latest(self) -> Optional[DigestArtifact]:
"""Return the most recent digest, or None."""
row = self._conn.execute(
"SELECT text, audio_path, sections, sources_used,"
" generated_at, model_used, voice_used,"
" quality_score, evaluator_feedback"
" FROM digests ORDER BY id DESC LIMIT 1"
).fetchone()
if row is None:
return None
return self._row_to_artifact(row)
def get_today(self, timezone_name: str = "UTC") -> Optional[DigestArtifact]:
"""Return today's digest if it exists, or None."""
try:
from zoneinfo import ZoneInfo
today = datetime.now(ZoneInfo(timezone_name)).strftime("%Y-%m-%d")
except ImportError:
today = datetime.now().strftime("%Y-%m-%d")
row = self._conn.execute(
"SELECT text, audio_path, sections, sources_used,"
" generated_at, model_used, voice_used,"
" quality_score, evaluator_feedback"
" FROM digests WHERE generated_at LIKE ? ORDER BY id DESC LIMIT 1",
(f"{today}%",),
).fetchone()
if row is None:
return None
return self._row_to_artifact(row)
def history(self, limit: int = 10) -> List[DigestArtifact]:
"""Return the N most recent digests."""
rows = self._conn.execute(
"SELECT text, audio_path, sections, sources_used,"
" generated_at, model_used, voice_used,"
" quality_score, evaluator_feedback"
" FROM digests ORDER BY id DESC LIMIT ?",
(limit,),
).fetchall()
return [self._row_to_artifact(r) for r in rows]
def close(self) -> None:
self._conn.close()
src/openjarvis/agents/morning_digest.py
+248
View File
@@ -0,0 +1,248 @@
"""Morning Digest Agent — synthesizes a daily briefing from multiple sources.
Thin orchestrator that delegates to digest_collect (data fetching),
the LLM (narrative synthesis), and text_to_speech (audio generation).
"""
from __future__ import annotations
import json
from datetime import datetime
from pathlib import Path
from typing import Any, List, Optional
from openjarvis.agents._stubs import AgentContext, AgentResult, ToolUsingAgent
from openjarvis.agents.digest_store import DigestArtifact, DigestStore
from openjarvis.core.registry import AgentRegistry
from openjarvis.core.types import Message, Role, ToolCall
def _load_persona(persona_name: str) -> str:
"""Load a persona prompt file by name."""
search_paths = [
Path("configs/openjarvis/prompts/personas") / f"{persona_name}.md",
Path.home() / ".openjarvis" / "prompts" / "personas" / f"{persona_name}.md",
]
for p in search_paths:
if p.exists():
return p.read_text(encoding="utf-8")
return ""
@AgentRegistry.register("morning_digest")
class MorningDigestAgent(ToolUsingAgent):
"""Pre-compute a daily digest from configured data sources."""
agent_id = "morning_digest"
def __init__(self, *args: Any, **kwargs: Any) -> None:
# Extract digest-specific kwargs before passing to parent
self._persona = kwargs.pop("persona", "jarvis")
self._sections = kwargs.pop(
"sections", ["messages", "calendar", "health", "world"]
)
self._section_sources = kwargs.pop("section_sources", {})
self._timezone = kwargs.pop("timezone", "America/Los_Angeles")
self._voice_id = kwargs.pop("voice_id", "")
self._voice_speed = kwargs.pop("voice_speed", 1.0)
self._tts_backend = kwargs.pop("tts_backend", "cartesia")
self._digest_store_path = kwargs.pop("digest_store_path", "")
self._honorific = kwargs.pop("honorific", "sir")
super().__init__(*args, **kwargs)
def _build_system_prompt(self) -> str:
"""Assemble the system prompt from persona + briefing structure."""
persona_text = _load_persona(self._persona)
now = datetime.now()
honorific = getattr(self, "_honorific", "sir")
return (
f"{persona_text}\n\n"
f"Today is {now.strftime('%A, %B %d, %Y')}. "
f"The time is {now.strftime('%I:%M %p')} in {self._timezone}.\n"
f"The user's preferred honorific is: {honorific}\n\n"
"You receive structured data from the user's connected services. "
"The data has ALREADY been collected — it appears in the user "
"message. You do NOT fetch anything yourself.\n\n"
"Produce a 2-4 minute spoken briefing in DECREASING order of "
"importance:\n\n"
"1. GREETING + PRIORITIES — Open with the honorific and "
"immediately state what needs attention: overdue tasks, today's "
"deadlines, events requiring preparation. Connect related items "
"('Your rebuttals are overdue and you have a dinner at 6, so "
"I'd tackle those first').\n\n"
"2. SCHEDULE — Today's upcoming events with time context: 'You "
"have 3 hours before your next meeting.' Skip past events.\n\n"
"3. MESSAGES — Triage across ALL channels (email, texts, Slack):\n"
" - First: messages from real people needing a REPLY or DECISION\n"
" - Second: messages containing deadlines or action items\n"
" - Last: brief acknowledgment of casual threads ('Your group "
"chat has been lively but nothing requiring a response')\n"
" - SKIP automated emails, newsletters, and marketing entirely\n"
" - Quote relevant message text when it helps\n\n"
"4. HEALTH — Interpret trends, not raw numbers. 'Your sleep has "
"improved three nights running and your readiness is strong' — "
"not 'HRV 53, HR 56.' If multiple days of data, compare.\n\n"
"5. WORLD — Weather forecast, top news (AI/tech, business, "
"general). Skip if no data.\n\n"
"6. CLOSING — One forward-looking sentence with the honorific.\n\n"
"ABSOLUTE RULES (violations are unacceptable):\n"
"- ONLY facts from the data. Zero hallucination.\n"
"- NEVER mention disconnected or unavailable sources.\n"
"- NEVER state raw health numbers. Say 'your sleep was solid' "
"NOT 'heart rate 56 bpm' or 'HRV 53' or '6000 steps' or "
"'readiness 82'. Interpret, never enumerate.\n"
"- NEVER describe actions you are taking.\n"
"- Acknowledge every source that returned data, even briefly.\n"
"- No markdown, emojis, bullets, or headers.\n"
"- STRICT LIMIT: 200 words. Be concise."
)
def _resolve_sources(self) -> List[str]:
"""Get the list of connector IDs to query."""
default_source_map = {
"messages": [
"gmail",
"slack",
"google_tasks",
"imessage",
"github_notifications",
],
"calendar": ["gcalendar"],
"health": ["oura", "apple_health"],
"world": ["weather", "hackernews", "news_rss"],
"music": ["spotify", "apple_music"],
}
sources = set()
for section in self._sections:
section_sources = self._section_sources.get(
section, default_source_map.get(section, [])
)
sources.update(section_sources)
return list(sources)
def run(
self,
input: str,
context: Optional[AgentContext] = None,
**kwargs: Any,
) -> AgentResult:
self._emit_turn_start(input)
# Step 1: Collect data from connectors
sources = self._resolve_sources()
collect_call = ToolCall(
id="digest-collect-1",
name="digest_collect",
arguments=json.dumps({"sources": sources, "hours_back": 24}),
)
collect_result = self._executor.execute(collect_call)
collected_data = collect_result.content
# Step 2: Synthesize narrative via LLM
system_prompt = self._build_system_prompt()
messages = [
Message(role=Role.SYSTEM, content=system_prompt),
Message(
role=Role.USER,
content=(
f"Here is the collected data from my sources:\n\n"
f"{collected_data}\n\n"
f"Synthesize my morning briefing. Remember:\n"
f"- Priority-first, connect related items\n"
f"- For health: say 'solid', 'improving', 'dipped' "
f"— NEVER say any number (no 82, no 56, no 6000)\n"
f"- Do NOT invent reasons for health changes\n"
f"- Do NOT mention disconnected sources\n"
f"- Do NOT repeat the greeting in your closing\n"
f"- Use the honorific ONLY 2-3 times total\n"
f"- Skip notifications from the user themselves\n"
f"- STRICT LIMIT: 200-250 words maximum"
),
),
]
result = self._generate(messages)
narrative = self._strip_think_tags(result.get("content", ""))
# Step 2b: Self-evaluate and optionally regenerate
quality_score = 0.0
evaluator_feedback = ""
try:
from openjarvis.agents.digest_evaluator import DigestEvaluator
evaluator = DigestEvaluator(self._engine, self._model)
quality_score, evaluator_feedback = evaluator.evaluate(
collected_data, narrative
)
if quality_score < 7.0 and evaluator_feedback:
# Regenerate with feedback
messages.append(
Message(
role=Role.USER,
content=(
f"Your briefing scored {quality_score:.1f}/10. "
f"Feedback: {evaluator_feedback}\n"
f"Please revise the briefing addressing this feedback."
),
)
)
result = self._generate(messages)
narrative = self._strip_think_tags(result.get("content", ""))
except Exception: # noqa: BLE001
pass # Evaluator failure shouldn't block digest delivery
# Step 3: Generate audio via TTS
# Strip any markdown that slipped through (##, *, -, etc.)
import re
tts_text = re.sub(r"^#{1,6}\s+", "", narrative, flags=re.MULTILINE)
tts_text = re.sub(r"^\s*[-*•]\s+", "", tts_text, flags=re.MULTILINE)
tts_text = re.sub(r"\*{1,2}([^*]+)\*{1,2}", r"\1", tts_text)
tts_text = tts_text.strip()
tts_call = ToolCall(
id="digest-tts-1",
name="text_to_speech",
arguments=json.dumps(
{
"text": tts_text,
"voice_id": self._voice_id,
"backend": self._tts_backend,
"speed": self._voice_speed,
}
),
)
tts_result = self._executor.execute(tts_call)
audio_path = (
tts_result.metadata.get("audio_path", "") if tts_result.success else ""
)
# Step 4: Store the artifact
artifact = DigestArtifact(
text=narrative,
audio_path=Path(audio_path) if audio_path else Path(""),
sections={},
sources_used=sources,
generated_at=datetime.now(),
model_used=self._model,
voice_used=self._voice_id,
quality_score=quality_score,
evaluator_feedback=evaluator_feedback,
)
store = DigestStore(db_path=self._digest_store_path)
store.save(artifact)
store.close()
self._emit_turn_end(turns=1)
return AgentResult(
content=narrative,
tool_results=[collect_result, tts_result],
turns=1,
metadata={
"audio_path": audio_path,
"sources_used": sources,
},
)
src/openjarvis/agents/native_react.py
+2 -1
View File
@@ -194,7 +194,8 @@ class NativeReActAgent(ToolUsingAgent):
# Max turns exceeded
msg_dicts = [_message_to_dict(m) for m in messages]
return self._max_turns_result(
all_tool_results, turns,
all_tool_results,
turns,
metadata={**total_usage, "messages": msg_dicts},
)
src/openjarvis/channels/sendblue.py
+3 -9
View File
@@ -57,15 +57,11 @@ class SendBlueChannel(BaseChannel):
webhook_secret: str = "",
bus: Optional[EventBus] = None,
) -> None:
self._api_key_id = api_key_id or os.environ.get(
"SENDBLUE_API_KEY_ID", ""
)
self._api_key_id = api_key_id or os.environ.get("SENDBLUE_API_KEY_ID", "")
self._api_secret_key = api_secret_key or os.environ.get(
"SENDBLUE_API_SECRET_KEY", ""
)
self._from_number = from_number or os.environ.get(
"SENDBLUE_FROM_NUMBER", ""
)
self._from_number = from_number or os.environ.get("SENDBLUE_FROM_NUMBER", "")
self._webhook_secret = webhook_secret
self._bus = bus
self._handlers: List[ChannelHandler] = []
@@ -215,9 +211,7 @@ class SendBlueChannel(BaseChannel):
# -- internal helpers -------------------------------------------------------
def _publish_sent(
self, channel: str, content: str, conversation_id: str
) -> None:
def _publish_sent(self, channel: str, content: str, conversation_id: str) -> None:
"""Publish a CHANNEL_MESSAGE_SENT event on the bus."""
if self._bus is not None:
self._bus.publish(
src/openjarvis/channels/slack_daemon.py
+12 -5
View File
@@ -63,7 +63,10 @@ def run_slack_daemon(
engine = OllamaEngine()
tools = _build_deep_research_tools(engine=engine, model=model)
agent = DeepResearchAgent(
engine=engine, model=model, tools=tools, max_turns=5,
engine=engine,
model=model,
tools=tools,
max_turns=5,
)
logger.info("Slack daemon: agent ready with %d tools", len(tools))
@@ -143,11 +146,15 @@ def start_slack_daemon(
proc = subprocess.Popen(
[
sys.executable, "-m",
sys.executable,
"-m",
"openjarvis.channels.slack_daemon",
"--bot-token", bot_token,
"--app-token", app_token,
"--model", model,
"--bot-token",
bot_token,
"--app-token",
app_token,
"--model",
model,
],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
src/openjarvis/cli/__init__.py
+2
View File
@@ -17,6 +17,7 @@ from openjarvis.cli.config_cmd import config
from openjarvis.cli.connect_cmd import connect
from openjarvis.cli.daemon_cmd import restart, start, status, stop
from openjarvis.cli.deep_research_setup_cmd import deep_research_setup
from openjarvis.cli.digest_cmd import digest
from openjarvis.cli.doctor_cmd import doctor
from openjarvis.cli.eval_cmd import eval_group
from openjarvis.cli.feedback_cmd import feedback_group
@@ -94,6 +95,7 @@ cli.add_command(registry, "registry")
cli.add_command(config, "config")
cli.add_command(scan, "scan")
cli.add_command(connect, "connect")
cli.add_command(digest, "digest")
cli.add_command(deep_research_setup, "deep-research-setup")
cli.add_command(deep_research_setup, "research")
src/openjarvis/cli/channel_cmd.py
+1 -3
View File
@@ -138,9 +138,7 @@ def _get_channel(
import os
kwargs["api_key_id"] = os.environ.get("SENDBLUE_API_KEY_ID", "")
kwargs["api_secret_key"] = os.environ.get(
"SENDBLUE_API_SECRET_KEY", ""
)
kwargs["api_secret_key"] = os.environ.get("SENDBLUE_API_SECRET_KEY", "")
kwargs["from_number"] = os.environ.get("SENDBLUE_FROM_NUMBER", "")
sbc = getattr(config.channel, "sendblue", None)
if sbc:
src/openjarvis/cli/connect_cmd.py
+55 -8
View File
@@ -95,18 +95,66 @@ def _connect_source(registry: object, source: str, path: str = "") -> None:
)
elif auth_type == "oauth":
# OAuth connectors need browser-based auth
# OAuth connectors — auto-open browser + catch callback
from openjarvis.connectors.oauth import (
get_client_credentials,
get_provider_for_connector,
run_connector_oauth,
save_client_credentials,
)
try:
instance = connector_cls()
url = instance.auth_url()
console.print(f"[cyan]Open this URL to authorise {source}:[/cyan]")
console.print(url)
code = click.prompt("Enter the authorisation code")
instance.handle_callback(code)
if instance.is_connected():
console.print(f"[green]{source} is already connected.[/green]")
return
provider = get_provider_for_connector(source)
if provider is None:
console.print(f"[red]No OAuth provider configured for {source}.[/red]")
return
client_id, client_secret = get_client_credentials(provider)
if not client_id or not client_secret:
console.print(f"[cyan]First-time setup for {source}.[/cyan]")
console.print(
f"[yellow]Create an OAuth app at: {provider.setup_url}[/yellow]"
)
console.print(f"[dim]{provider.setup_hint}[/dim]")
client_id = click.prompt("Client ID")
client_secret = click.prompt("Client Secret")
save_client_credentials(provider, client_id, client_secret)
run_connector_oauth(source, client_id, client_secret)
console.print(f"[green]{source} authorised successfully.[/green]")
except Exception as exc: # noqa: BLE001
console.print(f"[red]OAuth flow failed for {source}: {exc}[/red]")
elif auth_type == "token":
# Token-based connectors (e.g. Oura) — prompt for personal access token
import json
from pathlib import Path
from openjarvis.connectors.oauth import save_tokens
from openjarvis.core.config import DEFAULT_CONFIG_DIR
try:
instance = connector_cls()
if instance.is_connected():
console.print(f"[green]{source} is already connected.[/green]")
return
token = click.prompt(f"Enter your {source} personal access token")
token_dir = Path(DEFAULT_CONFIG_DIR) / "connectors"
token_dir.mkdir(parents=True, exist_ok=True)
token_file = token_dir / f"{source}.json"
token_file.write_text(json.dumps({"token": token}))
save_tokens(source, {"token": token})
console.print(f"[green]{source} connected successfully.[/green]")
except Exception as exc: # noqa: BLE001
console.print(f"[red]Token setup failed for {source}: {exc}[/red]")
else:
# Generic / bridge connectors
try:
@@ -154,8 +202,7 @@ def connect(
) -> None:
"""Manage data source connections (Gmail, Obsidian, etc.)."""
# Lazy imports to avoid top-level side effects
import openjarvis.connectors.gmail # noqa: F401
import openjarvis.connectors.obsidian # noqa: F401
import openjarvis.connectors # noqa: F401 — registers all connectors
from openjarvis.core.registry import ConnectorRegistry
if list_sources:
src/openjarvis/cli/digest_cmd.py
+264
View File
@@ -0,0 +1,264 @@
"""``jarvis digest`` — display and play the morning digest."""
from __future__ import annotations
import subprocess
import threading
from typing import Optional
import click
from rich.console import Console
from rich.markdown import Markdown
from openjarvis.agents.digest_store import DigestStore
from openjarvis.core.config import DEFAULT_CONFIG_PATH, load_config
def _play_audio(audio_path: str) -> None:
"""Play audio file in background using available system player."""
players = ["ffplay -nodisp -autoexit", "aplay", "afplay", "paplay"]
for player in players:
cmd_parts = player.split() + [audio_path]
try:
subprocess.run(
cmd_parts,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=True,
)
return
except (FileNotFoundError, subprocess.CalledProcessError):
continue
def _save_digest_schedule(enabled: bool, cron: str) -> None:
"""Persist digest schedule to config.toml."""
config_path = DEFAULT_CONFIG_PATH
# Read existing TOML content (or start fresh)
content = ""
if config_path.exists():
content = config_path.read_text()
# Check if [digest] section already exists
lines = content.split("\n")
new_lines: list[str] = []
in_digest = False
digest_written = False
for line in lines:
stripped = line.strip()
# Detect start of [digest] section
if stripped == "[digest]":
in_digest = True
digest_written = True
new_lines.append("[digest]")
new_lines.append(f"enabled = {str(enabled).lower()}")
new_lines.append(f'schedule = "{cron}"')
continue
# If inside [digest], skip old enabled/schedule keys
if in_digest:
if stripped.startswith("[") and stripped != "[digest]":
in_digest = False
new_lines.append(line)
elif stripped.startswith("enabled") or stripped.startswith("schedule"):
continue
else:
new_lines.append(line)
else:
new_lines.append(line)
if not digest_written:
# Append [digest] section at the end
if new_lines and new_lines[-1].strip():
new_lines.append("")
new_lines.append("[digest]")
new_lines.append(f"enabled = {str(enabled).lower()}")
new_lines.append(f'schedule = "{cron}"')
new_lines.append("")
config_path.parent.mkdir(parents=True, exist_ok=True)
config_path.write_text("\n".join(new_lines))
def _create_scheduler_task(cron: str) -> Optional[str]:
"""Create a digest task in the TaskScheduler. Returns task ID or None."""
try:
from openjarvis.scheduler.scheduler import TaskScheduler
from openjarvis.scheduler.store import SchedulerStore
db_path = DEFAULT_CONFIG_PATH.parent / "scheduler.db"
store = SchedulerStore(db_path)
scheduler = TaskScheduler(store)
# Cancel any existing digest tasks first
for task in scheduler.list_tasks(status="active"):
if task.agent == "morning_digest":
scheduler.cancel_task(task.id)
task = scheduler.create_task(
prompt="Generate my morning digest",
schedule_type="cron",
schedule_value=cron,
agent="morning_digest",
)
store.close()
return task.id
except Exception:
return None
def _cancel_scheduler_tasks() -> int:
"""Cancel all active digest tasks. Returns count cancelled."""
try:
from openjarvis.scheduler.scheduler import TaskScheduler
from openjarvis.scheduler.store import SchedulerStore
db_path = DEFAULT_CONFIG_PATH.parent / "scheduler.db"
store = SchedulerStore(db_path)
scheduler = TaskScheduler(store)
count = 0
for task in scheduler.list_tasks(status="active"):
if task.agent == "morning_digest":
scheduler.cancel_task(task.id)
count += 1
store.close()
return count
except Exception:
return 0
@click.command("digest", help="Display and play the morning digest.")
@click.option("--text-only", is_flag=True, help="Print text without audio playback.")
@click.option("--fresh", is_flag=True, help="Re-generate the digest (skip cache).")
@click.option("--history", is_flag=True, help="Show past digests.")
@click.option("--section", type=str, default="", help="Show only a specific section.")
@click.option("--db-path", type=str, default="", help="Path to digest database.")
@click.option(
"--schedule",
type=str,
default=None,
is_eager=True,
help=(
'Set cron schedule (e.g. "0 6 * * *"), '
'"off" to disable, or empty to show status.'
),
)
def digest(
text_only: bool,
fresh: bool,
history: bool,
section: str,
db_path: str,
schedule: Optional[str],
) -> None:
"""Display and optionally play the morning digest."""
console = Console()
# Handle --schedule flag
if schedule is not None:
_handle_schedule(console, schedule)
return
store = DigestStore(db_path=db_path) if db_path else DigestStore()
if history:
past = store.history(limit=10)
if not past:
console.print("[dim]No past digests found.[/dim]")
store.close()
return
for artifact in past:
console.print(
f"[bold]{artifact.generated_at.strftime('%Y-%m-%d %H:%M')}[/bold]"
f" — {artifact.model_used} / {artifact.voice_used}"
)
console.print(artifact.text[:200] + "...\n")
store.close()
return
if fresh:
# Trigger on-demand generation
console.print("[yellow]Generating fresh digest...[/yellow]")
try:
from openjarvis.sdk import Jarvis
with Jarvis() as j:
result = j.ask("Generate my morning digest", agent="morning_digest")
console.print(Markdown(result))
except Exception as exc:
console.print(f"[red]Failed to generate digest: {exc}[/red]")
store.close()
return
# Try to load today's cached digest
artifact = store.get_today()
if artifact is None:
console.print("[dim]No digest for today. Use --fresh to generate one.[/dim]")
store.close()
return
# Display text
text = artifact.text
if section:
# Try to extract just the requested section
lines = text.split("\n")
in_section = False
section_lines = []
for line in lines:
if line.strip().lower().startswith(
f"## {section.lower()}"
) or line.strip().lower().startswith(f"# {section.lower()}"):
in_section = True
section_lines.append(line)
elif in_section and line.strip().startswith("#"):
break
elif in_section:
section_lines.append(line)
text = "\n".join(section_lines) if section_lines else text
# Play audio in background while text renders
audio_path = str(artifact.audio_path)
if not text_only and audio_path and artifact.audio_path.exists():
audio_thread = threading.Thread(
target=_play_audio, args=(audio_path,), daemon=True
)
audio_thread.start()
console.print(Markdown(text))
store.close()
def _handle_schedule(console: Console, schedule: str) -> None:
"""Handle the --schedule option logic."""
cfg = load_config()
digest_cfg = cfg.digest
if schedule == "":
# Show current schedule status
status = "enabled" if digest_cfg.enabled else "disabled"
console.print(f"[bold]Digest schedule:[/bold] {status}")
console.print(f" Cron: {digest_cfg.schedule}")
console.print(f" Timezone: {digest_cfg.timezone}")
return
if schedule.lower() == "off":
# Disable the schedule
_save_digest_schedule(enabled=False, cron=digest_cfg.schedule)
cancelled = _cancel_scheduler_tasks()
console.print("[yellow]Digest schedule disabled.[/yellow]")
if cancelled:
console.print(f" Cancelled {cancelled} scheduler task(s).")
return
# Set a new cron schedule
_save_digest_schedule(enabled=True, cron=schedule)
task_id = _create_scheduler_task(schedule)
console.print(f"[green]Digest schedule set:[/green] {schedule}")
if task_id:
console.print(f" Scheduler task created: {task_id}")
else:
console.print(
" [dim]Note: scheduler task not created "
"(start the scheduler separately).[/dim]"
)
src/openjarvis/connectors/__init__.py
+55 -2
View File
@@ -13,8 +13,11 @@ __all__ = ["Attachment", "BaseConnector", "Document", "KnowledgeStore", "SyncSta
# Auto-register built-in connectors
import openjarvis.connectors.obsidian # noqa: F401
# gmail (REST API / OAuth) is not registered — use gmail_imap instead.
# The REST API connector requires a full OAuth flow that isn't wired up yet.
try:
import openjarvis.connectors.gmail # noqa: F401
except ImportError:
pass
try:
import openjarvis.connectors.gmail_imap # noqa: F401
except ImportError:
@@ -50,6 +53,11 @@ try:
except ImportError:
pass
try:
import openjarvis.connectors.apple_music # noqa: F401
except ImportError:
pass
try:
import openjarvis.connectors.slack_connector # noqa: F401
except ImportError:
@@ -74,3 +82,48 @@ try:
import openjarvis.connectors.whatsapp # noqa: F401
except ImportError:
pass
try:
import openjarvis.connectors.oura # noqa: F401
except ImportError:
pass
try:
import openjarvis.connectors.apple_health # noqa: F401
except ImportError:
pass
try:
import openjarvis.connectors.strava # noqa: F401
except ImportError:
pass
try:
import openjarvis.connectors.spotify # noqa: F401
except ImportError:
pass
try:
import openjarvis.connectors.google_tasks # noqa: F401
except ImportError:
pass
try:
import openjarvis.connectors.weather # noqa: F401
except ImportError:
pass
try:
import openjarvis.connectors.github_notifications # noqa: F401
except ImportError:
pass
try:
import openjarvis.connectors.hackernews # noqa: F401
except ImportError:
pass
try:
import openjarvis.connectors.news_rss # noqa: F401
except ImportError:
pass
src/openjarvis/connectors/apple_health.py
+419
View File
@@ -0,0 +1,419 @@
"""Apple Health connector -- reads HealthKit SQLite DB or iPhone Health export XML.
Two data sources are tried in order:
1. HealthKit SQLite DB at ``~/Library/Health/healthdb_secure.sqlite`` (macOS
with HealthKit sync enabled).
2. Health Export XML placed by the user at
``~/.openjarvis/connectors/apple_health_export/export.xml``.
Both are local-only; no API keys are needed.
"""
from __future__ import annotations
import hashlib
import json
import logging
import sqlite3
import xml.etree.ElementTree as ET
from collections import defaultdict
from datetime import datetime, timedelta, timezone
from pathlib import Path
from typing import Any, Dict, Iterator, List, Optional
from openjarvis.connectors._stubs import BaseConnector, Document, SyncStatus
from openjarvis.core.config import DEFAULT_CONFIG_DIR
from openjarvis.core.registry import ConnectorRegistry
logger = logging.getLogger(__name__)
_DEFAULT_EXPORT_PATH = str(
DEFAULT_CONFIG_DIR / "connectors" / "apple_health_export" / "export.xml"
)
_DEFAULT_HEALTHKIT_DB_PATH = str(
Path.home() / "Library" / "Health" / "healthdb_secure.sqlite"
)
# HK record type identifiers we care about
_STEP_COUNT = "HKQuantityTypeIdentifierStepCount"
_SLEEP_ANALYSIS = "HKCategoryTypeIdentifierSleepAnalysis"
_HEART_RATE = "HKQuantityTypeIdentifierHeartRate"
_RESTING_HEART_RATE = "HKQuantityTypeIdentifierRestingHeartRate"
_ACTIVE_ENERGY = "HKQuantityTypeIdentifierActiveEnergyBurned"
def _parse_health_date(date_str: str) -> datetime:
"""Parse date strings from Apple Health export XML.
Format is typically ``2024-03-15 08:00:00 -0700``.
"""
# Strip sub-second precision if present, then parse
try:
return datetime.strptime(date_str.strip(), "%Y-%m-%d %H:%M:%S %z")
except ValueError:
# Fallback: try ISO-8601
return datetime.fromisoformat(date_str.strip())
def _day_key(dt: datetime) -> str:
"""Return ``YYYY-MM-DD`` for a datetime."""
return dt.strftime("%Y-%m-%d")
def _format_duration(seconds: float) -> str:
"""Return a human-readable duration like ``7h 23m``."""
hours = int(seconds // 3600)
minutes = int((seconds % 3600) // 60)
if hours and minutes:
return f"{hours}h {minutes:02d}m"
if hours:
return f"{hours}h"
return f"{minutes}m"
@ConnectorRegistry.register("apple_health")
class AppleHealthConnector(BaseConnector):
"""Sync health data from Apple Health (local files only)."""
connector_id = "apple_health"
display_name = "Apple Health"
auth_type = "local"
def __init__(
self,
*,
export_path: str = _DEFAULT_EXPORT_PATH,
healthkit_db_path: str = _DEFAULT_HEALTHKIT_DB_PATH,
) -> None:
self._export_path = Path(export_path)
self._healthkit_db_path = Path(healthkit_db_path)
self._status = SyncStatus()
# ------------------------------------------------------------------
# BaseConnector interface
# ------------------------------------------------------------------
def is_connected(self) -> bool:
return self._healthkit_db_path.exists() or self._export_path.exists()
def disconnect(self) -> None:
# Local connector -- nothing to revoke.
pass
def sync(
self, *, since: Optional[datetime] = None, cursor: Optional[str] = None
) -> Iterator[Document]:
"""Yield health Documents, preferring HealthKit DB over export XML."""
self._status.state = "syncing"
try:
# Try HealthKit SQLite first
if self._healthkit_db_path.exists():
try:
yield from self._sync_healthkit_db(since=since)
self._status.state = "idle"
self._status.last_sync = datetime.now()
return
except Exception:
logger.debug(
"HealthKit DB not readable, falling back to export XML",
exc_info=True,
)
# Fall back to export XML
if self._export_path.exists():
yield from self._sync_export_xml(since=since)
self._status.state = "idle"
self._status.last_sync = datetime.now()
except Exception as exc:
self._status.state = "error"
self._status.error = str(exc)
raise
def sync_status(self) -> SyncStatus:
return self._status
# ------------------------------------------------------------------
# HealthKit SQLite DB
# ------------------------------------------------------------------
def _sync_healthkit_db(
self, *, since: Optional[datetime] = None
) -> Iterator[Document]:
"""Read health data from the local HealthKit SQLite database."""
conn = sqlite3.connect(str(self._healthkit_db_path))
conn.row_factory = sqlite3.Row
try:
# Query quantity_samples joined with samples for step count,
# heart rate, active energy. The schema uses Apple's CF
# absolute time (seconds since 2001-01-01).
cf_epoch = datetime(2001, 1, 1, tzinfo=timezone.utc)
since_cf = (
(since - cf_epoch).total_seconds()
if since and since.tzinfo
else (
(since.replace(tzinfo=timezone.utc) - cf_epoch).total_seconds()
if since
else 0
)
)
# Steps
steps_by_day: Dict[str, float] = defaultdict(float)
rows = conn.execute(
"""
SELECT s.start_date, qs.quantity
FROM samples s
JOIN quantity_samples qs ON s.data_id = qs.data_id
WHERE s.data_type = 7 AND s.start_date > ?
""",
(since_cf,),
).fetchall()
for row in rows:
dt = cf_epoch.replace(tzinfo=timezone.utc) + timedelta(
seconds=row["start_date"]
)
steps_by_day[_day_key(dt)] += row["quantity"]
for day, total in sorted(steps_by_day.items()):
yield Document(
doc_id=f"apple_health-steps-{day}",
source="apple_health",
doc_type="steps",
content=json.dumps({"date": day, "steps": int(total)}),
title=f"{int(total):,} steps",
timestamp=datetime.fromisoformat(day),
metadata={"data_type": "steps", "day": day},
)
finally:
conn.close()
# ------------------------------------------------------------------
# Export XML
# ------------------------------------------------------------------
def _sync_export_xml(
self, *, since: Optional[datetime] = None
) -> Iterator[Document]:
"""Stream-parse the Apple Health export XML and yield Documents."""
steps_by_day: Dict[str, float] = defaultdict(float)
sleep_by_day: Dict[str, List[Dict[str, Any]]] = defaultdict(list)
hr_by_day: Dict[str, List[float]] = defaultdict(list)
energy_by_day: Dict[str, float] = defaultdict(float)
workouts: List[Dict[str, Any]] = []
for event, elem in ET.iterparse(str(self._export_path), events=("end",)):
tag = elem.tag
if tag == "Record":
rec_type = elem.get("type", "")
start_str = elem.get("startDate", "")
if not start_str:
elem.clear()
continue
try:
start_dt = _parse_health_date(start_str)
except (ValueError, TypeError):
elem.clear()
continue
if since and start_dt.replace(tzinfo=None) < since.replace(tzinfo=None):
elem.clear()
continue
day = _day_key(start_dt)
if rec_type == _STEP_COUNT:
val = float(elem.get("value", 0))
steps_by_day[day] += val
elif rec_type == _SLEEP_ANALYSIS:
end_str = elem.get("endDate", start_str)
try:
end_dt = _parse_health_date(end_str)
except (ValueError, TypeError):
end_dt = start_dt
sleep_by_day[day].append(
{
"start": start_str,
"end": end_str,
"value": elem.get("value", ""),
"duration_seconds": (end_dt - start_dt).total_seconds(),
}
)
elif rec_type in (_HEART_RATE, _RESTING_HEART_RATE):
try:
hr_by_day[day].append(float(elem.get("value", 0)))
except (ValueError, TypeError):
pass
elif rec_type == _ACTIVE_ENERGY:
try:
energy_by_day[day] += float(elem.get("value", 0))
except (ValueError, TypeError):
pass
elem.clear()
elif tag == "Workout":
start_str = elem.get("startDate", "")
if not start_str:
elem.clear()
continue
try:
start_dt = _parse_health_date(start_str)
except (ValueError, TypeError):
elem.clear()
continue
if since and start_dt.replace(tzinfo=None) < since.replace(tzinfo=None):
elem.clear()
continue
workouts.append(
{
"activity_type": elem.get("workoutActivityType", ""),
"duration_seconds": float(elem.get("duration", 0)),
"start": start_str,
"end": elem.get("endDate", ""),
"total_distance": elem.get("totalDistance", ""),
"total_energy_burned": elem.get("totalEnergyBurned", ""),
}
)
elem.clear()
else:
# Don't clear the root or other structural elements yet
pass
# Yield aggregated documents
yield from self._yield_step_docs(steps_by_day)
yield from self._yield_sleep_docs(sleep_by_day)
yield from self._yield_hr_docs(hr_by_day)
yield from self._yield_energy_docs(energy_by_day)
yield from self._yield_workout_docs(workouts)
# ------------------------------------------------------------------
# Document builders
# ------------------------------------------------------------------
@staticmethod
def _yield_step_docs(
steps_by_day: Dict[str, float],
) -> Iterator[Document]:
for day in sorted(steps_by_day):
total = int(steps_by_day[day])
yield Document(
doc_id=f"apple_health-steps-{day}",
source="apple_health",
doc_type="steps",
content=json.dumps({"date": day, "steps": total}),
title=f"{total:,} steps",
timestamp=datetime.fromisoformat(day),
metadata={"data_type": "steps", "day": day},
)
@staticmethod
def _yield_sleep_docs(
sleep_by_day: Dict[str, List[Dict[str, Any]]],
) -> Iterator[Document]:
for day in sorted(sleep_by_day):
entries = sleep_by_day[day]
total_seconds = sum(e["duration_seconds"] for e in entries)
yield Document(
doc_id=f"apple_health-sleep-{day}",
source="apple_health",
doc_type="sleep",
content=json.dumps(
{"date": day, "total_seconds": total_seconds, "entries": entries}
),
title=f"{_format_duration(total_seconds)} sleep",
timestamp=datetime.fromisoformat(day),
metadata={"data_type": "sleep", "day": day},
)
@staticmethod
def _yield_hr_docs(
hr_by_day: Dict[str, List[float]],
) -> Iterator[Document]:
for day in sorted(hr_by_day):
values = hr_by_day[day]
avg = round(sum(values) / len(values), 1) if values else 0
yield Document(
doc_id=f"apple_health-heart_rate-{day}",
source="apple_health",
doc_type="heart_rate",
content=json.dumps(
{
"date": day,
"avg_bpm": avg,
"min_bpm": min(values),
"max_bpm": max(values),
"samples": len(values),
}
),
title=f"Avg {avg} bpm heart rate",
timestamp=datetime.fromisoformat(day),
metadata={"data_type": "heart_rate", "day": day},
)
@staticmethod
def _yield_energy_docs(
energy_by_day: Dict[str, float],
) -> Iterator[Document]:
for day in sorted(energy_by_day):
total = round(energy_by_day[day], 1)
yield Document(
doc_id=f"apple_health-active_energy-{day}",
source="apple_health",
doc_type="active_energy",
content=json.dumps({"date": day, "kcal": total}),
title=f"{total} kcal active energy",
timestamp=datetime.fromisoformat(day),
metadata={"data_type": "active_energy", "day": day},
)
@staticmethod
def _yield_workout_docs(
workouts: List[Dict[str, Any]],
) -> Iterator[Document]:
for w in workouts:
# Friendly activity name
raw = w["activity_type"]
activity = (
raw.replace("HKWorkoutActivityType", "")
if raw.startswith("HKWorkoutActivityType")
else raw
)
# Build a descriptive title
parts = [activity]
if w.get("total_distance"):
try:
parts.append(f"{float(w['total_distance']):.1f} km")
except (ValueError, TypeError):
pass
duration_s = w.get("duration_seconds", 0)
if duration_s:
parts.append(_format_duration(float(duration_s)))
title = " -- ".join(parts) if len(parts) > 1 else activity
# Deterministic doc_id from workout content
digest = hashlib.sha256(json.dumps(w, sort_keys=True).encode()).hexdigest()[
:12
]
yield Document(
doc_id=f"apple_health-workout-{digest}",
source="apple_health",
doc_type="workout",
content=json.dumps(w),
title=title,
timestamp=datetime.fromisoformat(
_day_key(_parse_health_date(w["start"]))
),
metadata={"data_type": "workout", "activity": activity},
)
src/openjarvis/connectors/apple_music.py
+242
View File
@@ -0,0 +1,242 @@
"""Apple Music connector -- reads tracks from the local Music.app via AppleScript.
Uses ``osascript`` subprocess calls to query Music.app on macOS. No API keys
or network access required; everything stays local.
Requires macOS (``sys.platform == "darwin"``) and the Music app to be
available.
"""
from __future__ import annotations
import hashlib
import json
import logging
import subprocess
import sys
from datetime import datetime
from typing import Iterator, Optional
from openjarvis.connectors._stubs import BaseConnector, Document, SyncStatus
from openjarvis.core.registry import ConnectorRegistry
logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# AppleScript helpers
# ---------------------------------------------------------------------------
_LIBRARY_CHECK_SCRIPT = 'tell application "Music" to get name of playlist "Library"'
_TRACKS_SCRIPT = """
tell application "Music"
set output to ""
set allTracks to every track of playlist "Library"
repeat with t in allTracks
set trackName to name of t
set trackArtist to artist of t
set trackAlbum to album of t
set trackDuration to duration of t
set trackGenre to genre of t
set trackPlayCount to played count of t
try
set pd to played date of t as string
on error
set pd to "never"
end try
set sep to "|||"
set row to trackName & sep & trackArtist & sep & trackAlbum
set row to row & sep & trackDuration & sep & trackGenre
set row to row & sep & trackPlayCount & sep & pd
set output to output & row & linefeed
end repeat
return output
end tell
"""
def _run_osascript(script: str, *, timeout: int = 120) -> Optional[str]:
"""Execute an AppleScript via ``osascript`` and return stdout.
Returns None on failure.
"""
try:
result = subprocess.run(
["osascript", "-e", script],
capture_output=True,
text=True,
timeout=timeout,
)
if result.returncode == 0:
return result.stdout.strip()
logger.debug("osascript failed (rc=%d): %s", result.returncode, result.stderr)
return None
except (subprocess.TimeoutExpired, FileNotFoundError, OSError) as exc:
logger.debug("osascript error: %s", exc)
return None
def _track_doc_id(name: str, artist: str) -> str:
"""Deterministic doc_id from track name + artist."""
key = f"{name}:{artist}"
h = hashlib.sha256(key.encode("utf-8")).hexdigest()[:12]
return f"apple_music-{h}"
def _parse_played_date(raw: str) -> Optional[datetime]:
"""Try to parse an AppleScript date string; return None if unparseable."""
if not raw or raw.strip().lower() == "never":
return None
# AppleScript renders dates in the user's locale. Common macOS formats:
# "Saturday, March 15, 2026 at 2:30:00 PM"
# "2026-03-15 14:30:00"
for fmt in (
"%A, %B %d, %Y at %I:%M:%S %p",
"%A, %B %d, %Y at %H:%M:%S",
"%Y-%m-%d %H:%M:%S",
"%m/%d/%Y %I:%M:%S %p",
"%d/%m/%Y %H:%M:%S",
):
try:
return datetime.strptime(raw.strip(), fmt)
except ValueError:
continue
logger.debug("Could not parse played_date: %r", raw)
return None
# ---------------------------------------------------------------------------
# AppleMusicConnector
# ---------------------------------------------------------------------------
@ConnectorRegistry.register("apple_music")
class AppleMusicConnector(BaseConnector):
"""Read tracks from the local Music.app library via AppleScript.
Only works on macOS where ``osascript`` is available and Music.app is
installed (ships with the OS).
"""
connector_id = "apple_music"
display_name = "Apple Music"
auth_type = "local"
def __init__(self) -> None:
self._status = SyncStatus()
# ------------------------------------------------------------------
# BaseConnector interface
# ------------------------------------------------------------------
def is_connected(self) -> bool:
"""Return True if running on macOS and Music.app responds."""
if sys.platform != "darwin":
return False
result = _run_osascript(_LIBRARY_CHECK_SCRIPT, timeout=10)
return result is not None
def disconnect(self) -> None:
"""No-op -- local connector with no credentials to revoke."""
def sync(
self,
*,
since: Optional[datetime] = None,
cursor: Optional[str] = None, # noqa: ARG002
) -> Iterator[Document]:
"""Query Music.app for all tracks and yield one Document per track.
Parameters
----------
since:
If provided, only yield tracks whose last-played date is after
this datetime.
cursor:
Not used for this local connector (included for API
compatibility).
Yields
------
Document
One document per track in the Music library.
"""
raw = _run_osascript(_TRACKS_SCRIPT)
if raw is None:
logger.warning("Could not retrieve tracks from Music.app")
self._status.state = "error"
self._status.error = "AppleScript query failed"
return
lines = [line for line in raw.split("\n") if line.strip()]
self._status.items_total = len(lines)
synced = 0
for line in lines:
parts = line.split("|||")
if len(parts) < 7:
logger.debug("Skipping malformed line: %r", line)
continue
name = parts[0].strip()
artist = parts[1].strip()
album = parts[2].strip()
duration_raw = parts[3].strip()
genre = parts[4].strip()
play_count_raw = parts[5].strip()
played_date_raw = parts[6].strip()
# Parse numeric fields
try:
duration_s = round(float(duration_raw), 2)
except (ValueError, TypeError):
duration_s = 0.0
try:
play_count = int(play_count_raw)
except (ValueError, TypeError):
play_count = 0
played_date = _parse_played_date(played_date_raw)
timestamp = played_date if played_date else datetime.now()
# Apply since filter
if since is not None and played_date is not None:
if played_date <= since:
continue
track_data = {
"name": name,
"artist": artist,
"album": album,
"duration_s": duration_s,
"genre": genre,
"play_count": play_count,
"played_date": played_date_raw,
}
doc = Document(
doc_id=_track_doc_id(name, artist),
source="apple_music",
doc_type="track",
content=json.dumps(track_data),
title=f"{name} \u2014 {artist}",
author=artist,
timestamp=timestamp,
metadata={
"album": album,
"duration_s": duration_s,
"genre": genre,
"play_count": play_count,
},
)
synced += 1
yield doc
self._status.items_synced = synced
self._status.state = "idle"
self._status.last_sync = datetime.now()
self._status.error = None
def sync_status(self) -> SyncStatus:
"""Return sync progress from the most recent :meth:`sync` call."""
return self._status
src/openjarvis/connectors/gcalendar.py
+20 -7
View File
@@ -7,16 +7,18 @@ to make them trivially mockable in tests.
from __future__ import annotations
from datetime import datetime
from datetime import datetime, timedelta
from typing import Any, Dict, Iterator, List, Optional
import httpx
from openjarvis.connectors._stubs import BaseConnector, Document, SyncStatus
from openjarvis.connectors.oauth import (
GOOGLE_ALL_SCOPES,
build_google_auth_url,
delete_tokens,
load_tokens,
resolve_google_credentials,
run_oauth_flow,
save_tokens,
)
@@ -204,7 +206,9 @@ class GCalendarConnector(BaseConnector):
auth_type = "oauth"
def __init__(self, credentials_path: str = "") -> None:
self._credentials_path = credentials_path or _DEFAULT_CREDENTIALS_PATH
self._credentials_path = resolve_google_credentials(
credentials_path or _DEFAULT_CREDENTIALS_PATH
)
self._items_synced: int = 0
self._items_total: int = 0
self._last_sync: Optional[datetime] = None
@@ -236,7 +240,7 @@ class GCalendarConnector(BaseConnector):
return "https://console.cloud.google.com/apis/credentials"
return build_google_auth_url(
client_id=client_id,
scopes=[_GCAL_SCOPE],
scopes=GOOGLE_ALL_SCOPES,
)
def handle_callback(self, code: str) -> None:
@@ -265,7 +269,7 @@ class GCalendarConnector(BaseConnector):
run_oauth_flow(
client_id=client_id.strip(),
client_secret=client_secret.strip(),
scopes=[_GCAL_SCOPE],
scopes=GOOGLE_ALL_SCOPES,
credentials_path=self._credentials_path,
)
except Exception: # noqa: BLE001
@@ -279,7 +283,7 @@ class GCalendarConnector(BaseConnector):
def sync(
self,
*,
since: Optional[datetime] = None, # noqa: ARG002 — reserved for future use
since: Optional[datetime] = None,
cursor: Optional[str] = None,
) -> Iterator[Document]:
"""Yield :class:`Document` objects for Google Calendar events.
@@ -290,7 +294,8 @@ class GCalendarConnector(BaseConnector):
Parameters
----------
since:
Not yet used (filtering is done server-side via ``timeMin``).
Only return events starting after this datetime. Defaults to
24 hours ago when ``None``.
cursor:
``nextPageToken`` from a previous sync to resume pagination.
"""
@@ -306,6 +311,11 @@ class GCalendarConnector(BaseConnector):
calendars_resp = _gcal_api_calendars_list(token)
calendars: List[Dict[str, Any]] = calendars_resp.get("items", [])
# Default to 24 hours ago so we don't dump the entire calendar history
if since is None:
since = datetime.now() - timedelta(days=1)
time_min = since.strftime("%Y-%m-%dT%H:%M:%SZ")
synced = 0
for calendar in calendars:
@@ -318,7 +328,10 @@ class GCalendarConnector(BaseConnector):
while True:
try:
events_resp = _gcal_api_events_list(
token, calendar_id, page_token=page_token
token,
calendar_id,
page_token=page_token,
time_min=time_min,
)
except httpx.HTTPStatusError:
break
src/openjarvis/connectors/gcontacts.py
+7 -3
View File
@@ -14,9 +14,11 @@ import httpx
from openjarvis.connectors._stubs import BaseConnector, Document, SyncStatus
from openjarvis.connectors.oauth import (
GOOGLE_ALL_SCOPES,
build_google_auth_url,
delete_tokens,
load_tokens,
resolve_google_credentials,
run_oauth_flow,
save_tokens,
)
@@ -151,7 +153,9 @@ class GContactsConnector(BaseConnector):
auth_type = "oauth"
def __init__(self, credentials_path: str = "") -> None:
self._credentials_path = credentials_path or _DEFAULT_CREDENTIALS_PATH
self._credentials_path = resolve_google_credentials(
credentials_path or _DEFAULT_CREDENTIALS_PATH
)
self._items_synced: int = 0
self._items_total: int = 0
self._last_sync: Optional[datetime] = None
@@ -183,7 +187,7 @@ class GContactsConnector(BaseConnector):
return "https://console.cloud.google.com/apis/credentials"
return build_google_auth_url(
client_id=client_id,
scopes=[_GCONTACTS_SCOPE],
scopes=GOOGLE_ALL_SCOPES,
)
def handle_callback(self, code: str) -> None:
@@ -212,7 +216,7 @@ class GContactsConnector(BaseConnector):
run_oauth_flow(
client_id=client_id.strip(),
client_secret=client_secret.strip(),
scopes=[_GCONTACTS_SCOPE],
scopes=GOOGLE_ALL_SCOPES,
credentials_path=self._credentials_path,
)
except Exception: # noqa: BLE001
src/openjarvis/connectors/gdrive.py
+7 -3
View File
@@ -14,9 +14,11 @@ import httpx
from openjarvis.connectors._stubs import BaseConnector, Document, SyncStatus
from openjarvis.connectors.oauth import (
GOOGLE_ALL_SCOPES,
build_google_auth_url,
delete_tokens,
load_tokens,
resolve_google_credentials,
run_oauth_flow,
save_tokens,
)
@@ -134,7 +136,9 @@ class GDriveConnector(BaseConnector):
auth_type = "oauth"
def __init__(self, credentials_path: str = "") -> None:
self._credentials_path = credentials_path or _DEFAULT_CREDENTIALS_PATH
self._credentials_path = resolve_google_credentials(
credentials_path or _DEFAULT_CREDENTIALS_PATH
)
self._items_synced: int = 0
self._items_total: int = 0
self._last_sync: Optional[datetime] = None
@@ -166,7 +170,7 @@ class GDriveConnector(BaseConnector):
return "https://console.cloud.google.com/apis/credentials"
return build_google_auth_url(
client_id=client_id,
scopes=[_GDRIVE_SCOPE],
scopes=GOOGLE_ALL_SCOPES,
)
def handle_callback(self, code: str) -> None:
@@ -197,7 +201,7 @@ class GDriveConnector(BaseConnector):
run_oauth_flow(
client_id=client_id.strip(),
client_secret=client_secret.strip(),
scopes=[_GDRIVE_SCOPE],
scopes=GOOGLE_ALL_SCOPES,
credentials_path=self._credentials_path,
)
except Exception: # noqa: BLE001
src/openjarvis/connectors/github_notifications.py
+111
View File
@@ -0,0 +1,111 @@
"""GitHub Notifications connector — unread notifications via GitHub REST API.
Uses a Personal Access Token stored in the connector config dir.
All API calls are in module-level functions for easy mocking in tests.
"""
from __future__ import annotations
import json
from datetime import datetime
from pathlib import Path
from typing import Any, Dict, Iterator, List, Optional
import httpx
from openjarvis.connectors._stubs import BaseConnector, Document, SyncStatus
from openjarvis.core.config import DEFAULT_CONFIG_DIR
from openjarvis.core.registry import ConnectorRegistry
_DEFAULT_TOKEN_PATH = str(DEFAULT_CONFIG_DIR / "connectors" / "github.json")
def _github_api_get(
token: str, params: Optional[Dict[str, str]] = None
) -> List[Dict[str, Any]]:
"""Fetch notifications from the GitHub API."""
resp = httpx.get(
"https://api.github.com/notifications",
headers={
"Authorization": f"Bearer {token}",
"Accept": "application/vnd.github+json",
},
params=params or {},
timeout=30.0,
)
resp.raise_for_status()
return resp.json()
@ConnectorRegistry.register("github_notifications")
class GitHubNotificationsConnector(BaseConnector):
"""Sync unread notifications from GitHub."""
connector_id = "github_notifications"
display_name = "GitHub Notifications"
auth_type = "token"
def __init__(self, *, token_path: str = _DEFAULT_TOKEN_PATH) -> None:
self._token_path = Path(token_path)
self._status = SyncStatus()
def _load_token(self) -> str:
"""Load the GitHub PAT from disk."""
data = json.loads(self._token_path.read_text(encoding="utf-8"))
return data["token"]
def is_connected(self) -> bool:
return self._token_path.exists()
def disconnect(self) -> None:
if self._token_path.exists():
self._token_path.unlink()
def sync(
self, *, since: Optional[datetime] = None, cursor: Optional[str] = None
) -> Iterator[Document]:
"""Yield Documents for each GitHub notification."""
token = self._load_token()
params: Dict[str, str] = {}
if since is not None:
params["since"] = f"{since.isoformat()}Z"
notifications = _github_api_get(token, params=params)
for notif in notifications:
subject = notif.get("subject", {})
repo = notif.get("repository", {}).get("full_name", "")
reason = notif.get("reason", "")
notif_type = subject.get("type", "")
title = subject.get("title", "")
notif_id = notif.get("id", "")
updated_at = notif.get("updated_at", "")
content = f"Reason: {reason}, Repository: {repo}"
ts = datetime.now()
if updated_at:
try:
ts = datetime.fromisoformat(updated_at.replace("Z", "+00:00"))
except ValueError:
pass
yield Document(
doc_id=f"github-notification-{notif_id}",
source="github_notifications",
doc_type="notification",
content=content,
title=title,
timestamp=ts,
url=subject.get("url"),
metadata={
"reason": reason,
"repo": repo,
"type": notif_type,
},
)
self._status.state = "idle"
self._status.last_sync = datetime.now()
def sync_status(self) -> SyncStatus:
return self._status
src/openjarvis/connectors/gmail.py
+8 -4
View File
@@ -16,9 +16,11 @@ import httpx
from openjarvis.connectors._stubs import BaseConnector, Document, SyncStatus
from openjarvis.connectors.oauth import (
GOOGLE_ALL_SCOPES,
build_google_auth_url,
delete_tokens,
load_tokens,
resolve_google_credentials,
save_tokens,
)
from openjarvis.core.config import DEFAULT_CONFIG_DIR
@@ -184,7 +186,9 @@ class GmailConnector(BaseConnector):
auth_type = "oauth"
def __init__(self, credentials_path: str = "") -> None:
self._credentials_path = credentials_path or _DEFAULT_CREDENTIALS_PATH
self._credentials_path = resolve_google_credentials(
credentials_path or _DEFAULT_CREDENTIALS_PATH
)
self._items_synced: int = 0
self._items_total: int = 0
self._last_sync: Optional[datetime] = None
@@ -211,7 +215,7 @@ class GmailConnector(BaseConnector):
"""Return a Google OAuth consent URL requesting ``gmail.readonly`` scope."""
return build_google_auth_url(
client_id="", # placeholder — real client_id from config
scopes=[_GMAIL_SCOPE],
scopes=GOOGLE_ALL_SCOPES,
)
def handle_callback(self, code: str) -> None:
@@ -249,11 +253,11 @@ class GmailConnector(BaseConnector):
if not token:
return
query = ""
query = "category:primary"
if since is not None:
# Gmail's after: operator accepts Unix epoch seconds.
epoch = int(since.timestamp())
query = f"after:{epoch}"
query = f"category:primary after:{epoch}"
page_token: Optional[str] = cursor
synced = 0
src/openjarvis/connectors/google_tasks.py
+116
View File
@@ -0,0 +1,116 @@
"""Google Tasks connector — tasks due today, overdue, and recently completed.
Uses OAuth2 tokens via the shared Google OAuth helper module.
"""
from __future__ import annotations
from datetime import datetime
from pathlib import Path
from typing import Any, Dict, Iterator, Optional
import httpx
from openjarvis.connectors._stubs import BaseConnector, Document, SyncStatus
from openjarvis.connectors.oauth import load_tokens, resolve_google_credentials
from openjarvis.core.config import DEFAULT_CONFIG_DIR
from openjarvis.core.registry import ConnectorRegistry
_TASKS_API_BASE = "https://tasks.googleapis.com/tasks/v1"
_DEFAULT_CREDENTIALS_PATH = str(DEFAULT_CONFIG_DIR / "connectors" / "google_tasks.json")
def _tasks_api_get(
token: str, endpoint: str, params: Optional[Dict[str, str]] = None
) -> Dict[str, Any]:
"""Call a Google Tasks API v1 endpoint."""
resp = httpx.get(
f"{_TASKS_API_BASE}/{endpoint}",
headers={"Authorization": f"Bearer {token}"},
params=params or {},
timeout=30.0,
)
resp.raise_for_status()
return resp.json()
@ConnectorRegistry.register("google_tasks")
class GoogleTasksConnector(BaseConnector):
"""Sync tasks from Google Tasks."""
connector_id = "google_tasks"
display_name = "Google Tasks"
auth_type = "oauth"
def __init__(self, *, credentials_path: str = "") -> None:
self._credentials_path = Path(
resolve_google_credentials(credentials_path or _DEFAULT_CREDENTIALS_PATH)
)
self._status = SyncStatus()
def _get_access_token(self) -> str:
tokens = load_tokens(str(self._credentials_path))
return tokens.get("access_token") or tokens.get("token", "")
def is_connected(self) -> bool:
return self._credentials_path.exists()
def disconnect(self) -> None:
if self._credentials_path.exists():
self._credentials_path.unlink()
def sync(
self, *, since: Optional[datetime] = None, cursor: Optional[str] = None
) -> Iterator[Document]:
token = self._get_access_token()
# List all task lists first
task_lists = _tasks_api_get(token, "users/@me/lists")
for tl in task_lists.get("items", []):
tl_id = tl["id"]
tl_title = tl.get("title", "My Tasks")
# Get tasks from this list, updated since cutoff
params: Dict[str, str] = {
"showCompleted": "true",
"showHidden": "false",
}
if since:
params["updatedMin"] = since.isoformat() + "Z"
tasks = _tasks_api_get(token, f"lists/{tl_id}/tasks", params=params)
for task in tasks.get("items", []):
due = task.get("due", "")
status = task.get("status", "needsAction")
ts = (
datetime.fromisoformat(
task.get("updated", "").replace("Z", "+00:00")
)
if task.get("updated")
else datetime.now()
)
yield Document(
doc_id=f"gtasks-{task['id']}",
source="google_tasks",
doc_type="task",
content=task.get("notes", ""),
title=task.get("title", "Untitled Task"),
timestamp=ts,
url=task.get("selfLink", ""),
metadata={
"task_list": tl_title,
"status": status,
"due": due,
"completed": task.get("completed", ""),
},
)
self._status.state = "idle"
self._status.last_sync = datetime.now()
def sync_status(self) -> SyncStatus:
return self._status
src/openjarvis/connectors/hackernews.py
+91
View File
@@ -0,0 +1,91 @@
"""Hacker News connector — top stories from the HN Firebase API.
No authentication required. All API calls are in module-level functions
for easy mocking in tests.
"""
from __future__ import annotations
from datetime import datetime
from typing import Any, Dict, Iterator, List, Optional
import httpx
from openjarvis.connectors._stubs import BaseConnector, Document, SyncStatus
from openjarvis.core.registry import ConnectorRegistry
_HN_API_BASE = "https://hacker-news.firebaseio.com/v0"
def _hn_top_story_ids() -> List[int]:
"""Fetch the list of top story IDs."""
resp = httpx.get(f"{_HN_API_BASE}/topstories.json", timeout=30.0)
resp.raise_for_status()
return resp.json()
def _hn_item(item_id: int) -> Dict[str, Any]:
"""Fetch a single HN item by ID."""
resp = httpx.get(f"{_HN_API_BASE}/item/{item_id}.json", timeout=30.0)
resp.raise_for_status()
return resp.json()
@ConnectorRegistry.register("hackernews")
class HackerNewsConnector(BaseConnector):
"""Fetch the current top stories from Hacker News."""
connector_id = "hackernews"
display_name = "Hacker News"
auth_type = "local"
def __init__(self) -> None:
self._status = SyncStatus()
def is_connected(self) -> bool:
return True
def disconnect(self) -> None:
pass # No credentials to revoke
def sync(
self, *, since: Optional[datetime] = None, cursor: Optional[str] = None
) -> Iterator[Document]:
"""Yield Documents for the top 5 Hacker News stories."""
top_ids = _hn_top_story_ids()
for story_id in top_ids[:5]:
item = _hn_item(story_id)
if item is None:
continue
title = item.get("title", "")
score = item.get("score", 0)
descendants = item.get("descendants", 0)
url = item.get("url", "")
by = item.get("by", "")
ts = datetime.now()
if item.get("time"):
ts = datetime.fromtimestamp(item["time"])
yield Document(
doc_id=f"hn-{story_id}",
source="hackernews",
doc_type="story",
content=f"Score: {score}, Comments: {descendants}",
title=title,
author=by,
timestamp=ts,
url=url or None,
metadata={
"story_id": story_id,
"score": score,
"descendants": descendants,
},
)
self._status.state = "idle"
self._status.last_sync = datetime.now()
def sync_status(self) -> SyncStatus:
return self._status
src/openjarvis/connectors/news_rss.py
+162
View File
@@ -0,0 +1,162 @@
"""News/RSS connector — aggregate headlines from RSS and Atom feeds.
Uses stdlib xml.etree.ElementTree for parsing (no extra dependencies).
Config file lists feeds to follow. All HTTP calls are in module-level
functions for easy mocking in tests.
"""
from __future__ import annotations
import json
import xml.etree.ElementTree as ET
from datetime import datetime
from email.utils import parsedate_to_datetime
from pathlib import Path
from typing import Dict, Iterator, List, Optional
import httpx
from openjarvis.connectors._stubs import BaseConnector, Document, SyncStatus
from openjarvis.core.config import DEFAULT_CONFIG_DIR
from openjarvis.core.registry import ConnectorRegistry
_DEFAULT_CONFIG_PATH = str(DEFAULT_CONFIG_DIR / "connectors" / "news_rss.json")
def _fetch_feed(url: str) -> str:
"""Download raw XML from a feed URL."""
resp = httpx.get(url, timeout=30.0, follow_redirects=True)
resp.raise_for_status()
return resp.text
def _parse_rss_items(xml_text: str, max_items: int = 5) -> List[Dict[str, str]]:
"""Parse RSS or Atom XML and return up to *max_items* entries."""
root = ET.fromstring(xml_text)
items: List[Dict[str, str]] = []
# RSS 2.0: <rss><channel><item>
for item_el in root.iter("item"):
if len(items) >= max_items:
break
items.append(
{
"title": (item_el.findtext("title") or "").strip(),
"description": (item_el.findtext("description") or "").strip()[:200],
"link": (item_el.findtext("link") or "").strip(),
"pubDate": (item_el.findtext("pubDate") or "").strip(),
}
)
# Atom: <feed><entry>
if not items:
ns = {"atom": "http://www.w3.org/2005/Atom"}
for entry_el in root.iter("{http://www.w3.org/2005/Atom}entry"):
if len(items) >= max_items:
break
link_el = entry_el.find("atom:link", ns)
link_href = link_el.get("href", "") if link_el is not None else ""
summary = entry_el.findtext("{http://www.w3.org/2005/Atom}summary") or ""
updated = entry_el.findtext("{http://www.w3.org/2005/Atom}updated") or ""
items.append(
{
"title": (
entry_el.findtext("{http://www.w3.org/2005/Atom}title") or ""
).strip(),
"description": summary.strip()[:200],
"link": link_href.strip(),
"pubDate": updated.strip(),
}
)
return items
def _parse_pub_date(date_str: str) -> Optional[datetime]:
"""Best-effort parse of an RSS pubDate or Atom updated timestamp."""
if not date_str:
return None
try:
return parsedate_to_datetime(date_str)
except (ValueError, TypeError):
pass
try:
return datetime.fromisoformat(date_str.replace("Z", "+00:00"))
except (ValueError, TypeError):
return None
@ConnectorRegistry.register("news_rss")
class NewsRSSConnector(BaseConnector):
"""Aggregate headlines from configured RSS/Atom feeds."""
connector_id = "news_rss"
display_name = "News / RSS"
auth_type = "local"
def __init__(self, *, config_path: str = _DEFAULT_CONFIG_PATH) -> None:
self._config_path = Path(config_path)
self._status = SyncStatus()
def _load_config(self) -> List[Dict[str, str]]:
"""Load feed list from disk."""
data = json.loads(self._config_path.read_text(encoding="utf-8"))
return data.get("feeds", [])
def is_connected(self) -> bool:
if not self._config_path.exists():
return False
try:
feeds = self._load_config()
return len(feeds) > 0
except (json.JSONDecodeError, OSError):
return False
def disconnect(self) -> None:
if self._config_path.exists():
self._config_path.unlink()
def sync(
self, *, since: Optional[datetime] = None, cursor: Optional[str] = None
) -> Iterator[Document]:
"""Yield Documents for recent items across all configured feeds."""
feeds = self._load_config()
for feed in feeds:
feed_name = feed.get("name", "Unknown Feed")
feed_url = feed.get("url", "")
if not feed_url:
continue
try:
xml_text = _fetch_feed(feed_url)
except httpx.HTTPError:
continue
items = _parse_rss_items(xml_text)
for item in items:
pub_dt = _parse_pub_date(item["pubDate"])
# Filter by since if the date is parseable
if since and pub_dt and pub_dt.replace(tzinfo=None) < since:
continue
title = item["title"] or "Untitled"
doc_id = f"rss-{feed_name}-{title[:40]}"
yield Document(
doc_id=doc_id,
source="news_rss",
doc_type="article",
content=item["description"],
title=title,
timestamp=pub_dt or datetime.now(),
url=item["link"] or None,
metadata={"feed_name": feed_name},
)
self._status.state = "idle"
self._status.last_sync = datetime.now()
def sync_status(self) -> SyncStatus:
return self._status
src/openjarvis/connectors/oauth.py
+381 -15
View File
@@ -1,21 +1,181 @@
"""Shared OAuth 2.0 helpers for Google connectors.
"""Shared OAuth 2.0 helpers for all connectors.
Provides URL builder, token persistence, and token cleanup utilities
that are reused by gmail, drive, calendar, and contacts connectors.
Provides:
- ``OAuthProvider`` registry with configs for Google, Strava, Spotify
- Generic ``run_connector_oauth()`` that opens browser + catches callback
- URL builder, token persistence, and token cleanup utilities
"""
from __future__ import annotations
import base64
import json
import os
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, Dict, List, Optional
from typing import Any, Dict, List, Optional, Tuple
from urllib.parse import urlencode
from openjarvis.core.config import DEFAULT_CONFIG_DIR
# ---------------------------------------------------------------------------
# Google OAuth endpoints / defaults
# Connector credentials directory
# ---------------------------------------------------------------------------
_CONNECTORS_DIR = DEFAULT_CONFIG_DIR / "connectors"
# ---------------------------------------------------------------------------
# OAuth provider registry
# ---------------------------------------------------------------------------
@dataclass(frozen=True)
class OAuthProvider:
"""Configuration for an OAuth 2.0 provider."""
name: str # "google", "strava", "spotify"
display_name: str
auth_endpoint: str
token_endpoint: str
scopes: List[str]
setup_url: str # URL where user creates OAuth credentials
setup_hint: str # One-line instruction for setup
callback_port: int = 8789
callback_host: str = "127.0.0.1"
callback_path: str = "/callback"
token_auth: str = "body" # "body" or "basic"
extra_auth_params: Dict[str, str] = field(default_factory=dict)
# Which connector IDs this provider covers (one flow → all connected)
connector_ids: Tuple[str, ...] = ()
# Filenames in ~/.openjarvis/connectors/ to save tokens to
credential_files: Tuple[str, ...] = ()
# Combined scopes for all Google connectors so a single OAuth consent
# authorises Drive, Calendar, Contacts, Gmail, and Tasks at once.
GOOGLE_ALL_SCOPES: List[str] = [
"openid",
"email",
"profile",
"https://www.googleapis.com/auth/drive.readonly",
"https://www.googleapis.com/auth/calendar.readonly",
"https://www.googleapis.com/auth/contacts.readonly",
"https://www.googleapis.com/auth/gmail.readonly",
"https://www.googleapis.com/auth/tasks.readonly",
]
OAUTH_PROVIDERS: Dict[str, OAuthProvider] = {
"google": OAuthProvider(
name="google",
display_name="Google",
auth_endpoint="https://accounts.google.com/o/oauth2/v2/auth",
token_endpoint="https://oauth2.googleapis.com/token",
scopes=GOOGLE_ALL_SCOPES,
setup_url="https://console.cloud.google.com/apis/credentials",
setup_hint="Create an OAuth 2.0 Client ID (Desktop app type)",
extra_auth_params={"access_type": "offline", "prompt": "consent"},
connector_ids=(
"gdrive",
"gcalendar",
"gcontacts",
"gmail",
"google_tasks",
),
credential_files=(
"google.json",
"gdrive.json",
"gcalendar.json",
"gcontacts.json",
"gmail.json",
"google_tasks.json",
),
),
"strava": OAuthProvider(
name="strava",
display_name="Strava",
auth_endpoint="https://www.strava.com/oauth/authorize",
token_endpoint="https://www.strava.com/oauth/token",
scopes=["activity:read_all"],
setup_url="https://www.strava.com/settings/api",
setup_hint="Create an API Application (callback domain: localhost)",
connector_ids=("strava",),
credential_files=("strava.json",),
),
"spotify": OAuthProvider(
name="spotify",
display_name="Spotify",
auth_endpoint="https://accounts.spotify.com/authorize",
token_endpoint="https://accounts.spotify.com/api/token",
scopes=["user-read-recently-played"],
setup_url="https://developer.spotify.com/dashboard",
setup_hint=("Create an app, add redirect URI: http://127.0.0.1:8888/callback"),
callback_port=8888,
token_auth="basic",
connector_ids=("spotify",),
credential_files=("spotify.json",),
),
}
def get_provider_for_connector(connector_id: str) -> Optional[OAuthProvider]:
"""Return the OAuthProvider that covers *connector_id*, or ``None``."""
for provider in OAUTH_PROVIDERS.values():
if connector_id in provider.connector_ids:
return provider
return None
# ---------------------------------------------------------------------------
# Credential helpers
# ---------------------------------------------------------------------------
def get_client_credentials(
provider: OAuthProvider,
) -> Optional[Tuple[str, str]]:
"""Load stored client_id and client_secret for *provider*.
Checks credential files in ``~/.openjarvis/connectors/`` and falls
back to environment variables ``OPENJARVIS_{NAME}_CLIENT_ID`` and
``OPENJARVIS_{NAME}_CLIENT_SECRET``.
"""
# Check credential files
for filename in provider.credential_files:
path = _CONNECTORS_DIR / filename
tokens = load_tokens(str(path))
if tokens and tokens.get("client_id") and tokens.get("client_secret"):
return tokens["client_id"], tokens["client_secret"]
# Check environment variables
prefix = f"OPENJARVIS_{provider.name.upper()}"
env_id = os.environ.get(f"{prefix}_CLIENT_ID", "")
env_secret = os.environ.get(f"{prefix}_CLIENT_SECRET", "")
if env_id and env_secret:
return env_id, env_secret
return None
def save_client_credentials(
provider: OAuthProvider,
client_id: str,
client_secret: str,
) -> None:
"""Persist client credentials so the user never has to enter them again."""
for filename in provider.credential_files:
path = _CONNECTORS_DIR / filename
existing = load_tokens(str(path)) or {}
existing["client_id"] = client_id
existing["client_secret"] = client_secret
save_tokens(str(path), existing)
# ---------------------------------------------------------------------------
# Shared credentials file — one OAuth flow covers all Google connectors
# ---------------------------------------------------------------------------
_SHARED_GOOGLE_CREDENTIALS_PATH: str = str(_CONNECTORS_DIR / "google.json")
_GOOGLE_AUTH_ENDPOINT = "https://accounts.google.com/o/oauth2/v2/auth"
_DEFAULT_REDIRECT_URI = "http://localhost:8789/callback"
_DEFAULT_SCOPES: List[str] = ["openid", "email", "profile"]
@@ -63,6 +223,20 @@ def build_google_auth_url(
return f"{_GOOGLE_AUTH_ENDPOINT}?{urlencode(params)}"
def resolve_google_credentials(connector_path: str) -> str:
"""Return the best available Google credentials file path.
Checks the connector-specific file first, then falls back to the
shared ``google.json``. Returns *connector_path* if neither exists
(so ``is_connected()`` correctly returns ``False``).
"""
if Path(connector_path).exists():
return connector_path
if Path(_SHARED_GOOGLE_CREDENTIALS_PATH).exists():
return _SHARED_GOOGLE_CREDENTIALS_PATH
return connector_path
def load_tokens(path: str) -> Optional[Dict[str, Any]]:
"""Load OAuth tokens from a JSON file.
@@ -281,16 +455,208 @@ def run_oauth_flow(
)
# Persist tokens together with client credentials (needed for refresh)
save_tokens(
credentials_path,
{
"access_token": tokens.get("access_token", ""),
"refresh_token": tokens.get("refresh_token", ""),
"token_type": tokens.get("token_type", "Bearer"),
"expires_in": tokens.get("expires_in", 3600),
"client_id": client_id,
"client_secret": client_secret,
},
token_payload = {
"access_token": tokens.get("access_token", ""),
"refresh_token": tokens.get("refresh_token", ""),
"token_type": tokens.get("token_type", "Bearer"),
"expires_in": tokens.get("expires_in", 3600),
"client_id": client_id,
"client_secret": client_secret,
}
save_tokens(credentials_path, token_payload)
# Also save to the shared Google credentials file so that all Google
# connectors can use this token without a separate OAuth flow.
if credentials_path != _SHARED_GOOGLE_CREDENTIALS_PATH:
save_tokens(_SHARED_GOOGLE_CREDENTIALS_PATH, token_payload)
return tokens
# ---------------------------------------------------------------------------
# Generic OAuth flow — works with any OAuthProvider
# ---------------------------------------------------------------------------
def _wait_for_callback_code(
host: str = "127.0.0.1",
port: int = 8789,
path: str = "/callback",
timeout: int = 120,
) -> str:
"""Start a localhost HTTP server and wait for ``?code=`` on *path*.
Returns the authorization code received from the OAuth redirect.
"""
from http.server import BaseHTTPRequestHandler, HTTPServer
from urllib.parse import parse_qs, urlparse
auth_code: List[str] = []
error: List[str] = []
class _Handler(BaseHTTPRequestHandler):
def do_GET(self) -> None: # noqa: N802
params = parse_qs(urlparse(self.path).query)
if "code" in params:
auth_code.append(params["code"][0])
self.send_response(200)
self.send_header("Content-Type", "text/html")
self.end_headers()
self.wfile.write(
b"<html><body style='font-family:system-ui;text-align:center;"
b"padding:60px'>"
b"<h2 style='color:#22c55e'>Connected!</h2>"
b"<p>You can close this tab and return to OpenJarvis.</p>"
b"</body></html>"
)
elif "error" in params:
error.append(params.get("error", ["unknown"])[0])
self.send_response(400)
self.send_header("Content-Type", "text/html")
self.end_headers()
self.wfile.write(
b"<html><body style='font-family:system-ui;text-align:center;"
b"padding:60px'>"
b"<h2 style='color:#ef4444'>Authorization Failed</h2>"
b"<p>Please close this tab and try again.</p>"
b"</body></html>"
)
else:
self.send_response(400)
self.end_headers()
def log_message(self, *_args: Any) -> None:
pass
# Ensure port is free
import socket
test_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
test_sock.bind((host, port))
except OSError:
pass
finally:
test_sock.close()
import time
time.sleep(0.3)
server = HTTPServer((host, port), _Handler)
server.timeout = timeout
while not auth_code and not error:
server.handle_request()
server.server_close()
if error:
raise RuntimeError(f"OAuth authorization denied: {error[0]}")
if not auth_code:
raise RuntimeError("OAuth callback timed out")
return auth_code[0]
def _exchange_token(
provider: OAuthProvider,
code: str,
client_id: str,
client_secret: str,
redirect_uri: str,
) -> Dict[str, Any]:
"""Exchange an authorization *code* for tokens using *provider* config."""
import httpx
data: Dict[str, str] = {
"code": code,
"redirect_uri": redirect_uri,
"grant_type": "authorization_code",
}
headers: Dict[str, str] = {}
if provider.token_auth == "basic":
creds = base64.b64encode(f"{client_id}:{client_secret}".encode()).decode()
headers["Authorization"] = f"Basic {creds}"
else:
data["client_id"] = client_id
data["client_secret"] = client_secret
resp = httpx.post(provider.token_endpoint, data=data, headers=headers, timeout=30.0)
resp.raise_for_status()
return resp.json()
def run_connector_oauth(
connector_id: str,
client_id: str = "",
client_secret: str = "",
) -> Dict[str, Any]:
"""Run a complete OAuth flow for *connector_id*.
1. Look up the ``OAuthProvider``
2. Resolve client credentials (arg → stored → env)
3. Build auth URL and open the user's browser
4. Start localhost callback server and wait for the code
5. Exchange the code for tokens
6. Save tokens to all relevant credential files
Returns the raw token response dict.
"""
import webbrowser
provider = get_provider_for_connector(connector_id)
if provider is None:
raise ValueError(f"No OAuth provider configured for '{connector_id}'")
# Resolve credentials
if not (client_id and client_secret):
creds = get_client_credentials(provider)
if creds:
client_id, client_secret = creds
if not (client_id and client_secret):
raise RuntimeError(
f"No client credentials for {provider.display_name}. "
f"Set them up at: {provider.setup_url}"
)
redirect_uri = (
f"http://{provider.callback_host}:{provider.callback_port}"
f"{provider.callback_path}"
)
# Build auth URL
params: Dict[str, str] = {
"client_id": client_id,
"redirect_uri": redirect_uri,
"response_type": "code",
"scope": " ".join(provider.scopes),
**provider.extra_auth_params,
}
auth_url = f"{provider.auth_endpoint}?{urlencode(params)}"
# Open browser and wait for callback
webbrowser.open(auth_url)
code = _wait_for_callback_code(
host=provider.callback_host,
port=provider.callback_port,
path=provider.callback_path,
)
# Exchange code for tokens
tokens = _exchange_token(provider, code, client_id, client_secret, redirect_uri)
# Build payload with client credentials included (needed for refresh)
payload = {
"access_token": tokens.get("access_token", ""),
"refresh_token": tokens.get("refresh_token", ""),
"token_type": tokens.get("token_type", "Bearer"),
"expires_in": tokens.get("expires_in", 3600),
"client_id": client_id,
"client_secret": client_secret,
}
# Save to all credential files for this provider
for filename in provider.credential_files:
save_tokens(str(_CONNECTORS_DIR / filename), payload)
return tokens
src/openjarvis/connectors/oura.py
+90
View File
@@ -0,0 +1,90 @@
"""Oura Ring connector — sleep, readiness, and activity via REST API v2.
Uses a Personal Access Token (PAT) stored in the connector config dir.
All API calls are in module-level functions for easy mocking in tests.
"""
from __future__ import annotations
import json
from datetime import datetime, timedelta
from pathlib import Path
from typing import Any, Dict, Iterator, Optional
import httpx
from openjarvis.connectors._stubs import BaseConnector, Document, SyncStatus
from openjarvis.core.config import DEFAULT_CONFIG_DIR
from openjarvis.core.registry import ConnectorRegistry
_OURA_API_BASE = "https://api.ouraring.com/v2/usercollection"
_DEFAULT_TOKEN_PATH = str(DEFAULT_CONFIG_DIR / "connectors" / "oura.json")
def _oura_api_get(
token: str, endpoint: str, params: Optional[Dict[str, str]] = None
) -> Dict[str, Any]:
"""Call an Oura API v2 endpoint."""
resp = httpx.get(
f"{_OURA_API_BASE}/{endpoint}",
headers={"Authorization": f"Bearer {token}"},
params=params or {},
timeout=30.0,
)
resp.raise_for_status()
return resp.json()
@ConnectorRegistry.register("oura")
class OuraConnector(BaseConnector):
"""Sync sleep, readiness, and activity data from Oura Ring."""
connector_id = "oura"
display_name = "Oura Ring"
auth_type = "token"
def __init__(self, *, token_path: str = _DEFAULT_TOKEN_PATH) -> None:
self._token_path = Path(token_path)
self._status = SyncStatus()
def _load_token(self) -> str:
"""Load the Oura PAT from disk."""
data = json.loads(self._token_path.read_text(encoding="utf-8"))
return data["token"]
def is_connected(self) -> bool:
return self._token_path.exists()
def disconnect(self) -> None:
if self._token_path.exists():
self._token_path.unlink()
def sync(
self, *, since: Optional[datetime] = None, cursor: Optional[str] = None
) -> Iterator[Document]:
"""Yield Documents for sleep, readiness, and activity."""
token = self._load_token()
start = (since or datetime.now() - timedelta(days=7)).strftime("%Y-%m-%d")
end = datetime.now().strftime("%Y-%m-%d")
for data_type in ("sleep", "daily_readiness", "daily_activity"):
data = _oura_api_get(
token, data_type, params={"start_date": start, "end_date": end}
)
for item in data.get("data", []):
day = item.get("day", start)
yield Document(
doc_id=f"oura-{data_type}-{day}",
source="oura",
doc_type=data_type,
content=json.dumps(item),
title=f"Oura {data_type.replace('_', ' ').title()} — {day}",
timestamp=datetime.fromisoformat(day),
metadata={"data_type": data_type, "day": day},
)
self._status.state = "idle"
self._status.last_sync = datetime.now()
def sync_status(self) -> SyncStatus:
return self._status
src/openjarvis/connectors/slack_connector.py
+3 -1
View File
@@ -302,7 +302,9 @@ class SlackConnector(BaseConnector):
# Try to join the public channel
try:
join_resp = _slack_api_with_retry(
"conversations.join", token, {"channel": chan_id},
"conversations.join",
token,
{"channel": chan_id},
http_method="POST",
)
if not join_resp.get("ok"):
src/openjarvis/connectors/spotify.py
+157
View File
@@ -0,0 +1,157 @@
"""Spotify connector — recently played tracks via Spotify Web API.
Uses OAuth2 tokens stored locally. Requires user-read-recently-played scope.
"""
from __future__ import annotations
import json
from datetime import datetime, timedelta
from pathlib import Path
from typing import Any, Dict, Iterator, Optional
import httpx
from openjarvis.connectors._stubs import BaseConnector, Document, SyncStatus
from openjarvis.core.config import DEFAULT_CONFIG_DIR
from openjarvis.core.registry import ConnectorRegistry
_SPOTIFY_API_BASE = "https://api.spotify.com/v1"
_SPOTIFY_TOKEN_URL = "https://accounts.spotify.com/api/token"
_DEFAULT_TOKEN_PATH = str(DEFAULT_CONFIG_DIR / "connectors" / "spotify.json")
def _spotify_api_get(
token: str, endpoint: str, params: Optional[Dict[str, Any]] = None
) -> Dict[str, Any]:
"""Call a Spotify Web API endpoint."""
resp = httpx.get(
f"{_SPOTIFY_API_BASE}/{endpoint}",
headers={"Authorization": f"Bearer {token}"},
params=params or {},
timeout=30.0,
)
resp.raise_for_status()
return resp.json()
@ConnectorRegistry.register("spotify")
class SpotifyConnector(BaseConnector):
"""Sync recently played tracks from Spotify."""
connector_id = "spotify"
display_name = "Spotify"
auth_type = "oauth"
def __init__(self, *, token_path: str = _DEFAULT_TOKEN_PATH) -> None:
self._token_path = Path(token_path)
self._status = SyncStatus()
def _load_tokens(self) -> Dict[str, str]:
return json.loads(self._token_path.read_text(encoding="utf-8"))
def _get_access_token(self) -> str:
return self._load_tokens()["access_token"]
def is_connected(self) -> bool:
return self._token_path.exists()
def disconnect(self) -> None:
if self._token_path.exists():
self._token_path.unlink()
def auth_url(self) -> str:
"""Return Spotify OAuth authorization URL."""
from urllib.parse import urlencode
from openjarvis.connectors.oauth import (
get_client_credentials,
get_provider_for_connector,
)
provider = get_provider_for_connector("spotify")
if not provider:
return "https://developer.spotify.com/dashboard"
creds = get_client_credentials(provider)
if not creds:
return "https://developer.spotify.com/dashboard"
client_id, _ = creds
redirect_uri = f"http://{provider.callback_host}:{provider.callback_port}{provider.callback_path}"
params = {
"client_id": client_id,
"redirect_uri": redirect_uri,
"response_type": "code",
"scope": " ".join(provider.scopes),
}
return f"{provider.auth_endpoint}?{urlencode(params)}"
def handle_callback(self, code: str) -> None:
"""Exchange authorization code for tokens and save."""
from openjarvis.connectors.oauth import (
_CONNECTORS_DIR,
_exchange_token,
get_client_credentials,
get_provider_for_connector,
save_tokens,
)
provider = get_provider_for_connector("spotify")
creds = get_client_credentials(provider) if provider else None
if not provider or not creds:
raise RuntimeError("Spotify client credentials not configured")
client_id, client_secret = creds
redirect_uri = f"http://{provider.callback_host}:{provider.callback_port}{provider.callback_path}"
tokens = _exchange_token(provider, code, client_id, client_secret, redirect_uri)
payload = {
"access_token": tokens.get("access_token", ""),
"refresh_token": tokens.get("refresh_token", ""),
"client_id": client_id,
"client_secret": client_secret,
}
for filename in provider.credential_files:
save_tokens(str(_CONNECTORS_DIR / filename), payload)
def sync(
self, *, since: Optional[datetime] = None, cursor: Optional[str] = None
) -> Iterator[Document]:
token = self._get_access_token()
after_ms = int((since or datetime.now() - timedelta(days=1)).timestamp() * 1000)
data = _spotify_api_get(
token,
"me/player/recently-played",
params={"limit": "50", "after": str(after_ms)},
)
for item in data.get("items", []):
track = item.get("track", {})
played_at = item.get("played_at", "")
artists = ", ".join(a["name"] for a in track.get("artists", []))
ts = (
datetime.fromisoformat(played_at.replace("Z", "+00:00"))
if played_at
else datetime.now()
)
yield Document(
doc_id=f"spotify-{track.get('id', '')}-{played_at}",
source="spotify",
doc_type="recently_played",
content=json.dumps(item),
title=f"{track.get('name', 'Unknown')} — {artists}",
author=artists,
timestamp=ts,
url=track.get("external_urls", {}).get("spotify", ""),
metadata={
"track_name": track.get("name", ""),
"album": track.get("album", {}).get("name", ""),
"duration_ms": track.get("duration_ms", 0),
},
)
self._status.state = "idle"
self._status.last_sync = datetime.now()
def sync_status(self) -> SyncStatus:
return self._status
src/openjarvis/connectors/strava.py
+173
View File
@@ -0,0 +1,173 @@
"""Strava connector — recent activities via REST API v3.
Uses OAuth2 tokens stored locally. Refresh handled automatically.
"""
from __future__ import annotations
import json
from datetime import datetime, timedelta
from pathlib import Path
from typing import Any, Dict, Iterator, Optional
import httpx
from openjarvis.connectors._stubs import BaseConnector, Document, SyncStatus
from openjarvis.core.config import DEFAULT_CONFIG_DIR
from openjarvis.core.registry import ConnectorRegistry
_STRAVA_API_BASE = "https://www.strava.com/api/v3"
_STRAVA_TOKEN_URL = "https://www.strava.com/oauth/token"
_DEFAULT_TOKEN_PATH = str(DEFAULT_CONFIG_DIR / "connectors" / "strava.json")
def _strava_api_get(
token: str, endpoint: str, params: Optional[Dict[str, Any]] = None
) -> Any:
"""Call a Strava API v3 endpoint."""
resp = httpx.get(
f"{_STRAVA_API_BASE}/{endpoint}",
headers={"Authorization": f"Bearer {token}"},
params=params or {},
timeout=30.0,
)
resp.raise_for_status()
return resp.json()
def _strava_refresh_token(
client_id: str, client_secret: str, refresh_token: str
) -> Dict[str, Any]:
"""Refresh an expired Strava OAuth2 token."""
resp = httpx.post(
_STRAVA_TOKEN_URL,
data={
"client_id": client_id,
"client_secret": client_secret,
"refresh_token": refresh_token,
"grant_type": "refresh_token",
},
timeout=30.0,
)
resp.raise_for_status()
return resp.json()
@ConnectorRegistry.register("strava")
class StravaConnector(BaseConnector):
"""Sync recent activities from Strava."""
connector_id = "strava"
display_name = "Strava"
auth_type = "oauth"
def __init__(self, *, token_path: str = _DEFAULT_TOKEN_PATH) -> None:
self._token_path = Path(token_path)
self._status = SyncStatus()
def _load_tokens(self) -> Dict[str, str]:
return json.loads(self._token_path.read_text(encoding="utf-8"))
def _save_tokens(self, tokens: Dict[str, str]) -> None:
self._token_path.write_text(json.dumps(tokens), encoding="utf-8")
def _get_access_token(self) -> str:
tokens = self._load_tokens()
return tokens["access_token"]
def is_connected(self) -> bool:
return self._token_path.exists()
def disconnect(self) -> None:
if self._token_path.exists():
self._token_path.unlink()
def auth_url(self) -> str:
"""Return Strava OAuth authorization URL."""
from urllib.parse import urlencode
from openjarvis.connectors.oauth import (
get_client_credentials,
get_provider_for_connector,
)
provider = get_provider_for_connector("strava")
if not provider:
return "https://www.strava.com/settings/api"
creds = get_client_credentials(provider)
if not creds:
return "https://www.strava.com/settings/api"
client_id, _ = creds
redirect_uri = f"http://{provider.callback_host}:{provider.callback_port}{provider.callback_path}"
params = {
"client_id": client_id,
"redirect_uri": redirect_uri,
"response_type": "code",
"scope": " ".join(provider.scopes),
}
return f"{provider.auth_endpoint}?{urlencode(params)}"
def handle_callback(self, code: str) -> None:
"""Exchange authorization code for tokens and save."""
from openjarvis.connectors.oauth import (
_CONNECTORS_DIR,
_exchange_token,
get_client_credentials,
get_provider_for_connector,
save_tokens,
)
provider = get_provider_for_connector("strava")
creds = get_client_credentials(provider) if provider else None
if not provider or not creds:
raise RuntimeError("Strava client credentials not configured")
client_id, client_secret = creds
redirect_uri = f"http://{provider.callback_host}:{provider.callback_port}{provider.callback_path}"
tokens = _exchange_token(provider, code, client_id, client_secret, redirect_uri)
payload = {
"access_token": tokens.get("access_token", ""),
"refresh_token": tokens.get("refresh_token", ""),
"client_id": client_id,
"client_secret": client_secret,
}
for filename in provider.credential_files:
save_tokens(str(_CONNECTORS_DIR / filename), payload)
def sync(
self, *, since: Optional[datetime] = None, cursor: Optional[str] = None
) -> Iterator[Document]:
token = self._get_access_token()
after_epoch = int((since or datetime.now() - timedelta(days=7)).timestamp())
activities = _strava_api_get(
token,
"athlete/activities",
params={"after": str(after_epoch), "per_page": "50"},
)
for act in activities:
ts = (
datetime.fromisoformat(act["start_date_local"].replace("Z", "+00:00"))
if "start_date_local" in act
else datetime.now()
)
yield Document(
doc_id=f"strava-{act['id']}",
source="strava",
doc_type=act.get("type", "Activity").lower(),
content=json.dumps(act),
title=act.get("name", "Untitled Activity"),
timestamp=ts,
metadata={
"distance_m": act.get("distance", 0),
"moving_time_s": act.get("moving_time", 0),
"sport_type": act.get("sport_type", ""),
},
)
self._status.state = "idle"
self._status.last_sync = datetime.now()
def sync_status(self) -> SyncStatus:
return self._status
src/openjarvis/connectors/weather.py
+131
View File
@@ -0,0 +1,131 @@
"""Weather connector — current conditions and forecast via OpenWeatherMap API.
Uses an API key stored in the connector config dir.
All API calls are in module-level functions for easy mocking in tests.
"""
from __future__ import annotations
import json
from datetime import datetime
from pathlib import Path
from typing import Any, Dict, Iterator, Optional
import httpx
from openjarvis.connectors._stubs import BaseConnector, Document, SyncStatus
from openjarvis.core.config import DEFAULT_CONFIG_DIR
from openjarvis.core.registry import ConnectorRegistry
_DEFAULT_TOKEN_PATH = str(DEFAULT_CONFIG_DIR / "connectors" / "weather.json")
def _weather_api_get(url: str, params: Dict[str, str]) -> Dict[str, Any]:
"""Call an OpenWeatherMap API endpoint."""
resp = httpx.get(url, params=params, timeout=30.0)
resp.raise_for_status()
return resp.json()
@ConnectorRegistry.register("weather")
class WeatherConnector(BaseConnector):
"""Fetch current weather and short-term forecast from OpenWeatherMap."""
connector_id = "weather"
display_name = "Weather"
auth_type = "token"
def __init__(self, *, token_path: str = _DEFAULT_TOKEN_PATH) -> None:
self._token_path = Path(token_path)
self._status = SyncStatus()
def _load_config(self) -> Dict[str, str]:
"""Load API key and location from disk."""
data = json.loads(self._token_path.read_text(encoding="utf-8"))
return data
def is_connected(self) -> bool:
if not self._token_path.exists():
return False
try:
data = json.loads(self._token_path.read_text(encoding="utf-8"))
return bool(data.get("api_key"))
except (json.JSONDecodeError, OSError):
return False
def disconnect(self) -> None:
if self._token_path.exists():
self._token_path.unlink()
def sync(
self, *, since: Optional[datetime] = None, cursor: Optional[str] = None
) -> Iterator[Document]:
"""Yield Documents for current weather and forecast."""
config = self._load_config()
api_key = config["api_key"]
location = config.get("location", "San Francisco,CA")
# Current weather
current = _weather_api_get(
"https://api.openweathermap.org/data/2.5/weather",
params={"q": location, "appid": api_key, "units": "imperial"},
)
main = current.get("main", {})
weather_desc = ", ".join(
w.get("description", "") for w in current.get("weather", [])
)
content = (
f"Temperature: {main.get('temp')}°F, "
f"Conditions: {weather_desc}, "
f"Humidity: {main.get('humidity')}%, "
f"Wind: {current.get('wind', {}).get('speed')} mph"
)
yield Document(
doc_id=f"weather-current-{location}",
source="weather",
doc_type="current",
content=content,
title=f"Current Weather — {location}",
timestamp=datetime.now(),
metadata={
"location": location,
"temp": main.get("temp"),
"conditions": weather_desc,
"humidity": main.get("humidity"),
"wind_speed": current.get("wind", {}).get("speed"),
},
)
# Forecast (next ~12 hours, 4 x 3-hour intervals)
forecast = _weather_api_get(
"https://api.openweathermap.org/data/2.5/forecast",
params={
"q": location,
"appid": api_key,
"units": "imperial",
"cnt": "4",
},
)
summaries = []
for entry in forecast.get("list", []):
dt_txt = entry.get("dt_txt", "")
temp = entry.get("main", {}).get("temp")
desc = ", ".join(w.get("description", "") for w in entry.get("weather", []))
summaries.append(f"{dt_txt}: {temp}°F, {desc}")
forecast_content = "Forecast:\n" + "\n".join(summaries)
yield Document(
doc_id=f"weather-forecast-{location}",
source="weather",
doc_type="forecast",
content=forecast_content,
title=f"Weather Forecast — {location}",
timestamp=datetime.now(),
metadata={"location": location},
)
self._status.state = "idle"
self._status.last_sync = datetime.now()
def sync_status(self) -> SyncStatus:
return self._status
src/openjarvis/core/config.py
+46 -1
View File
@@ -14,7 +14,7 @@ import shutil
import subprocess
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, Dict, Optional
from typing import Any, Dict, List, Optional
try:
import tomllib # Python 3.11+
@@ -1235,6 +1235,49 @@ class SkillsConfig:
auto_discover: bool = True
@dataclass
class DigestSectionConfig:
"""Configuration for a single digest section."""
sources: List[str] = field(default_factory=list)
max_items: int = 10
priority_contacts: List[str] = field(default_factory=list)
@dataclass
class DigestConfig:
"""Configuration for the morning digest feature."""
enabled: bool = False
schedule: str = "0 6 * * *"
timezone: str = "America/Los_Angeles"
persona: str = "jarvis"
sections: List[str] = field(
default_factory=lambda: ["messages", "calendar", "health", "world"]
)
optional_sections: List[str] = field(
default_factory=lambda: ["github", "financial", "music", "fitness"]
)
honorific: str = "sir"
voice_id: str = ""
voice_speed: float = 1.0
tts_backend: str = "cartesia"
messages: DigestSectionConfig = field(
default_factory=lambda: DigestSectionConfig(
sources=["gmail", "slack", "google_tasks"]
)
)
calendar: DigestSectionConfig = field(
default_factory=lambda: DigestSectionConfig(sources=["gcalendar"])
)
health: DigestSectionConfig = field(
default_factory=lambda: DigestSectionConfig(sources=["oura", "apple_health"])
)
world: DigestSectionConfig = field(
default_factory=lambda: DigestSectionConfig(sources=[])
)
@dataclass
class JarvisConfig:
"""Top-level configuration for OpenJarvis."""
@@ -1263,6 +1306,7 @@ class JarvisConfig:
system_prompt: SystemPromptConfig = field(default_factory=SystemPromptConfig)
compression: CompressionConfig = field(default_factory=CompressionConfig)
skills: SkillsConfig = field(default_factory=SkillsConfig)
digest: DigestConfig = field(default_factory=DigestConfig)
@property
def memory(self) -> StorageConfig:
@@ -1472,6 +1516,7 @@ def load_config(path: Optional[Path] = None) -> JarvisConfig:
"speech",
"optimize",
"agent_manager",
"digest",
)
for section_name in top_sections:
if section_name in data:
src/openjarvis/core/registry.py
+5
View File
@@ -145,6 +145,10 @@ class CompressionRegistry(RegistryBase[Any]):
"""Registry for context compression strategies."""
class TTSRegistry(RegistryBase[Any]):
"""Registry for text-to-speech backend implementations."""
class ConnectorRegistry(RegistryBase[Any]):
"""Registry for data source connectors (Gmail, Slack, etc.)."""
@@ -163,5 +167,6 @@ __all__ = [
"RouterPolicyRegistry",
"SkillRegistry",
"SpeechRegistry",
"TTSRegistry",
"ToolRegistry",
]
src/openjarvis/evals/cli.py
+3
View File
@@ -326,6 +326,7 @@ def _build_dataset(benchmark: str, subset: str | None = None):
return PinchBenchDataset(path=subset)
elif benchmark == "taubench":
from openjarvis.evals.datasets.taubench import TauBenchDataset
domains = subset.split(",") if subset else None
return TauBenchDataset(domains=domains)
elif benchmark == "livecodebench":
@@ -478,6 +479,7 @@ def _build_scorer(benchmark: str, judge_backend, judge_model: str):
return PinchBenchScorer(judge_backend, judge_model)
elif benchmark == "taubench":
from openjarvis.evals.scorers.taubench import TauBenchScorer
return TauBenchScorer(judge_backend, judge_model)
elif benchmark == "livecodebench":
from openjarvis.evals.scorers.livecodebench import LiveCodeBenchScorer
@@ -597,6 +599,7 @@ def _run_terminalbench_native(config, console: Console) -> object:
)
import re
# Docker compose project names must be lowercase alphanumeric + hyphens/underscores
model_slug = re.sub(r"[^a-z0-9_-]", "-", model.lower().replace("/", "-"))
run_id = f"tb2-{model_slug}"
src/openjarvis/evals/core/runner.py
+3 -9
View File
@@ -361,9 +361,7 @@ class EvalRunner:
"completion_tokens", 0
)
if param_b > 0 and total_tokens > 0:
flops_per_tok = estimate_model_flops_per_token(
param_b, active_b
)
flops_per_tok = estimate_model_flops_per_token(param_b, active_b)
estimated_flops = flops_per_tok * total_tokens
# Extract derived and ITL metrics from _telemetry dict
@@ -1098,12 +1096,8 @@ def _summary_to_dict(s: RunSummary) -> Dict[str, Any]:
"energy_stats": _metric_stats_to_dict(s.energy_stats),
"power_stats": _metric_stats_to_dict(s.power_stats),
"flops_stats": _metric_stats_to_dict(s.flops_stats),
"ipw": (
s.efficiency.get("ipw") if s.efficiency else None
),
"ipj": (
s.efficiency.get("ipj") if s.efficiency else None
),
"ipw": (s.efficiency.get("ipw") if s.efficiency else None),
"ipj": (s.efficiency.get("ipj") if s.efficiency else None),
},
}
src/openjarvis/evals/datasets/livecodebench.py
+7 -13
View File
@@ -163,25 +163,17 @@ class LiveCodeBenchDataset(DatasetProvider):
# Extract title
title = str(
raw.get("question_title")
or raw.get("title")
or raw.get("name")
or ""
raw.get("question_title") or raw.get("title") or raw.get("name") or ""
).strip()
# Extract difficulty
difficulty = str(
raw.get("difficulty")
or raw.get("question_difficulty")
or ""
raw.get("difficulty") or raw.get("question_difficulty") or ""
).strip()
# Extract platform/source
platform = str(
raw.get("platform")
or raw.get("source")
or raw.get("contest_source")
or ""
raw.get("platform") or raw.get("source") or raw.get("contest_source") or ""
).strip()
# Extract input/output format (may not always be separate fields)
@@ -266,8 +258,10 @@ class LiveCodeBenchDataset(DatasetProvider):
metadata[key] = val
# Build subject from platform and difficulty
subject = f"{platform}/{difficulty}" if platform and difficulty else (
platform or difficulty or "competitive-programming"
subject = (
f"{platform}/{difficulty}"
if platform and difficulty
else (platform or difficulty or "competitive-programming")
)
return EvalRecord(
src/openjarvis/evals/datasets/taubench.py
+7 -2
View File
@@ -129,6 +129,7 @@ class TauBenchDataset(DatasetProvider):
# Load tasks, filtering to test split when available
from tau2.runner import load_task_splits
try:
task_splits = load_task_splits(domain)
test_ids = (
@@ -144,12 +145,14 @@ class TauBenchDataset(DatasetProvider):
tasks = [t for t in tasks if str(t.id) in test_ids]
LOGGER.info(
"TauBench: loaded %d test-split tasks for domain '%s'",
len(tasks), domain,
len(tasks),
domain,
)
else:
LOGGER.info(
"TauBench: loaded %d tasks for domain '%s' (no test split)",
len(tasks), domain,
len(tasks),
domain,
)
for task in tasks:
@@ -206,6 +209,7 @@ class TauBenchDataset(DatasetProvider):
if seed is not None:
import random
random.Random(seed).shuffle(all_records)
if max_samples is not None:
all_records = all_records[:max_samples]
@@ -226,6 +230,7 @@ class TauBenchDataset(DatasetProvider):
def create_task_env(self, record: EvalRecord):
"""Create a TauBench task environment for evaluation."""
from openjarvis.evals.execution.taubench_env import TauBenchTaskEnv
return TauBenchTaskEnv(
record,
engine_key=self._engine_key,
src/openjarvis/evals/datasets/toolcall15.py
+179 -153
View File
@@ -310,18 +310,18 @@ SCENARIOS: List[Dict[str, Any]] = [
"category": "A-ToolSelection",
"user_message": "What's the weather like in Berlin right now?",
"mock_tool_outputs": {
"get_weather": json.dumps({
"location": "Berlin",
"temperature": 8,
"units": "celsius",
"condition": "Overcast",
"humidity": 72,
}),
"web_search": json.dumps({
"results": [
{"snippet": "Berlin weather right now: 8C and overcast."}
]
}),
"get_weather": json.dumps(
{
"location": "Berlin",
"temperature": 8,
"units": "celsius",
"condition": "Overcast",
"humidity": 72,
}
),
"web_search": json.dumps(
{"results": [{"snippet": "Berlin weather right now: 8C and overcast."}]}
),
},
},
{
@@ -330,18 +330,18 @@ SCENARIOS: List[Dict[str, Any]] = [
"category": "A-ToolSelection",
"user_message": "What is the current price of AAPL stock?",
"mock_tool_outputs": {
"get_stock_price": json.dumps({
"ticker": "AAPL",
"price": 187.42,
"currency": "USD",
"change": "+1.23",
"change_percent": "+0.66%",
}),
"web_search": json.dumps({
"results": [
{"snippet": "AAPL is trading around $187.42."}
]
}),
"get_stock_price": json.dumps(
{
"ticker": "AAPL",
"price": 187.42,
"currency": "USD",
"change": "+1.23",
"change_percent": "+0.66%",
}
),
"web_search": json.dumps(
{"results": [{"snippet": "AAPL is trading around $187.42."}]}
),
},
},
{
@@ -350,15 +350,15 @@ SCENARIOS: List[Dict[str, Any]] = [
"category": "A-ToolSelection",
"user_message": "I need to let Sarah know the meeting moved to 3pm.",
"mock_tool_outputs": {
"get_contacts": json.dumps({
"results": [
{"name": "Sarah Chen", "email": "sarah.chen@company.com"}
]
}),
"send_email": json.dumps({
"status": "sent",
"message_id": "msg_8821",
}),
"get_contacts": json.dumps(
{"results": [{"name": "Sarah Chen", "email": "sarah.chen@company.com"}]}
),
"send_email": json.dumps(
{
"status": "sent",
"message_id": "msg_8821",
}
),
},
},
# --- Category B: Parameter Precision ---
@@ -368,21 +368,25 @@ SCENARIOS: List[Dict[str, Any]] = [
"category": "B-ParameterPrecision",
"user_message": "What's the temperature in Tokyo in Fahrenheit?",
"mock_tool_outputs": {
"get_weather": json.dumps({
"location": "Tokyo",
"temperature": 64,
"units": "fahrenheit",
"condition": "Clear",
"humidity": 55,
}),
"get_weather": json.dumps(
{
"location": "Tokyo",
"temperature": 64,
"units": "fahrenheit",
"condition": "Clear",
"humidity": 55,
}
),
# Default (celsius) mock for partial-credit scoring
"get_weather__default": json.dumps({
"location": "Tokyo",
"temperature": 18,
"units": "celsius",
"condition": "Clear",
"humidity": 55,
}),
"get_weather__default": json.dumps(
{
"location": "Tokyo",
"temperature": 18,
"units": "celsius",
"condition": "Clear",
"humidity": 55,
}
),
},
},
{
@@ -394,18 +398,22 @@ SCENARIOS: List[Dict[str, Any]] = [
"30 minutes, with Alex and Jamie."
),
"mock_tool_outputs": {
"get_contacts": json.dumps({
"results": [
{"name": "Alex Stone", "email": "alex.stone@company.com"},
{"name": "Jamie Liu", "email": "jamie.liu@company.com"},
]
}),
"create_calendar_event": json.dumps({
"event_id": "evt_4412",
"status": "created",
"title": "Team Standup",
"date": "2026-03-23",
}),
"get_contacts": json.dumps(
{
"results": [
{"name": "Alex Stone", "email": "alex.stone@company.com"},
{"name": "Jamie Liu", "email": "jamie.liu@company.com"},
]
}
),
"create_calendar_event": json.dumps(
{
"event_id": "evt_4412",
"status": "created",
"title": "Team Standup",
"date": "2026-03-23",
}
),
},
# Reference date is 2026-03-20 (Friday), next Monday = 2026-03-23
"reference_date": "2026-03-20",
@@ -419,12 +427,16 @@ SCENARIOS: List[Dict[str, Any]] = [
"from English to both Spanish and Japanese."
),
"mock_tool_outputs": {
"translate_text__spanish": json.dumps({
"translated": "\u00bfD\u00f3nde est\u00e1 el hospital m\u00e1s cercano?",
}),
"translate_text__japanese": json.dumps({
"translated": "\u6700\u5bc4\u308a\u306e\u75c5\u9662\u306f\u3069\u3053\u3067\u3059\u304b\uff1f",
}),
"translate_text__spanish": json.dumps(
{
"translated": "\u00bfD\u00f3nde est\u00e1 el hospital m\u00e1s cercano?",
}
),
"translate_text__japanese": json.dumps(
{
"translated": "\u6700\u5bc4\u308a\u306e\u75c5\u9662\u306f\u3069\u3053\u3067\u3059\u304b\uff1f",
}
),
},
},
# --- Category C: Multi-Step Chains ---
@@ -436,29 +448,35 @@ SCENARIOS: List[Dict[str, Any]] = [
"Find the Q3 budget report and email the total to my manager."
),
"mock_tool_outputs": {
"search_files": json.dumps({
"results": [
{
"file_id": "file_091",
"name": "Q3_Budget_Report_2025.xlsx",
}
]
}),
"read_file": json.dumps({
"content": (
"Department budgets: Engineering $2.1M, Marketing $800K, "
"Sales $1.5M. Total: $4.4M"
),
}),
"get_contacts": json.dumps({
"results": [
{
"name": "Jordan Park",
"email": "jordan.park@company.com",
"role": "manager",
}
]
}),
"search_files": json.dumps(
{
"results": [
{
"file_id": "file_091",
"name": "Q3_Budget_Report_2025.xlsx",
}
]
}
),
"read_file": json.dumps(
{
"content": (
"Department budgets: Engineering $2.1M, Marketing $800K, "
"Sales $1.5M. Total: $4.4M"
),
}
),
"get_contacts": json.dumps(
{
"results": [
{
"name": "Jordan Park",
"email": "jordan.park@company.com",
"role": "manager",
}
]
}
),
"send_email": json.dumps({"status": "sent"}),
},
},
@@ -471,16 +489,20 @@ SCENARIOS: List[Dict[str, Any]] = [
"remind me to bring an umbrella tomorrow at 8am."
),
"mock_tool_outputs": {
"get_weather": json.dumps({
"location": "Paris",
"temperature": 11,
"condition": "Light rain",
"humidity": 89,
}),
"set_reminder": json.dumps({
"reminder_id": "rem_553",
"status": "set",
}),
"get_weather": json.dumps(
{
"location": "Paris",
"temperature": 11,
"condition": "Light rain",
"humidity": 89,
}
),
"set_reminder": json.dumps(
{
"reminder_id": "rem_553",
"status": "set",
}
),
},
"reference_date": "2026-03-20",
},
@@ -488,29 +510,33 @@ SCENARIOS: List[Dict[str, Any]] = [
"id": "TC-09",
"name": "Parallel Independence",
"category": "C-MultiStepChains",
"user_message": (
"What's the weather in London and the stock price of MSFT?"
),
"user_message": ("What's the weather in London and the stock price of MSFT?"),
"mock_tool_outputs": {
"get_weather": json.dumps({
"location": "London",
"temperature": 12,
"condition": "Cloudy",
}),
"get_stock_price": json.dumps({
"ticker": "MSFT",
"price": 412.78,
"currency": "USD",
}),
"web_search": json.dumps({
"results": [
{
"snippet": (
"London is cloudy at 12C and MSFT is around $412.78."
)
}
]
}),
"get_weather": json.dumps(
{
"location": "London",
"temperature": 12,
"condition": "Cloudy",
}
),
"get_stock_price": json.dumps(
{
"ticker": "MSFT",
"price": 412.78,
"currency": "USD",
}
),
"web_search": json.dumps(
{
"results": [
{
"snippet": (
"London is cloudy at 12C and MSFT is around $412.78."
)
}
]
}
),
},
},
# --- Category D: Restraint & Refusal ---
@@ -546,14 +572,16 @@ SCENARIOS: List[Dict[str, Any]] = [
"mock_tool_outputs": {
# First call returns empty, second (broader) returns a result
"search_files__first": json.dumps({"results": []}),
"search_files__retry": json.dumps({
"results": [
{
"file_id": "file_117",
"name": "Johnson_Project_Proposal_v2.docx",
}
]
}),
"search_files__retry": json.dumps(
{
"results": [
{
"file_id": "file_117",
"name": "Johnson_Project_Proposal_v2.docx",
}
]
}
),
},
},
{
@@ -562,14 +590,14 @@ SCENARIOS: List[Dict[str, Any]] = [
"category": "E-ErrorRecovery",
"user_message": "What's Apple's stock price?",
"mock_tool_outputs": {
"get_stock_price": json.dumps({
"error": "Service temporarily unavailable. Rate limit exceeded.",
}),
"web_search": json.dumps({
"results": [
{"snippet": "Apple (AAPL) is trading around $187.42."}
]
}),
"get_stock_price": json.dumps(
{
"error": "Service temporarily unavailable. Rate limit exceeded.",
}
),
"web_search": json.dumps(
{"results": [{"snippet": "Apple (AAPL) is trading around $187.42."}]}
),
},
},
{
@@ -577,20 +605,21 @@ SCENARIOS: List[Dict[str, Any]] = [
"name": "Conflicting Information",
"category": "E-ErrorRecovery",
"user_message": (
"Search for the population of Iceland "
"and calculate what 2% of it would be."
"Search for the population of Iceland and calculate what 2% of it would be."
),
"mock_tool_outputs": {
"web_search": json.dumps({
"results": [
{
"snippet": (
"Iceland has a population of approximately "
"372,520 as of 2025."
)
}
]
}),
"web_search": json.dumps(
{
"results": [
{
"snippet": (
"Iceland has a population of approximately "
"372,520 as of 2025."
)
}
]
}
),
"calculator": json.dumps({"result": 7450.4}),
},
},
@@ -629,9 +658,7 @@ class ToolCall15Dataset(DatasetProvider):
scenarios = [
s
for s in scenarios
if any(
s["category"].upper().startswith(fc) for fc in filter_cats
)
if any(s["category"].upper().startswith(fc) for fc in filter_cats)
]
if seed is not None:
@@ -645,8 +672,7 @@ class ToolCall15Dataset(DatasetProvider):
# instructions, available tools, and user message so the
# model can respond with tool calls via any backend.
tool_descriptions = "\n".join(
f"- {t['function']['name']}: "
f"{t['function']['description']}"
f"- {t['function']['name']}: {t['function']['description']}"
for t in TOOLS
)
prompt = (
src/openjarvis/evals/execution/taubench_env.py
+18 -9
View File
@@ -22,6 +22,7 @@ LOGGER = logging.getLogger(__name__)
# Message conversion helpers
# ---------------------------------------------------------------------------
def _tau2_to_oj_messages(
tau2_messages: list,
) -> list:
@@ -59,13 +60,12 @@ def _tau2_to_oj_messages(
)
)
else:
oj_msgs.append(
Message(role=Role.ASSISTANT, content=m.content or "")
)
oj_msgs.append(Message(role=Role.ASSISTANT, content=m.content or ""))
elif role_str == "tool":
# tau2 ToolMessage uses 'id', not 'tool_call_id'
raw_id = getattr(m, "id", "") or getattr(m, "tool_call_id", "") or ""
import re as _re
clean_id = _re.sub(r"[^a-zA-Z0-9_-]", "_", raw_id)
oj_msgs.append(
Message(
@@ -111,6 +111,7 @@ def _oj_result_to_tau2_msg(result: dict):
# Jarvis-powered tau2 agent
# ---------------------------------------------------------------------------
class JarvisHalfDuplexAgent:
"""A tau2 HalfDuplexAgent backed by OpenJarvis's inference engine.
@@ -139,6 +140,7 @@ class JarvisHalfDuplexAgent:
@property
def system_prompt(self) -> str:
from tau2.agent.llm_agent import AGENT_INSTRUCTION, SYSTEM_PROMPT
return SYSTEM_PROMPT.format(
domain_policy=self.domain_policy,
agent_instruction=AGENT_INSTRUCTION,
@@ -147,6 +149,7 @@ class JarvisHalfDuplexAgent:
def get_init_state(self, message_history=None):
from tau2.agent.llm_agent import LLMAgentState
from tau2.data_model.message import SystemMessage
return LLMAgentState(
system_messages=[
SystemMessage(role="system", content=self.system_prompt),
@@ -211,6 +214,7 @@ class JarvisHalfDuplexAgent:
# Task environment
# ---------------------------------------------------------------------------
class TauBenchTaskEnv:
"""Per-task environment for TauBench evaluation.
@@ -315,9 +319,7 @@ class TauBenchTaskEnv:
)
reward = (
simulation.reward_info.reward
if simulation.reward_info
else 0.0
simulation.reward_info.reward if simulation.reward_info else 0.0
)
info: dict = {}
if simulation.reward_info:
@@ -331,11 +333,16 @@ class TauBenchTaskEnv:
trial_label = (
f" (trial {trial + 1}/{self._num_trials})"
if self._num_trials > 1 else ""
if self._num_trials > 1
else ""
)
LOGGER.info(
"TauBench %s/%s: reward=%.2f messages=%d%s",
domain, task_id, reward, n_messages, trial_label,
domain,
task_id,
reward,
n_messages,
trial_label,
)
if reward > best_reward:
@@ -350,7 +357,9 @@ class TauBenchTaskEnv:
except Exception as exc:
LOGGER.error(
"TauBench simulation failed for %s/%s: %s",
domain, task_id, exc,
domain,
task_id,
exc,
)
self._record.metadata["tau_reward"] = best_reward
src/openjarvis/evals/scorers/livecodebench.py
+22 -10
View File
@@ -43,11 +43,21 @@ def _extract_code(answer: str) -> str:
in_code = False
for line in lines:
stripped = line.lstrip()
if stripped.startswith((
"def ", "class ", "from ", "import ",
"n = ", "t = ", "for ", "while ",
"input(", "sys.stdin", "print(",
)):
if stripped.startswith(
(
"def ",
"class ",
"from ",
"import ",
"n = ",
"t = ",
"for ",
"while ",
"input(",
"sys.stdin",
"print(",
)
):
in_code = True
if in_code:
code_lines.append(line)
@@ -211,11 +221,13 @@ class LiveCodeBenchScorer(Scorer):
ok, detail = _run_single_test(code, inp, expected, timeout=timeout)
if ok:
passed += 1
test_details.append({
"test_index": i,
"passed": ok,
"detail": detail,
})
test_details.append(
{
"test_index": i,
"passed": ok,
"detail": detail,
}
)
if total == 0:
return None, {"reason": "no_test_cases_after_filtering"}
src/openjarvis/evals/scorers/liveresearch.py
+1 -3
View File
@@ -277,9 +277,7 @@ class LiveResearchBenchScorer(LLMJudgeScorer):
try:
raw = self._ask_judge(prompt, temperature=0.0, max_tokens=4096)
except Exception as exc:
LOGGER.error(
"LLM judge call failed for %s: %s", record.record_id, exc
)
LOGGER.error("LLM judge call failed for %s: %s", record.record_id, exc)
return None, {"error": str(exc), "score": 0.0}
parsed = _parse_judge_response(raw)
src/openjarvis/evals/scorers/taubench.py
+3 -1
View File
@@ -28,7 +28,9 @@ class TauBenchScorer(Scorer):
self._judge_model = judge_model
def score(
self, record: EvalRecord, model_answer: str,
self,
record: EvalRecord,
model_answer: str,
) -> Tuple[Optional[bool], Dict[str, Any]]:
reward = record.metadata.get("tau_reward", 0.0)
info = record.metadata.get("tau_info", {})
src/openjarvis/evals/scorers/toolcall15.py
+88 -60
View File
@@ -41,20 +41,24 @@ def _extract_tool_calls(
for tc in getattr(turn, "tool_calls", []):
if tc is None:
continue
tool_calls.append({
"name": tc.get("name", ""),
"arguments": tc.get("arguments") or {},
})
tool_calls.append(
{
"name": tc.get("name", ""),
"arguments": tc.get("arguments") or {},
}
)
if tool_calls:
return tool_calls
# Try tool_results list (from JarvisAgentBackend)
tool_results = record.metadata.get("tool_results", [])
for tr in tool_results:
tool_calls.append({
"name": tr.get("tool_name", ""),
"arguments": tr.get("arguments") or {},
})
tool_calls.append(
{
"name": tr.get("tool_name", ""),
"arguments": tr.get("arguments") or {},
}
)
if tool_calls:
return tool_calls
@@ -133,7 +137,8 @@ def _arg_equals(tc: Dict[str, Any], key: str, value: str) -> bool:
def _score_tc01(
tool_calls: List[Dict[str, Any]], answer: str,
tool_calls: List[Dict[str, Any]],
answer: str,
) -> Tuple[int, str]:
"""TC-01: Direct Specialist Match — get_weather for Berlin."""
weather = _tools_called(tool_calls, "get_weather")
@@ -151,7 +156,8 @@ def _score_tc01(
def _score_tc02(
tool_calls: List[Dict[str, Any]], answer: str,
tool_calls: List[Dict[str, Any]],
answer: str,
) -> Tuple[int, str]:
"""TC-02: Distractor Resistance — get_stock_price for AAPL only."""
stock = _tools_called(tool_calls, "get_stock_price")
@@ -169,16 +175,15 @@ def _score_tc02(
def _score_tc03(
tool_calls: List[Dict[str, Any]], answer: str,
tool_calls: List[Dict[str, Any]],
answer: str,
) -> Tuple[int, str]:
"""TC-03: Implicit Tool Need — get_contacts then send_email."""
contacts = _tools_called(tool_calls, "get_contacts")
email = _tools_called(tool_calls, "send_email")
if contacts and email:
contact_has_sarah = any(
_arg_contains(tc, "query", "sarah") for tc in contacts
)
contact_has_sarah = any(_arg_contains(tc, "query", "sarah") for tc in contacts)
email_has_addr = any(
_arg_contains(tc, "to", "sarah.chen@company.com") for tc in email
)
@@ -197,7 +202,8 @@ def _score_tc03(
def _score_tc04(
tool_calls: List[Dict[str, Any]], answer: str,
tool_calls: List[Dict[str, Any]],
answer: str,
) -> Tuple[int, str]:
"""TC-04: Unit Handling — get_weather(Tokyo, units=fahrenheit)."""
weather = _tools_called(tool_calls, "get_weather")
@@ -219,7 +225,8 @@ def _score_tc04(
def _score_tc05(
tool_calls: List[Dict[str, Any]], answer: str,
tool_calls: List[Dict[str, Any]],
answer: str,
) -> Tuple[int, str]:
"""TC-05: Date and Time Parsing — create_calendar_event with correct fields."""
events = _tools_called(tool_calls, "create_calendar_event")
@@ -239,7 +246,9 @@ def _score_tc05(
time_ok = time_val in ("09:30", "9:30")
duration_ok = duration == 30 or str(duration) == "30"
attendees_lower = [a.lower() if isinstance(a, str) else "" for a in (attendees or [])]
attendees_lower = [
a.lower() if isinstance(a, str) else "" for a in (attendees or [])
]
attendees_str = " ".join(attendees_lower)
has_alex = "alex" in attendees_str
has_jamie = "jamie" in attendees_str
@@ -252,7 +261,8 @@ def _score_tc05(
def _score_tc06(
tool_calls: List[Dict[str, Any]], answer: str,
tool_calls: List[Dict[str, Any]],
answer: str,
) -> Tuple[int, str]:
"""TC-06: Multi-Value Extraction — two translate_text calls."""
translates = _tools_called(tool_calls, "translate_text")
@@ -278,7 +288,8 @@ def _score_tc06(
def _score_tc07(
tool_calls: List[Dict[str, Any]], answer: str,
tool_calls: List[Dict[str, Any]],
answer: str,
) -> Tuple[int, str]:
"""TC-07: Search -> Read -> Act — 4-step chain."""
search = _tools_called(tool_calls, "search_files")
@@ -286,24 +297,20 @@ def _score_tc07(
contacts = _tools_called(tool_calls, "get_contacts")
email = _tools_called(tool_calls, "send_email")
steps_done = sum([
bool(search),
bool(read),
bool(contacts),
bool(email),
])
steps_done = sum(
[
bool(search),
bool(read),
bool(contacts),
bool(email),
]
)
if steps_done == 4:
# Verify data threading
has_file_id = any(
_arg_contains(tc, "file_id", "file_091") for tc in read
)
has_manager = any(
_arg_contains(tc, "query", "manager") for tc in contacts
)
has_total = any(
_arg_contains(tc, "body", "4.4") for tc in email
)
has_file_id = any(_arg_contains(tc, "file_id", "file_091") for tc in read)
has_manager = any(_arg_contains(tc, "query", "manager") for tc in contacts)
has_total = any(_arg_contains(tc, "body", "4.4") for tc in email)
email_to = any(
_arg_contains(tc, "to", "jordan.park@company.com") for tc in email
)
@@ -317,16 +324,15 @@ def _score_tc07(
def _score_tc08(
tool_calls: List[Dict[str, Any]], answer: str,
tool_calls: List[Dict[str, Any]],
answer: str,
) -> Tuple[int, str]:
"""TC-08: Conditional Branching — weather check then conditional reminder."""
weather = _tools_called(tool_calls, "get_weather")
reminder = _tools_called(tool_calls, "set_reminder")
if weather and reminder:
weather_paris = any(
_arg_contains(tc, "location", "paris") for tc in weather
)
weather_paris = any(_arg_contains(tc, "location", "paris") for tc in weather)
reminder_umbrella = any(
_arg_contains(tc, "message", "umbrella") for tc in reminder
)
@@ -345,19 +351,16 @@ def _score_tc08(
def _score_tc09(
tool_calls: List[Dict[str, Any]], answer: str,
tool_calls: List[Dict[str, Any]],
answer: str,
) -> Tuple[int, str]:
"""TC-09: Parallel Independence — weather AND stock price."""
weather = _tools_called(tool_calls, "get_weather")
stock = _tools_called(tool_calls, "get_stock_price")
web = _tools_called(tool_calls, "web_search")
weather_london = any(
_arg_contains(tc, "location", "london") for tc in weather
)
stock_msft = any(
_arg_contains(tc, "ticker", "msft") for tc in stock
)
weather_london = any(_arg_contains(tc, "location", "london") for tc in weather)
stock_msft = any(_arg_contains(tc, "ticker", "msft") for tc in stock)
if weather_london and stock_msft:
return 2, "PASS: both get_weather(London) and get_stock_price(MSFT)"
@@ -370,7 +373,8 @@ def _score_tc09(
def _score_tc10(
tool_calls: List[Dict[str, Any]], answer: str,
tool_calls: List[Dict[str, Any]],
answer: str,
) -> Tuple[int, str]:
"""TC-10: Trivial Knowledge — answer without tools."""
if not tool_calls and "1945" in answer:
@@ -381,7 +385,8 @@ def _score_tc10(
def _score_tc11(
tool_calls: List[Dict[str, Any]], answer: str,
tool_calls: List[Dict[str, Any]],
answer: str,
) -> Tuple[int, str]:
"""TC-11: Simple Math — answer without tools."""
if not tool_calls and "30" in answer:
@@ -400,14 +405,22 @@ def _score_tc11(
def _score_tc12(
tool_calls: List[Dict[str, Any]], answer: str,
tool_calls: List[Dict[str, Any]],
answer: str,
) -> Tuple[int, str]:
"""TC-12: Impossible Request — refuse without tool calls."""
lower = answer.lower()
refusal_phrases = [
"cannot", "can't", "do not have", "don't have",
"not able", "unable", "no tool", "not available",
"not possible", "no way to",
"cannot",
"can't",
"do not have",
"don't have",
"not able",
"unable",
"no tool",
"not available",
"not possible",
"no way to",
]
has_refusal = any(phrase in lower for phrase in refusal_phrases)
mentions_delete = "delete" in lower or "available tool" in lower or "tool" in lower
@@ -422,7 +435,8 @@ def _score_tc12(
def _score_tc13(
tool_calls: List[Dict[str, Any]], answer: str,
tool_calls: List[Dict[str, Any]],
answer: str,
) -> Tuple[int, str]:
"""TC-13: Empty Results — retry or ask for clarification."""
search = _tools_called(tool_calls, "search_files")
@@ -432,8 +446,15 @@ def _score_tc13(
if len(search) == 1:
lower = answer.lower()
clarification_phrases = [
"could you", "can you", "more details", "more specific",
"clarify", "which", "what type", "not found", "no results",
"could you",
"can you",
"more details",
"more specific",
"clarify",
"which",
"what type",
"not found",
"no results",
]
if any(phrase in lower for phrase in clarification_phrases):
return 2, "PASS: single search with clarification request"
@@ -441,7 +462,8 @@ def _score_tc13(
def _score_tc14(
tool_calls: List[Dict[str, Any]], answer: str,
tool_calls: List[Dict[str, Any]],
answer: str,
) -> Tuple[int, str]:
"""TC-14: Malformed Response — surface error, offer fallback."""
stock = _tools_called(tool_calls, "get_stock_price")
@@ -449,8 +471,13 @@ def _score_tc14(
lower = answer.lower()
error_ack_phrases = [
"temporarily unavailable", "rate limit", "service",
"couldn't", "error", "unable", "failed",
"temporarily unavailable",
"rate limit",
"service",
"couldn't",
"error",
"unable",
"failed",
"get_stock_price",
]
has_error_ack = any(phrase in lower for phrase in error_ack_phrases)
@@ -466,15 +493,16 @@ def _score_tc14(
def _score_tc15(
tool_calls: List[Dict[str, Any]], answer: str,
tool_calls: List[Dict[str, Any]],
answer: str,
) -> Tuple[int, str]:
"""TC-15: Conflicting Information — use search result in calculator."""
web = _tools_called(tool_calls, "web_search")
calc = _tools_called(tool_calls, "calculator")
web_has_iceland = any(
_arg_contains(tc, "query", "iceland") or
_arg_contains(tc, "query", "population")
_arg_contains(tc, "query", "iceland")
or _arg_contains(tc, "query", "population")
for tc in web
)
src/openjarvis/sdk.py
+28
View File
@@ -482,6 +482,34 @@ class Jarvis:
if self._capability_policy is not None:
agent_kwargs["capability_policy"] = self._capability_policy
# Inject DigestConfig for morning_digest agent
if agent_name == "morning_digest" and hasattr(self._config, "digest"):
dc = self._config.digest
section_sources: Dict[str, Any] = {}
for s in dc.sections:
sc = getattr(dc, s, None)
if sc and hasattr(sc, "sources"):
section_sources[s] = sc.sources
agent_kwargs.update(
{
"persona": dc.persona,
"sections": dc.sections,
"section_sources": section_sources,
"timezone": dc.timezone,
"voice_id": dc.voice_id,
"voice_speed": dc.voice_speed,
"tts_backend": dc.tts_backend,
"honorific": dc.honorific,
}
)
# Ensure digest agent always has its required tools
from openjarvis.tools.digest_collect import DigestCollectTool
from openjarvis.tools.text_to_speech import TextToSpeechTool
digest_tools = [DigestCollectTool(), TextToSpeechTool()]
existing = agent_kwargs.get("tools", [])
agent_kwargs["tools"] = digest_tools + list(existing)
agent_obj = agent_cls(self._engine, model_name, **agent_kwargs)
ctx = AgentContext()
src/openjarvis/server/agent_manager_routes.py
+70 -71
View File
@@ -112,7 +112,8 @@ def _make_lightweight_system(
)
plain_engine = InstrumentedEngine(
plain_engine, get_event_bus(),
plain_engine,
get_event_bus(),
)
except Exception:
pass # telemetry is optional
@@ -559,7 +560,8 @@ async def _stream_managed_agent(
agent_type = agent_record.get("agent_type", "")
if agent_type == "deep_research":
dr_tools = _build_deep_research_tools(
engine=engine, model=model,
engine=engine,
model=model,
)
# Store on app_state so streaming loop can access them
if app_state is not None and dr_tools:
@@ -628,9 +630,7 @@ async def _stream_managed_agent(
def _tracked_execute(tc):
tool_name = tc.name
args_str = (
tc.arguments[:80] if tc.arguments else ""
)
args_str = tc.arguments[:80] if tc.arguments else ""
# Log tool call start
try:
manager.add_learning_log(
@@ -642,11 +642,13 @@ async def _stream_managed_agent(
except Exception as _tc_exc:
logger.warning("Log tool_call failed: %s", _tc_exc)
progress_q.put({
"type": "tool_start",
"tool": tool_name,
"args": args_str,
})
progress_q.put(
{
"type": "tool_start",
"tool": tool_name,
"args": args_str,
}
)
result = original_execute(tc)
# Log tool result
@@ -666,11 +668,13 @@ async def _stream_managed_agent(
except Exception as _tr_exc:
logger.warning("Log tool_result failed: %s", _tr_exc)
progress_q.put({
"type": "tool_end",
"tool": tool_name,
"success": result.success,
})
progress_q.put(
{
"type": "tool_end",
"tool": tool_name,
"success": result.success,
}
)
return result
dr_agent._executor.execute = _tracked_execute
@@ -679,9 +683,7 @@ async def _stream_managed_agent(
agent_metadata = {}
try:
result = dr_agent.run(user_content)
content = (
result.content or "No results found."
)
content = result.content or "No results found."
agent_metadata = result.metadata or {}
except Exception as exc:
content = f"Error: {exc}"
@@ -703,15 +705,18 @@ async def _stream_managed_agent(
)
except Exception as _qc_exc:
logger.warning(
"Log failed: %s", _qc_exc,
"Log failed: %s",
_qc_exc,
)
progress_q.put({
"type": "error" if content.startswith("Error:") else "done",
"content": content,
"metadata": agent_metadata,
"elapsed": elapsed,
})
progress_q.put(
{
"type": "error" if content.startswith("Error:") else "done",
"content": content,
"metadata": agent_metadata,
"elapsed": elapsed,
}
)
thread = threading.Thread(target=_run_agent, daemon=True)
thread.start()
@@ -795,7 +800,8 @@ async def _stream_managed_agent(
# Persist
manager.store_agent_response(
agent_id, content,
agent_id,
content,
)
break
@@ -1235,7 +1241,9 @@ def create_agent_manager_router(
@agents_router.post("/{agent_id}/channels")
async def bind_channel(
agent_id: str, req: BindChannelRequest, request: Request,
agent_id: str,
req: BindChannelRequest,
request: Request,
):
if not manager.get_agent(agent_id):
raise HTTPException(status_code=404, detail="Agent not found")
@@ -1315,9 +1323,7 @@ def create_agent_manager_router(
request.app.state.sendblue_channel = sb_channel
# Create or update the channel bridge
bridge = getattr(
request.app.state, "channel_bridge", None
)
bridge = getattr(request.app.state, "channel_bridge", None)
if bridge and hasattr(bridge, "_channels"):
bridge._channels["sendblue"] = sb_channel
else:
@@ -1330,39 +1336,30 @@ def create_agent_manager_router(
)
session_store = SessionStore()
engine = getattr(
request.app.state, "engine", None
)
engine = getattr(request.app.state, "engine", None)
dr_agent = None
if engine:
from openjarvis.server.agent_manager_routes import (
_build_deep_research_tools as _bdr,
)
tools = _bdr(
engine=engine, model=""
)
tools = _bdr(engine=engine, model="")
if tools:
from openjarvis.agents.deep_research import (
DeepResearchAgent,
)
model_name = (
getattr(engine, "_model", "")
or getattr(
request.app.state,
"model",
"",
)
model_name = getattr(engine, "_model", "") or getattr(
request.app.state,
"model",
"",
)
dr_agent = DeepResearchAgent(
engine=engine,
model=model_name,
tools=tools,
)
bus = getattr(
request.app.state, "bus", None
)
bus = getattr(request.app.state, "bus", None)
if bus is None:
from openjarvis.core.events import EventBus
@@ -1381,9 +1378,7 @@ def create_agent_manager_router(
from_number,
)
except Exception as exc:
logger.warning(
"Failed to init SendBlue channel: %s", exc
)
logger.warning("Failed to init SendBlue channel: %s", exc)
# Start Slack via slack-bolt Socket Mode
if req.channel_type == "slack":
@@ -1403,31 +1398,40 @@ def create_agent_manager_router(
stop_slack()
# Spawn as subprocess (reliable)
srv_model = getattr(
srv_model = (
getattr(
request.app.state, "engine", None,
),
"_model",
"qwen3.5:9b",
) or "qwen3.5:9b"
getattr(
request.app.state,
"engine",
None,
),
"_model",
"qwen3.5:9b",
)
or "qwen3.5:9b"
)
pid = start_slack_daemon(
bot_token=bot_token,
app_token=app_token,
model=srv_model,
)
logger.info(
"Slack daemon started (PID %d)", pid,
"Slack daemon started (PID %d)",
pid,
)
except Exception as exc:
logger.warning(
"Failed to start Slack: %s", exc,
"Failed to start Slack: %s",
exc,
)
return binding
@agents_router.delete("/{agent_id}/channels/{binding_id}")
async def unbind_channel(
agent_id: str, binding_id: str, request: Request,
agent_id: str,
binding_id: str,
request: Request,
):
try:
binding = manager._get_binding(binding_id)
@@ -1747,9 +1751,7 @@ def create_agent_manager_router(
# ── SendBlue auto-setup helpers ─────────────────────────
sendblue_router = APIRouter(
prefix="/v1/channels/sendblue", tags=["sendblue"]
)
sendblue_router = APIRouter(prefix="/v1/channels/sendblue", tags=["sendblue"])
@sendblue_router.post("/verify")
async def sendblue_verify(request: Request):
@@ -1786,8 +1788,10 @@ def create_agent_manager_router(
)
data = resp.json()
# data might be a list of lines or {"lines": [...]}
lines = data if isinstance(data, list) else data.get(
"lines", data.get("data", [])
lines = (
data
if isinstance(data, list)
else data.get("lines", data.get("data", []))
)
numbers = []
for line in lines:
@@ -1843,9 +1847,7 @@ def create_agent_manager_router(
return {
"registered": resp.status_code < 300,
"status": resp.status_code,
"response": resp.json()
if resp.status_code < 300
else resp.text[:200],
"response": resp.json() if resp.status_code < 300 else resp.text[:200],
}
except Exception as exc:
raise HTTPException(
@@ -1894,9 +1896,7 @@ def create_agent_manager_router(
return {
"sent": resp.status_code < 300,
"status": resp.status_code,
"response": resp.json()
if resp.status_code < 300
else resp.text[:200],
"response": resp.json() if resp.status_code < 300 else resp.text[:200],
}
except Exception as exc:
raise HTTPException(
@@ -1910,8 +1910,7 @@ def create_agent_manager_router(
sb = getattr(request.app.state, "sendblue_channel", None)
bridge = getattr(request.app.state, "channel_bridge", None)
has_bridge = bridge is not None and (
hasattr(bridge, "_channels")
and "sendblue" in bridge._channels
hasattr(bridge, "_channels") and "sendblue" in bridge._channels
)
return {
"channel_connected": sb is not None,
src/openjarvis/server/app.py
+7 -6
View File
@@ -14,6 +14,7 @@ from openjarvis.server.api_routes import include_all_routes
from openjarvis.server.comparison import comparison_router
from openjarvis.server.connectors_router import create_connectors_router
from openjarvis.server.dashboard import dashboard_router
from openjarvis.server.digest_routes import create_digest_router
from openjarvis.server.routes import router
from openjarvis.server.upload_router import router as upload_router
@@ -72,17 +73,15 @@ def _restore_sendblue_bindings(app: FastAPI) -> None:
_build_deep_research_tools,
)
tools = _build_deep_research_tools(
engine=engine, model=""
)
tools = _build_deep_research_tools(engine=engine, model="")
if tools:
from openjarvis.agents.deep_research import (
DeepResearchAgent,
)
model_name = getattr(
app.state, "model", ""
) or getattr(engine, "_model", "")
model_name = getattr(app.state, "model", "") or getattr(
engine, "_model", ""
)
dr_agent = DeepResearchAgent(
engine=engine,
model=model_name,
@@ -111,6 +110,7 @@ def _restore_sendblue_bindings(app: FastAPI) -> None:
except Exception as exc:
logger.debug("SendBlue binding restore skipped: %s", exc)
# No-cache headers applied to static file responses
_NO_CACHE_HEADERS = {
"Cache-Control": "no-cache, no-store, must-revalidate",
@@ -223,6 +223,7 @@ def create_app(
app.include_router(dashboard_router)
app.include_router(comparison_router)
app.include_router(create_connectors_router())
app.include_router(create_digest_router())
app.include_router(upload_router)
include_all_routes(app)
src/openjarvis/server/connectors_router.py
+159 -1
View File
@@ -78,7 +78,7 @@ def create_connectors_router():
this package.
"""
try:
from fastapi import APIRouter, HTTPException
from fastapi import APIRouter, HTTPException, Request
except ImportError as exc:
raise ImportError(
"fastapi and pydantic are required for the connectors router"
@@ -175,6 +175,26 @@ def create_connectors_router():
except Exception:
pass
# Include OAuth provider setup info if applicable
oauth_setup = None
try:
from openjarvis.connectors.oauth import (
get_client_credentials,
get_provider_for_connector,
)
provider = get_provider_for_connector(connector_id)
if provider:
has_creds = get_client_credentials(provider) is not None
oauth_setup = {
"provider": provider.name,
"setup_url": provider.setup_url,
"setup_hint": provider.setup_hint,
"has_credentials": has_creds,
}
except Exception:
pass
return {
"connector_id": connector_id,
"display_name": getattr(instance, "display_name", connector_id),
@@ -182,6 +202,7 @@ def create_connectors_router():
"connected": instance.is_connected(),
"auth_url": auth_url,
"mcp_tools": mcp_tools,
"oauth_setup": oauth_setup,
}
@router.post("/{connector_id}/connect")
@@ -282,6 +303,143 @@ def create_connectors_router():
"status": "disconnected",
}
@router.get("/{connector_id}/oauth/start")
async def oauth_start(connector_id: str, request: Request):
"""Redirect to the OAuth provider's consent page.
The callback will come back to /v1/connectors/{id}/oauth/callback.
"""
from urllib.parse import urlencode
from openjarvis.connectors.oauth import (
get_client_credentials,
get_provider_for_connector,
)
_ensure_connectors_registered()
if not ConnectorRegistry.contains(connector_id):
raise HTTPException(404, f"Connector '{connector_id}' not found")
provider = get_provider_for_connector(connector_id)
if not provider:
raise HTTPException(400, f"No OAuth provider for '{connector_id}'")
creds = get_client_credentials(provider)
if not creds:
raise HTTPException(
400,
f"No client credentials configured for {provider.display_name}. "
f"Set up at: {provider.setup_url}",
)
client_id, _ = creds
# Build callback URL pointing to our own server
base_url = str(request.base_url).rstrip("/")
callback_url = f"{base_url}/v1/connectors/{connector_id}/oauth/callback"
params = {
"client_id": client_id,
"redirect_uri": callback_url,
"response_type": "code",
"scope": " ".join(provider.scopes),
**provider.extra_auth_params,
}
auth_url = f"{provider.auth_endpoint}?{urlencode(params)}"
from fastapi.responses import RedirectResponse
return RedirectResponse(url=auth_url)
@router.get("/{connector_id}/oauth/callback")
async def oauth_callback(
connector_id: str,
code: str = "",
error: str = "",
request: Request = None,
):
"""Handle OAuth callback from the provider."""
from fastapi.responses import HTMLResponse
from openjarvis.connectors.oauth import (
_CONNECTORS_DIR,
_exchange_token,
get_client_credentials,
get_provider_for_connector,
save_tokens,
)
_ensure_connectors_registered()
if error:
_style = "font-family:system-ui;text-align:center;padding:60px"
return HTMLResponse(
content=(
f"<html><body style='{_style}'>"
f"<h2 style='color:#ef4444'>Authorization Failed</h2>"
f"<p>{error}</p>"
"<script>setTimeout(()=>window.close(),3000)</script>"
"</body></html>"
),
status_code=400,
)
if not code:
raise HTTPException(400, "Missing authorization code")
provider = get_provider_for_connector(connector_id)
if not provider:
raise HTTPException(400, f"No OAuth provider for '{connector_id}'")
creds = get_client_credentials(provider)
if not creds:
raise HTTPException(400, "No client credentials configured")
client_id, client_secret = creds
base_url = str(request.base_url).rstrip("/")
redirect_uri = f"{base_url}/v1/connectors/{connector_id}/oauth/callback"
try:
tokens = _exchange_token(
provider, code, client_id, client_secret, redirect_uri
)
except Exception as exc:
_style = "font-family:system-ui;text-align:center;padding:60px"
return HTMLResponse(
content=(
f"<html><body style='{_style}'>"
f"<h2 style='color:#ef4444'>Token Exchange Failed</h2>"
f"<p>{exc}</p>"
"</body></html>"
),
status_code=500,
)
payload = {
"access_token": tokens.get("access_token", ""),
"refresh_token": tokens.get("refresh_token", ""),
"token_type": tokens.get("token_type", "Bearer"),
"expires_in": tokens.get("expires_in", 3600),
"client_id": client_id,
"client_secret": client_secret,
}
for filename in provider.credential_files:
save_tokens(str(_CONNECTORS_DIR / filename), payload)
# Clear cached instance so it picks up new credentials
_instances.pop(connector_id, None)
_style = "font-family:system-ui;text-align:center;padding:60px"
return HTMLResponse(
content=(
f"<html><body style='{_style}'>"
"<h2 style='color:#22c55e'>Connected!</h2>"
"<p>You can close this tab and return to OpenJarvis.</p>"
"<script>setTimeout(()=>window.close(),2000)</script>"
"</body></html>"
)
)
# Track background sync state per connector
_sync_threads: Dict[str, Any] = {}
_sync_state: Dict[str, Dict[str, Any]] = {} # {connector_id: {state, error}}
src/openjarvis/server/digest_routes.py
+124
View File
@@ -0,0 +1,124 @@
"""FastAPI routes for the morning digest."""
from __future__ import annotations
from typing import Optional
from fastapi import APIRouter, HTTPException
from fastapi.responses import FileResponse
from pydantic import BaseModel
from openjarvis.agents.digest_store import DigestStore
from openjarvis.cli.digest_cmd import (
_cancel_scheduler_tasks,
_create_scheduler_task,
_save_digest_schedule,
)
from openjarvis.core.config import load_config
class ScheduleUpdate(BaseModel):
"""Request body for updating the digest schedule."""
enabled: bool
cron: Optional[str] = None
def create_digest_router(*, db_path: str = "") -> APIRouter:
"""Create a digest API router with the given store path."""
router = APIRouter(prefix="/api/digest", tags=["digest"])
store = DigestStore(db_path=db_path) if db_path else DigestStore()
@router.get("")
async def get_digest():
"""Return the latest digest artifact."""
artifact = store.get_today()
if artifact is None:
raise HTTPException(status_code=404, detail="No digest for today")
return {
"text": artifact.text,
"sections": artifact.sections,
"sources_used": artifact.sources_used,
"generated_at": artifact.generated_at.isoformat(),
"model_used": artifact.model_used,
"voice_used": artifact.voice_used,
"audio_available": (
artifact.audio_path.exists() if artifact.audio_path.name else False
),
}
@router.get("/audio")
async def get_digest_audio():
"""Stream the digest audio file."""
artifact = store.get_today()
if artifact is None:
raise HTTPException(status_code=404, detail="No digest for today")
if not artifact.audio_path.exists():
raise HTTPException(status_code=404, detail="Audio not available")
return FileResponse(
str(artifact.audio_path),
media_type="audio/mpeg",
filename="digest.mp3",
)
@router.post("/generate")
async def generate_digest():
"""Force re-generation of the digest."""
try:
from openjarvis.sdk import Jarvis
with Jarvis() as j:
result = j.ask("Generate my morning digest", agent="morning_digest")
return {"status": "ok", "text": result}
except Exception as exc:
raise HTTPException(status_code=500, detail=str(exc))
@router.get("/history")
async def get_digest_history():
"""Return past digests."""
history = store.history(limit=10)
return [
{
"text": a.text[:200],
"generated_at": a.generated_at.isoformat(),
"model_used": a.model_used,
"voice_used": a.voice_used,
}
for a in history
]
@router.get("/schedule")
async def get_schedule():
"""Return the current digest schedule configuration."""
cfg = load_config()
return {
"enabled": cfg.digest.enabled,
"cron": cfg.digest.schedule,
}
@router.post("/schedule")
async def update_schedule(body: ScheduleUpdate):
"""Update the digest schedule configuration."""
cfg = load_config()
cron = body.cron if body.cron is not None else cfg.digest.schedule
try:
_save_digest_schedule(enabled=body.enabled, cron=cron)
except Exception as exc:
raise HTTPException(
status_code=500,
detail=f"Failed to save config: {exc}",
)
# Sync with the TaskScheduler
if body.enabled:
_create_scheduler_task(cron)
else:
_cancel_scheduler_tasks()
return {
"enabled": body.enabled,
"cron": cron,
}
return router
src/openjarvis/server/models.py
+5
View File
@@ -41,10 +41,15 @@ class UsageInfo(BaseModel):
total_tokens: int = 0
class AudioMeta(BaseModel):
url: str
class ChoiceMessage(BaseModel):
role: str = "assistant"
content: Optional[str] = ""
tool_calls: Optional[List[Dict[str, Any]]] = None
audio: Optional[AudioMeta] = None
class Choice(BaseModel):
src/openjarvis/server/routes.py
+16 -1
View File
@@ -263,11 +263,26 @@ def _handle_agent(
total_tokens=result.metadata.get("total_tokens", 0),
)
# Include audio metadata if the agent produced audio (e.g. morning digest)
audio_meta = None
audio_path = result.metadata.get("audio_path", "")
if audio_path:
from pathlib import Path
from openjarvis.server.models import AudioMeta
if Path(audio_path).exists():
audio_meta = AudioMeta(url="/api/digest/audio")
return ChatCompletionResponse(
model=model,
choices=[
Choice(
message=ChoiceMessage(role="assistant", content=result.content),
message=ChoiceMessage(
role="assistant",
content=result.content,
audio=audio_meta,
),
finish_reason="stop",
)
],
src/openjarvis/server/upload_router.py
+1 -4
View File
@@ -171,10 +171,7 @@ async def ingest_files(
allowed = ", ".join(sorted(_ALLOWED_EXTENSIONS))
raise HTTPException(
status_code=400,
detail=(
f"Unsupported file type: {ext}. "
f"Allowed: {allowed}"
),
detail=(f"Unsupported file type: {ext}. Allowed: {allowed}"),
)
data = await upload.read()
src/openjarvis/server/webhook_routes.py
+30 -31
View File
@@ -129,7 +129,9 @@ def create_webhook_router(
# Use bridge or app.state.channel_bridge
active_bridge = bridge or getattr(
request.app.state, "channel_bridge", None,
request.app.state,
"channel_bridge",
None,
)
def _handle_twilio() -> None:
@@ -140,7 +142,9 @@ def create_webhook_router(
# Get creds from app state bindings
mgr = getattr(
request.app.state, "agent_manager", None,
request.app.state,
"agent_manager",
None,
)
twilio_client = None
twilio_from = ""
@@ -153,20 +157,19 @@ def create_webhook_router(
sid = cfg.get("account_sid", "")
tok = cfg.get("auth_token", "")
twilio_from = cfg.get(
"phone_number", "",
"phone_number",
"",
)
if sid and tok:
twilio_client = Client(
sid, tok,
sid,
tok,
)
break
if twilio_client and twilio_from:
twilio_client.messages.create(
body=(
"Message received! "
"Working on it now..."
),
body=("Message received! Working on it now..."),
from_=twilio_from,
to=from_number,
)
@@ -177,7 +180,9 @@ def create_webhook_router(
response = ""
if active_bridge:
response = active_bridge.handle_incoming(
from_number, body, "twilio",
from_number,
body,
"twilio",
max_length=1600,
)
else:
@@ -191,16 +196,21 @@ def create_webhook_router(
)
engine = getattr(
request.app.state, "engine", None,
request.app.state,
"engine",
None,
)
if engine:
tools = _build_deep_research_tools(
engine=engine, model="",
engine=engine,
model="",
)
agent = DeepResearchAgent(
engine=engine,
model=getattr(
engine, "_model", "",
engine,
"_model",
"",
),
tools=tools,
max_turns=5,
@@ -224,7 +234,8 @@ def create_webhook_router(
)
except Exception as _e:
logger.warning(
"Twilio reply failed: %s", _e,
"Twilio reply failed: %s",
_e,
)
task = asyncio.create_task(
@@ -336,9 +347,7 @@ def create_webhook_router(
payload = await request.json()
# Get the SendBlue channel — may be passed at init or set later
sb = sendblue_channel or getattr(
request.app.state, "sendblue_channel", None
)
sb = sendblue_channel or getattr(request.app.state, "sendblue_channel", None)
# Verify webhook secret if configured
if sb and sb.webhook_secret:
@@ -364,18 +373,14 @@ def create_webhook_router(
# Capture sb for the closure
reply_channel = sb
# Also check for a dynamically-created bridge on app.state
active_bridge = bridge or getattr(
request.app.state, "channel_bridge", None
)
active_bridge = bridge or getattr(request.app.state, "channel_bridge", None)
if not active_bridge:
logger.warning("No channel bridge — cannot process SendBlue msg")
return Response("OK", status_code=200)
# Message queue tracking (per-sender)
_sendblue_queues = getattr(
request.app.state, "_sendblue_queues", None
)
_sendblue_queues = getattr(request.app.state, "_sendblue_queues", None)
if _sendblue_queues is None:
import threading as _th
@@ -390,9 +395,7 @@ def create_webhook_router(
# Track queue depth for this sender
with lock:
q = _sendblue_queues.setdefault(
from_number, {"pending": 0}
)
q = _sendblue_queues.setdefault(from_number, {"pending": 0})
q["pending"] += 1
position = q["pending"]
@@ -421,9 +424,7 @@ def create_webhook_router(
"Still working! Will reply ASAP",
)
reminder = threading.Thread(
target=_send_reminders, daemon=True
)
reminder = threading.Thread(target=_send_reminders, daemon=True)
reminder.start()
try:
@@ -437,9 +438,7 @@ def create_webhook_router(
# Format response for clean text message display
if response and reply_channel:
reply_channel.send(
from_number, _format_for_sms(response)
)
reply_channel.send(from_number, _format_for_sms(response))
task = asyncio.create_task(asyncio.to_thread(_handle_and_reply))
task.add_done_callback(_log_task_exception)
src/openjarvis/speech/__init__.py
+9 -2
View File
@@ -1,10 +1,17 @@
"""Speech subsystem — speech-to-text backends."""
"""Speech subsystem — speech-to-text and text-to-speech backends."""
import importlib
# Optional backends — each registers itself via @SpeechRegistry.register()
# Optional STT backends — each registers itself via @SpeechRegistry.register()
for _mod in ("faster_whisper", "openai_whisper", "deepgram"):
try:
importlib.import_module(f".{_mod}", __name__)
except ImportError:
pass
# Optional TTS backends — each registers itself via @TTSRegistry.register()
for _mod in ("cartesia_tts", "kokoro_tts", "openai_tts"):
try:
importlib.import_module(f".{_mod}", __name__)
except ImportError:
pass
src/openjarvis/speech/cartesia_tts.py
+109
View File
@@ -0,0 +1,109 @@
"""Cartesia text-to-speech backend.
Uses the Cartesia REST API for high-quality, low-latency voice synthesis.
Requires CARTESIA_API_KEY environment variable or config.
"""
from __future__ import annotations
import os
from typing import List
import httpx
from openjarvis.core.registry import TTSRegistry
from openjarvis.speech.tts import TTSBackend, TTSResult
_CARTESIA_API_BASE = "https://api.cartesia.ai"
def _cartesia_synthesize(
api_key: str,
text: str,
voice_id: str,
model: str = "sonic",
output_format: str = "mp3",
speed: float = 1.0,
) -> bytes:
"""Call the Cartesia TTS API and return raw audio bytes."""
resp = httpx.post(
f"{_CARTESIA_API_BASE}/tts/bytes",
headers={
"X-API-Key": api_key,
"Cartesia-Version": "2024-06-10",
},
json={
"model_id": model,
"transcript": text,
"voice": {"mode": "id", "id": voice_id},
"output_format": {
"container": output_format,
"sample_rate": 24000,
"encoding": "mp3" if output_format == "mp3" else "pcm_f32le",
},
"language": "en",
**({"speed": speed} if speed != 1.0 else {}),
},
timeout=120.0,
)
resp.raise_for_status()
return resp.content
@TTSRegistry.register("cartesia")
class CartesiaTTSBackend(TTSBackend):
"""Cartesia TTS backend — fast, high-quality synthesis."""
backend_id = "cartesia"
def __init__(self, *, api_key: str = "", model: str = "sonic") -> None:
self._api_key = api_key or os.environ.get("CARTESIA_API_KEY", "")
self._model = model
def synthesize(
self,
text: str,
*,
voice_id: str = "",
speed: float = 1.0,
output_format: str = "mp3",
) -> TTSResult:
if not self._api_key:
raise RuntimeError("CARTESIA_API_KEY not set")
# Default to "British Butler" voice — warm, authoritative, Jarvis-like
if not voice_id:
voice_id = "a0e99841-438c-4a64-b679-ae501e7d6091"
audio = _cartesia_synthesize(
self._api_key,
text,
voice_id=voice_id,
model=self._model,
output_format=output_format,
speed=speed,
)
return TTSResult(
audio=audio,
format=output_format,
voice_id=voice_id,
metadata={"backend": "cartesia", "model": self._model},
)
def available_voices(self) -> List[str]:
if not self._api_key:
return []
resp = httpx.get(
f"{_CARTESIA_API_BASE}/voices",
headers={
"X-API-Key": self._api_key,
"Cartesia-Version": "2024-06-10",
},
timeout=30.0,
)
resp.raise_for_status()
return [v["id"] for v in resp.json()]
def health(self) -> bool:
return bool(self._api_key)
src/openjarvis/speech/kokoro_tts.py
+80
View File
@@ -0,0 +1,80 @@
"""Kokoro TTS backend — fully open-source, runs locally.
Requires the kokoro package: pip install kokoro
Falls back gracefully if not installed.
"""
from __future__ import annotations
import io
from typing import List
from openjarvis.core.registry import TTSRegistry
from openjarvis.speech.tts import TTSBackend, TTSResult
@TTSRegistry.register("kokoro")
class KokoroTTSBackend(TTSBackend):
"""Kokoro TTS — local open-source voice synthesis."""
backend_id = "kokoro"
def __init__(self, *, model_path: str = "", device: str = "auto") -> None:
self._model_path = model_path
self._device = device
self._pipeline = None
def _ensure_pipeline(self) -> None:
if self._pipeline is not None:
return
try:
from kokoro import KPipeline
self._pipeline = KPipeline(lang_code="a")
except ImportError:
raise RuntimeError(
"kokoro package not installed. Install with: pip install kokoro"
)
def synthesize(
self,
text: str,
*,
voice_id: str = "af_heart",
speed: float = 1.0,
output_format: str = "wav",
) -> TTSResult:
self._ensure_pipeline()
import numpy as np
import soundfile as sf
samples = []
for _, _, audio in self._pipeline(text, voice=voice_id, speed=speed):
samples.append(audio)
if not samples:
return TTSResult(audio=b"", format=output_format, voice_id=voice_id)
combined = np.concatenate(samples)
buf = io.BytesIO()
sf.write(buf, combined, 24000, format=output_format.upper())
buf.seek(0)
return TTSResult(
audio=buf.read(),
format=output_format,
voice_id=voice_id,
sample_rate=24000,
duration_seconds=len(combined) / 24000,
metadata={"backend": "kokoro"},
)
def available_voices(self) -> List[str]:
return ["af_heart", "af_bella", "am_adam", "am_michael"]
def health(self) -> bool:
try:
self._ensure_pipeline()
return True
except RuntimeError:
return False
src/openjarvis/speech/openai_tts.py
+82
View File
@@ -0,0 +1,82 @@
"""OpenAI TTS backend — cloud-based voice synthesis via OpenAI API."""
from __future__ import annotations
import os
from typing import List
import httpx
from openjarvis.core.registry import TTSRegistry
from openjarvis.speech.tts import TTSBackend, TTSResult
_OPENAI_TTS_URL = "https://api.openai.com/v1/audio/speech"
def _openai_tts_request(
api_key: str,
text: str,
voice: str,
model: str = "tts-1",
speed: float = 1.0,
response_format: str = "mp3",
) -> bytes:
"""Call the OpenAI TTS API and return raw audio bytes."""
resp = httpx.post(
_OPENAI_TTS_URL,
headers={"Authorization": f"Bearer {api_key}"},
json={
"model": model,
"input": text,
"voice": voice,
"speed": speed,
"response_format": response_format,
},
timeout=120.0,
)
resp.raise_for_status()
return resp.content
@TTSRegistry.register("openai_tts")
class OpenAITTSBackend(TTSBackend):
"""OpenAI TTS backend — cloud synthesis."""
backend_id = "openai_tts"
def __init__(self, *, api_key: str = "", model: str = "tts-1") -> None:
self._api_key = api_key or os.environ.get("OPENAI_API_KEY", "")
self._model = model
def synthesize(
self,
text: str,
*,
voice_id: str = "nova",
speed: float = 1.0,
output_format: str = "mp3",
) -> TTSResult:
if not self._api_key:
raise RuntimeError("OPENAI_API_KEY not set")
audio = _openai_tts_request(
self._api_key,
text,
voice=voice_id,
model=self._model,
speed=speed,
response_format=output_format,
)
return TTSResult(
audio=audio,
format=output_format,
voice_id=voice_id,
metadata={"backend": "openai_tts", "model": self._model},
)
def available_voices(self) -> List[str]:
return ["alloy", "echo", "fable", "onyx", "nova", "shimmer"]
def health(self) -> bool:
return bool(self._api_key)
src/openjarvis/speech/tts.py
+53
View File
@@ -0,0 +1,53 @@
"""Abstract base classes and data types for text-to-speech backends."""
from __future__ import annotations
from abc import ABC, abstractmethod
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, Dict, List
@dataclass
class TTSResult:
"""Result of a text-to-speech synthesis."""
audio: bytes
format: str = "mp3"
duration_seconds: float = 0.0
voice_id: str = ""
sample_rate: int = 24000
metadata: Dict[str, Any] = field(default_factory=dict)
def save(self, path: Path) -> Path:
"""Write audio bytes to a file and return the path."""
path.write_bytes(self.audio)
return path
class TTSBackend(ABC):
"""Abstract base class for text-to-speech backends."""
backend_id: str = ""
@abstractmethod
def synthesize(
self,
text: str,
*,
voice_id: str = "",
speed: float = 1.0,
output_format: str = "mp3",
) -> TTSResult:
"""Synthesize text to audio."""
@abstractmethod
def available_voices(self) -> List[str]:
"""Return list of available voice IDs."""
@abstractmethod
def health(self) -> bool:
"""Check if the backend is ready."""
__all__ = ["TTSBackend", "TTSResult"]
src/openjarvis/system.py
+58 -1
View File
@@ -97,6 +97,11 @@ class JarvisSystem:
# Agent mode
use_agent = agent or self.agent_name
# Auto-detect agent intent if no explicit agent was requested
if not agent and use_agent != "none":
detected = self._detect_agent_intent(query)
if detected:
use_agent = detected
if use_agent and use_agent != "none":
return self._run_agent(
query,
@@ -124,6 +129,27 @@ class JarvisSystem:
"engine": self.engine_key,
}
def _detect_agent_intent(self, query: str) -> Optional[str]:
"""Detect if a query should be routed to a specific agent.
Uses lightweight pattern matching for known intent triggers.
Returns the agent name or None to use the default.
"""
import re
from openjarvis.core.registry import AgentRegistry
# Morning digest triggers
if re.search(
r"\b(good\s+morning|morning\s+digest|daily\s+briefing|morning\s+briefing)\b",
query,
re.IGNORECASE,
):
if AgentRegistry.contains("morning_digest"):
return "morning_digest"
return None
def _run_agent(
self,
query,
@@ -185,6 +211,34 @@ class JarvisSystem:
agent_kwargs["session_store"] = self.session_store
agent_kwargs["memory_backend"] = self.memory_backend
# Inject DigestConfig when instantiating the morning_digest agent
if agent_name == "morning_digest" and hasattr(self.config, "digest"):
dc = self.config.digest
section_sources = {}
for s in dc.sections:
sc = getattr(dc, s, None)
if sc and hasattr(sc, "sources"):
section_sources[s] = sc.sources
agent_kwargs.update(
{
"persona": dc.persona,
"sections": dc.sections,
"section_sources": section_sources,
"timezone": dc.timezone,
"voice_id": dc.voice_id,
"voice_speed": dc.voice_speed,
"tts_backend": dc.tts_backend,
"honorific": dc.honorific,
}
)
# Ensure digest agent always has its required tools
from openjarvis.tools.digest_collect import DigestCollectTool
from openjarvis.tools.text_to_speech import TextToSpeechTool
digest_tools = [DigestCollectTool(), TextToSpeechTool()]
existing = agent_kwargs.get("tools", [])
agent_kwargs["tools"] = digest_tools + list(existing)
try:
ag = agent_cls(self.engine, self.model, **agent_kwargs)
except TypeError:
@@ -210,7 +264,9 @@ class JarvisSystem:
from openjarvis.traces.collector import TraceCollector
collector = TraceCollector(
ag, store=self.trace_store, bus=self.bus,
ag,
store=self.trace_store,
bus=self.bus,
)
result = collector.run(query, context=ctx)
self.trace_collector = collector
@@ -278,6 +334,7 @@ class JarvisSystem:
for tr in getattr(result, "tool_results", [])
],
"turns": getattr(result, "turns", 1),
"metadata": getattr(result, "metadata", {}),
"model": self.model,
"engine": self.engine_key,
"_telemetry": _telemetry,
src/openjarvis/tools/__init__.py
+10
View File
@@ -131,4 +131,14 @@ try:
except ImportError:
pass
try:
import openjarvis.tools.text_to_speech # noqa: F401
except ImportError:
pass
try:
import openjarvis.tools.digest_collect # noqa: F401
except ImportError:
pass
__all__ = ["BaseTool", "ToolExecutor", "ToolSpec"]
src/openjarvis/tools/_stubs.py
+1 -4
View File
@@ -133,10 +133,7 @@ class ToolExecutor:
)
# Boundary guard: scan external tool arguments
if (
self._boundary_guard is not None
and not getattr(tool, "is_local", True)
):
if self._boundary_guard is not None and not getattr(tool, "is_local", True):
try:
tool_call = self._boundary_guard.check_outbound(tool_call)
# Re-parse arguments after potential redaction
src/openjarvis/tools/digest_collect.py
+519
View File
@@ -0,0 +1,519 @@
"""Digest collection tool — fetches recent data from configured connectors."""
from __future__ import annotations
import json
import re
from datetime import datetime, timedelta
from typing import Any, Dict, List
from openjarvis.connectors._stubs import Document
from openjarvis.core.registry import ConnectorRegistry, ToolRegistry
from openjarvis.core.types import ToolResult
from openjarvis.tools._stubs import BaseTool, ToolSpec
# ---------------------------------------------------------------------------
# Section definitions: ordered list of (section_name, connector_ids)
# ---------------------------------------------------------------------------
_SECTION_ORDER: List[tuple] = [
("HEALTH", {"oura", "apple_health", "strava"}),
(
"MESSAGES",
{
"gmail",
"gmail_imap",
"google_tasks",
"slack",
"imessage",
"whatsapp",
"outlook",
"notion",
"github_notifications",
},
),
("CALENDAR", {"gcalendar"}),
("WORLD", {"weather", "hackernews", "news_rss"}),
("MUSIC", {"spotify", "apple_music"}),
]
_CONNECTOR_TO_SECTION: Dict[str, str] = {}
for _section_name, _ids in _SECTION_ORDER:
for _cid in _ids:
_CONNECTOR_TO_SECTION[_cid] = _section_name
# ---------------------------------------------------------------------------
# Per-connector human-readable formatters
# ---------------------------------------------------------------------------
def _format_duration(seconds: float) -> str:
"""Format seconds as 'Xh Ym'."""
hours = int(seconds // 3600)
minutes = int((seconds % 3600) // 60)
parts = []
if hours:
parts.append(f"{hours}h")
if minutes or not parts:
parts.append(f"{minutes}m")
return " ".join(parts)
def _time_ago(ts: datetime) -> str:
"""Return a human-readable relative time like '2h ago' or '15m ago'."""
now = datetime.now()
ts_naive = ts.replace(tzinfo=None) if ts.tzinfo else ts
delta = now - ts_naive
total_seconds = max(0, int(delta.total_seconds()))
if total_seconds < 60:
return "just now"
if total_seconds < 3600:
return f"{total_seconds // 60}m ago"
if total_seconds < 86400:
return f"{total_seconds // 3600}h ago"
days = total_seconds // 86400
return f"{days}d ago"
def _format_date(ts: datetime) -> str:
"""Format a datetime as 'April 1' style."""
return ts.strftime("%B %-d") if hasattr(ts, "strftime") else str(ts)
def _format_time(ts: datetime) -> str:
"""Format a datetime as '10:30 AM' style."""
return ts.strftime("%-I:%M %p") if hasattr(ts, "strftime") else str(ts)
def _parse_content_json(doc: Document) -> Dict[str, Any]:
"""Try to parse the document content as JSON; return {} on failure."""
try:
return json.loads(doc.content)
except (json.JSONDecodeError, TypeError):
return {}
def _format_oura(doc: Document) -> str:
"""Format an Oura Ring document into a readable line."""
data = _parse_content_json(doc)
data_type = doc.metadata.get("data_type", "")
day = doc.metadata.get("day", "")
day_str = day or _format_date(doc.timestamp)
if data_type == "sleep":
hr = data.get("average_heart_rate", "?")
hrv = data.get("average_hrv", data.get("average_heart_rate_variability", "?"))
total = data.get("total_sleep_duration")
awake = data.get("awake_time")
score = data.get("score")
parts = []
if score is not None:
parts.append(f"score {score}")
parts.append(f"avg HR {hr} bpm")
if hrv != "?":
parts.append(f"HRV {hrv}")
if total is not None:
parts.append(f"{_format_duration(total)} total sleep")
if awake is not None:
parts.append(f"awake {_format_duration(awake)}")
return f"[oura] Sleep — {day_str}: {', '.join(parts)}"
if data_type == "daily_readiness":
score = data.get("score", "?")
temp = data.get(
"temperature_deviation",
data.get("temperature_trend_deviation"),
)
line = f"[oura] Readiness — {day_str}: score {score}"
if temp is not None:
sign = "+" if temp >= 0 else ""
line += f", temperature deviation {sign}{temp}"
return line
if data_type == "daily_activity":
steps = data.get("steps", "?")
cal = data.get("total_calories", data.get("active_calories", "?"))
score = data.get("score")
parts = []
if score is not None:
parts.append(f"score {score}")
parts.append(f"steps {steps}")
parts.append(f"calories {cal}")
return f"[oura] Activity — {day_str}: {', '.join(parts)}"
# Fallback for unknown Oura doc types
return f"[oura] {doc.title}"
def _format_apple_health(doc: Document) -> str:
"""Format an Apple Health document."""
return f"[apple_health] {doc.title}"
def _format_strava(doc: Document) -> str:
"""Format a Strava activity document."""
return f"[strava] {doc.title}"
def _format_gmail(doc: Document) -> str:
"""Format a Gmail email document."""
sender = doc.author or "Unknown"
subject = doc.title or "(no subject)"
ago = _time_ago(doc.timestamp)
# Include body preview (first 150 chars, single line)
body = doc.content.replace("\n", " ").strip()[:150] if doc.content else ""
line = f'[gmail] From: {sender} — "{subject}" ({ago})'
if body:
line += f"\n Preview: {body}"
return line
def _format_gmail_imap(doc: Document) -> str:
"""Format a Gmail IMAP email document."""
sender = doc.author or "Unknown"
subject = doc.title or "(no subject)"
ago = _time_ago(doc.timestamp)
return f'[gmail] From: {sender} — "{subject}" ({ago})'
def _format_google_tasks(doc: Document) -> str:
"""Format a Google Tasks document."""
title = doc.title or "Untitled Task"
status = doc.metadata.get("status", "")
due = doc.metadata.get("due", "")
parts = [f"[google_tasks] {title}"]
extras = []
if due:
extras.append(f"due {due}")
if status == "completed":
extras.append("completed")
if extras:
parts.append(f"({', '.join(extras)})")
return " ".join(parts)
def _format_slack(doc: Document) -> str:
"""Format a Slack message document."""
author = doc.author or "Unknown"
channel = doc.metadata.get("channel", "")
ago = _time_ago(doc.timestamp)
snippet = doc.content[:150].replace("\n", " ").strip()
content_preview = snippet if doc.content else ""
prefix = f"[slack] #{channel}" if channel else "[slack]"
line = f"{prefix} {author} ({ago})"
if content_preview:
line += f": {content_preview}"
return line
def _format_imessage(doc: Document) -> str:
"""Format an iMessage document."""
sender = doc.author or "Unknown"
ago = _time_ago(doc.timestamp)
snippet = doc.content[:150].replace("\n", " ").strip()
content_preview = snippet if doc.content else ""
line = f"[imessage] {sender} ({ago})"
if content_preview:
line += f": {content_preview}"
return line
def _format_whatsapp(doc: Document) -> str:
"""Format a WhatsApp message document."""
sender = doc.author or "Unknown"
content_preview = doc.content[:80].replace("\n", " ").strip() if doc.content else ""
return f"[whatsapp] {sender}: {content_preview}"
def _format_outlook(doc: Document) -> str:
"""Format an Outlook email document."""
sender = doc.author or "Unknown"
subject = doc.title or "(no subject)"
ago = _time_ago(doc.timestamp)
return f'[outlook] From: {sender} — "{subject}" ({ago})'
def _format_notion(doc: Document) -> str:
"""Format a Notion page document."""
title = doc.title or "Untitled"
ago = _time_ago(doc.timestamp)
return f"[notion] {title} (updated {ago})"
def _format_gcalendar(doc: Document) -> str:
"""Format a Google Calendar event document."""
title = doc.title or "(No title)"
time_str = _format_time(doc.timestamp)
# Try to extract duration from content
duration_match = (
re.search(r"When:\s*(.+?)$", doc.content, re.MULTILINE) if doc.content else None
)
time_range = ""
if duration_match:
when = duration_match.group(1).strip()
# Extract just the times from the ISO strings
parts = when.split(" – ")
if len(parts) == 2:
try:
start_dt = datetime.fromisoformat(parts[0].replace("Z", "+00:00"))
end_dt = datetime.fromisoformat(parts[1].replace("Z", "+00:00"))
diff = end_dt - start_dt
mins = int(diff.total_seconds() / 60)
if mins >= 60:
hrs = mins // 60
remaining = mins % 60
duration = f"{hrs} hour" + ("s" if hrs > 1 else "")
if remaining:
duration += f" {remaining} min"
else:
duration = f"{mins} min"
time_range = f" ({duration})"
except (ValueError, TypeError):
pass
return f"[gcalendar] {time_str} — {title}{time_range}"
def _format_spotify(doc: Document) -> str:
"""Format a Spotify recently-played track — returns 'Track — Artist'."""
# doc.title is already "Track — Artist" from the connector
return doc.title
def _format_apple_music(doc: Document) -> str:
"""Format an Apple Music track — returns 'Track — Artist'."""
# doc.title is already "Track — Artist" from the connector
return doc.title
def _format_weather(doc: Document) -> str:
"""Format a weather document."""
data = _parse_content_json(doc)
if doc.doc_type == "current":
temp = data.get("temp_f", "?")
cond = data.get("conditions", "?")
humidity = data.get("humidity", "?")
return f"[weather] Current: {temp}°F, {cond}, humidity {humidity}%"
if doc.doc_type == "forecast":
return f"[weather] Forecast: {doc.content[:200]}"
return f"[weather] {doc.title}"
def _format_github_notifications(doc: Document) -> str:
"""Format a GitHub notification."""
reason = doc.metadata.get("reason", "")
repo = doc.metadata.get("repo", "")
title = doc.title or "(no title)"
ago = _time_ago(doc.timestamp)
reason_str = f" ({reason})" if reason else ""
repo_str = f" in {repo}" if repo else ""
return f"[github] {title}{repo_str}{reason_str} ({ago})"
def _format_hackernews(doc: Document) -> str:
"""Format a Hacker News story."""
score = doc.metadata.get("score", "?")
comments = doc.metadata.get("descendants", "?")
return f"[hackernews] {doc.title} (score {score}, {comments} comments)"
def _format_news_rss(doc: Document) -> str:
"""Format an RSS news item."""
feed_name = doc.metadata.get("feed_name", "")
prefix = f"[{feed_name}]" if feed_name else "[news]"
description = doc.content[:150].replace("\n", " ").strip() if doc.content else ""
line = f"{prefix} {doc.title}"
if description:
line += f" — {description}"
return line
# Map connector IDs to their formatting functions
_FORMATTERS: Dict[str, Any] = {
"oura": _format_oura,
"apple_health": _format_apple_health,
"strava": _format_strava,
"gmail": _format_gmail,
"gmail_imap": _format_gmail_imap,
"google_tasks": _format_google_tasks,
"slack": _format_slack,
"imessage": _format_imessage,
"whatsapp": _format_whatsapp,
"outlook": _format_outlook,
"notion": _format_notion,
"gcalendar": _format_gcalendar,
"weather": _format_weather,
"github_notifications": _format_github_notifications,
"hackernews": _format_hackernews,
"news_rss": _format_news_rss,
"spotify": _format_spotify,
"apple_music": _format_apple_music,
}
def _format_doc(source: str, doc: Document) -> str:
"""Format a document using the source-specific formatter, with fallback."""
formatter = _FORMATTERS.get(source)
if formatter:
try:
return formatter(doc)
except Exception: # noqa: BLE001
pass
# Fallback: connector name + title
return f"[{source}] {doc.title}"
def _format_music_section(
collected_docs: Dict[str, List[Document]],
music_connectors: set,
) -> List[str]:
"""Format music connectors as grouped lists instead of per-track lines."""
lines: List[str] = []
for source in sorted(music_connectors):
docs = collected_docs.get(source, [])
if not docs:
continue
tracks = []
for doc in docs:
tracks.append(doc.title)
label = "Recently played" if source == "spotify" else "Library"
lines.append(f"[{source}] {label}: {', '.join(tracks)}")
return lines
@ToolRegistry.register("digest_collect")
class DigestCollectTool(BaseTool):
"""Collect recent data from multiple connectors for digest synthesis."""
tool_id = "digest_collect"
is_local = True
@property
def spec(self) -> ToolSpec:
return ToolSpec(
name="digest_collect",
description=(
"Fetch recent data from configured connectors (email, calendar, "
"health, tasks, etc.) and return a structured, human-readable "
"summary grouped by section (Health, Messages, Calendar, Music) "
"for digest synthesis."
),
parameters={
"type": "object",
"properties": {
"sources": {
"type": "array",
"items": {"type": "string"},
"description": (
"List of connector IDs to fetch from "
"(e.g., ['gmail', 'oura', 'gcalendar'])."
),
},
"hours_back": {
"type": "number",
"description": "How many hours back to look (default: 24).",
},
},
"required": ["sources"],
},
category="data",
timeout_seconds=60.0,
)
def execute(self, **params: Any) -> ToolResult:
# Ensure connectors are registered
import openjarvis.connectors # noqa: F401
sources: List[str] = params.get("sources", [])
hours_back: float = params.get("hours_back", 24)
since = datetime.now() - timedelta(hours=hours_back)
# Collect raw documents per source
collected_docs: Dict[str, List[Document]] = {}
errors: List[str] = []
for source in sources:
if not ConnectorRegistry.contains(source):
errors.append(f"Connector '{source}' not available")
continue
try:
connector_cls = ConnectorRegistry.get(source)
connector = connector_cls()
if not connector.is_connected():
errors.append(
f"Connector '{source}' not connected (no credentials)"
)
continue
# Cap per-source to avoid overwhelming the LLM context
max_per_source = 15
docs: List[Document] = []
for d in connector.sync(since=since):
docs.append(d)
if len(docs) >= max_per_source:
break
collected_docs[source] = docs
except Exception as exc:
errors.append(f"Error fetching from '{source}': {exc}")
# Group by section and build human-readable output
summary_parts: List[str] = []
for section_name, section_connectors in _SECTION_ORDER:
# Gather all sources that belong to this section and have data
section_sources = [
s for s in sources if s in section_connectors and s in collected_docs
]
if not section_sources:
continue
section_lines: List[str] = []
if section_name == "MUSIC":
# Music gets special grouped formatting
section_lines = _format_music_section(
collected_docs, section_connectors
)
else:
for source in section_sources:
for doc in collected_docs[source]:
section_lines.append(_format_doc(source, doc))
if section_lines:
summary_parts.append(f"=== {section_name} ===")
summary_parts.extend(section_lines)
summary_parts.append("") # blank line between sections
# Handle any connectors not in a known section (fallback)
known_connectors = set()
for _, cids in _SECTION_ORDER:
known_connectors |= cids
uncategorized_sources = [
s for s in sources if s not in known_connectors and s in collected_docs
]
if uncategorized_sources:
summary_parts.append("=== OTHER ===")
for source in uncategorized_sources:
for doc in collected_docs[source]:
summary_parts.append(_format_doc(source, doc))
summary_parts.append("")
# Errors at the end, not inline
if errors:
summary_parts.append("=== ERRORS ===")
summary_parts.extend(errors)
return ToolResult(
tool_name="digest_collect",
content="\n".join(summary_parts),
success=True,
metadata={
"sources_queried": sources,
"sources_ok": list(collected_docs.keys()),
"sources_failed": errors,
"total_items": sum(len(v) for v in collected_docs.values()),
},
)
src/openjarvis/tools/text_to_speech.py
+106
View File
@@ -0,0 +1,106 @@
"""Text-to-speech tool — synthesize text to audio via configurable TTS backend."""
from __future__ import annotations
import tempfile
from pathlib import Path
from typing import Any
from openjarvis.core.registry import ToolRegistry, TTSRegistry
from openjarvis.core.types import ToolResult
from openjarvis.tools._stubs import BaseTool, ToolSpec
@ToolRegistry.register("text_to_speech")
class TextToSpeechTool(BaseTool):
"""Synthesize text into spoken audio using a TTS backend."""
tool_id = "text_to_speech"
is_local = False
@property
def spec(self) -> ToolSpec:
return ToolSpec(
name="text_to_speech",
description=(
"Convert text to spoken audio. Returns the file path to the "
"generated audio file."
),
parameters={
"type": "object",
"properties": {
"text": {
"type": "string",
"description": "The text to synthesize into speech.",
},
"voice_id": {
"type": "string",
"description": "Voice identifier for the TTS backend.",
},
"backend": {
"type": "string",
"description": "TTS backend (cartesia, kokoro, openai_tts).",
},
"output_dir": {
"type": "string",
"description": "Directory to save the audio file.",
},
},
"required": ["text"],
},
category="audio",
timeout_seconds=120.0,
)
def execute(self, **params: Any) -> ToolResult:
# Ensure TTS backends are registered
import openjarvis.speech # noqa: F401
text = params.get("text", "")
voice_id = params.get("voice_id", "")
backend_key = params.get("backend", "cartesia")
output_dir = params.get("output_dir", "")
speed = float(params.get("speed", 1.0))
if not text:
return ToolResult(
tool_name="text_to_speech",
content="No text provided.",
success=False,
)
if not TTSRegistry.contains(backend_key):
return ToolResult(
tool_name="text_to_speech",
content=f"TTS backend '{backend_key}' not available.",
success=False,
)
backend_cls = TTSRegistry.get(backend_key)
backend = backend_cls()
result = backend.synthesize(text, voice_id=voice_id, speed=speed)
# Save to file
if output_dir:
out_dir = Path(output_dir)
else:
out_dir = Path(tempfile.mkdtemp(prefix="jarvis-tts-"))
out_dir.mkdir(parents=True, exist_ok=True)
ext = result.format or "mp3"
audio_path = out_dir / f"digest.{ext}"
result.save(audio_path)
return ToolResult(
tool_name="text_to_speech",
content=str(audio_path),
success=True,
metadata={
"audio_path": str(audio_path),
"format": ext,
"duration_seconds": result.duration_seconds,
"voice_id": result.voice_id,
"backend": backend_key,
},
)
tests/agents/test_digest_store.py
+77
View File
@@ -0,0 +1,77 @@
"""Tests for DigestStore and DigestArtifact."""
from __future__ import annotations
from datetime import datetime
from pathlib import Path
from openjarvis.agents.digest_store import DigestArtifact, DigestStore
def test_store_and_retrieve(tmp_path):
store = DigestStore(db_path=str(tmp_path / "digest.db"))
artifact = DigestArtifact(
text="Good morning sir.",
audio_path=Path("/tmp/digest.mp3"),
sections={"messages": "You have 3 emails.", "calendar": "2 meetings today."},
sources_used=["gmail", "gcalendar"],
generated_at=datetime(2026, 4, 1, 6, 0, 0),
model_used="claude-sonnet-4-6",
voice_used="jarvis-v1",
)
store.save(artifact)
retrieved = store.get_latest()
assert retrieved is not None
assert retrieved.text == "Good morning sir."
assert retrieved.sections["messages"] == "You have 3 emails."
assert retrieved.sources_used == ["gmail", "gcalendar"]
assert retrieved.voice_used == "jarvis-v1"
store.close()
def test_get_today(tmp_path):
store = DigestStore(db_path=str(tmp_path / "digest.db"))
artifact = DigestArtifact(
text="Today's digest",
audio_path=Path("/tmp/today.mp3"),
sections={"messages": "Nothing urgent."},
sources_used=["gmail"],
generated_at=datetime.now(),
model_used="test-model",
voice_used="jarvis",
)
store.save(artifact)
today = store.get_today()
assert today is not None
assert today.text == "Today's digest"
store.close()
def test_get_today_returns_none_when_empty(tmp_path):
store = DigestStore(db_path=str(tmp_path / "digest.db"))
assert store.get_today() is None
store.close()
def test_history(tmp_path):
store = DigestStore(db_path=str(tmp_path / "digest.db"))
for i in range(3):
store.save(
DigestArtifact(
text=f"Digest {i}",
audio_path=Path(f"/tmp/d{i}.mp3"),
sections={},
sources_used=[],
generated_at=datetime(2026, 4, 1 + i, 6, 0, 0),
model_used="test",
voice_used="jarvis",
)
)
history = store.history(limit=2)
assert len(history) == 2
assert history[0].text == "Digest 2" # Most recent first
store.close()
tests/agents/test_morning_digest.py
+71
View File
@@ -0,0 +1,71 @@
"""Tests for MorningDigestAgent."""
from __future__ import annotations
from unittest.mock import MagicMock, patch
from openjarvis.agents._stubs import AgentResult
from openjarvis.core.registry import AgentRegistry
from openjarvis.core.types import ToolResult
def test_morning_digest_registered():
from openjarvis.agents.morning_digest import MorningDigestAgent
AgentRegistry.register_value("morning_digest", MorningDigestAgent)
assert AgentRegistry.contains("morning_digest")
def test_morning_digest_run(tmp_path):
from openjarvis.agents.morning_digest import MorningDigestAgent
mock_engine = MagicMock()
mock_engine.generate.return_value = {
"content": "Good morning sir. You have 3 emails and 2 meetings today.",
"finish_reason": "stop",
"usage": {},
}
# Mock collect result
mock_collect_result = ToolResult(
tool_name="digest_collect",
content='=== MESSAGES ===\n[gmail] From: alice@co.com — "Budget" (1h ago)\n',
success=True,
metadata={"total_items": 2},
)
# Mock TTS result
mock_tts_result = ToolResult(
tool_name="text_to_speech",
content=str(tmp_path / "digest.mp3"),
success=True,
metadata={"audio_path": str(tmp_path / "digest.mp3")},
)
agent = MorningDigestAgent(
mock_engine,
"test-model",
tools=[],
persona="neutral",
digest_store_path=str(tmp_path / "digest.db"),
)
with patch.object(
agent._executor,
"execute",
side_effect=[mock_collect_result, mock_tts_result],
):
result = agent.run("Generate morning digest")
assert isinstance(result, AgentResult)
assert "Good morning" in result.content
assert result.turns == 1
assert len(result.tool_results) == 2
def test_load_persona():
from openjarvis.agents.morning_digest import _load_persona
# Nonexistent persona returns empty string
result = _load_persona("nonexistent_persona_xyz")
assert result == ""
tests/channels/test_sendblue.py
+44 -34
View File
@@ -28,6 +28,7 @@ def _register_sendblue():
# Helpers
# ---------------------------------------------------------------------------
def _make_channel(**overrides):
from openjarvis.channels.sendblue import SendBlueChannel
@@ -167,15 +168,17 @@ class TestWebhookHandler:
received = []
ch.on_message(lambda msg: received.append(msg))
ch.handle_webhook({
"from_number": "+19127130720",
"to_number": "+15551234567",
"content": "Hello Jarvis",
"message_handle": "msg-001",
"is_outbound": False,
"status": "RECEIVED",
"service": "iMessage",
})
ch.handle_webhook(
{
"from_number": "+19127130720",
"to_number": "+15551234567",
"content": "Hello Jarvis",
"message_handle": "msg-001",
"is_outbound": False,
"status": "RECEIVED",
"service": "iMessage",
}
)
assert len(received) == 1
assert received[0].sender == "+19127130720"
@@ -187,11 +190,13 @@ class TestWebhookHandler:
received = []
ch.on_message(lambda msg: received.append(msg))
ch.handle_webhook({
"from_number": "+15551234567",
"content": "Outbound message",
"is_outbound": True,
})
ch.handle_webhook(
{
"from_number": "+15551234567",
"content": "Outbound message",
"is_outbound": True,
}
)
assert len(received) == 0
@@ -200,11 +205,13 @@ class TestWebhookHandler:
received = []
ch.on_message(lambda msg: received.append(msg))
ch.handle_webhook({
"from_number": "+19127130720",
"content": "",
"is_outbound": False,
})
ch.handle_webhook(
{
"from_number": "+19127130720",
"content": "",
"is_outbound": False,
}
)
assert len(received) == 0
@@ -212,19 +219,20 @@ class TestWebhookHandler:
bus = EventBus(record_history=True)
ch = _make_channel(bus=bus)
ch.handle_webhook({
"from_number": "+19127130720",
"content": "Test",
"message_handle": "msg-002",
"is_outbound": False,
"service": "iMessage",
})
ch.handle_webhook(
{
"from_number": "+19127130720",
"content": "Test",
"message_handle": "msg-002",
"is_outbound": False,
"service": "iMessage",
}
)
event_types = [e.event_type for e in bus.history]
assert EventType.CHANNEL_MESSAGE_RECEIVED in event_types
event = [
e for e in bus.history
if e.event_type == EventType.CHANNEL_MESSAGE_RECEIVED
e for e in bus.history if e.event_type == EventType.CHANNEL_MESSAGE_RECEIVED
][0]
assert event.data["sender"] == "+19127130720"
assert event.data["service"] == "iMessage"
@@ -234,12 +242,14 @@ class TestWebhookHandler:
ch.on_message(lambda msg: 1 / 0) # Will raise ZeroDivisionError
# Should not raise
ch.handle_webhook({
"from_number": "+19127130720",
"content": "Test",
"message_handle": "msg-003",
"is_outbound": False,
})
ch.handle_webhook(
{
"from_number": "+19127130720",
"content": "Test",
"message_handle": "msg-003",
"is_outbound": False,
}
)
# ---------------------------------------------------------------------------
tests/cli/test_digest_cmd.py
+54
View File
@@ -0,0 +1,54 @@
"""Tests for `jarvis digest` CLI command."""
from __future__ import annotations
from datetime import datetime
from pathlib import Path
from click.testing import CliRunner
from openjarvis.agents.digest_store import DigestArtifact, DigestStore
def test_digest_command_exists():
"""The digest command is registered on the CLI."""
from openjarvis.cli import cli
runner = CliRunner()
result = runner.invoke(cli, ["digest", "--help"])
assert result.exit_code == 0
assert "digest" in result.output.lower()
def test_digest_displays_cached(tmp_path):
from openjarvis.cli import cli
db_path = str(tmp_path / "digest.db")
store = DigestStore(db_path=db_path)
store.save(
DigestArtifact(
text="# Messages\nYou have 3 emails.\n# Calendar\n2 meetings today.",
audio_path=Path("/nonexistent/audio.mp3"),
sections={},
sources_used=["gmail"],
generated_at=datetime.now(),
model_used="test",
voice_used="jarvis",
)
)
store.close()
runner = CliRunner()
result = runner.invoke(cli, ["digest", "--text-only", "--db-path", db_path])
assert result.exit_code == 0
assert "3 emails" in result.output
def test_digest_no_cache(tmp_path):
from openjarvis.cli import cli
db_path = str(tmp_path / "empty.db")
runner = CliRunner()
result = runner.invoke(cli, ["digest", "--db-path", db_path])
assert result.exit_code == 0
assert "No digest for today" in result.output
tests/conftest.py
+2
View File
@@ -21,6 +21,7 @@ from openjarvis.core.registry import (
RouterPolicyRegistry,
SpeechRegistry,
ToolRegistry,
TTSRegistry,
)
@@ -38,6 +39,7 @@ def _clean_registries() -> None:
SpeechRegistry.clear()
CompressionRegistry.clear()
ConnectorRegistry.clear()
TTSRegistry.clear()
reset_event_bus()
tests/connectors/test_apple_health.py
+161
View File
@@ -0,0 +1,161 @@
"""Tests for AppleHealthConnector -- local HealthKit DB / export XML."""
from __future__ import annotations
import json
from datetime import datetime
from openjarvis.connectors._stubs import Document
from openjarvis.core.registry import ConnectorRegistry
_SAMPLE_EXPORT_XML = """\
<?xml version="1.0" encoding="UTF-8"?>
<HealthData>
<Record type="HKQuantityTypeIdentifierStepCount" \
startDate="2024-03-15 08:00:00 -0700" \
endDate="2024-03-15 08:30:00 -0700" \
value="1234" sourceName="iPhone" unit="count"/>
<Record type="HKQuantityTypeIdentifierStepCount" \
startDate="2024-03-15 12:00:00 -0700" \
endDate="2024-03-15 12:15:00 -0700" \
value="500" sourceName="iPhone" unit="count"/>
<Record type="HKCategoryTypeIdentifierSleepAnalysis" \
startDate="2024-03-15 22:00:00 -0700" \
endDate="2024-03-16 06:00:00 -0700" \
value="HKCategoryValueSleepAnalysisAsleepCore" sourceName="iPhone"/>
<Record type="HKQuantityTypeIdentifierHeartRate" \
startDate="2024-03-15 09:00:00 -0700" \
value="65" unit="count/min" sourceName="iPhone"/>
<Record type="HKQuantityTypeIdentifierHeartRate" \
startDate="2024-03-15 15:00:00 -0700" \
value="75" unit="count/min" sourceName="iPhone"/>
<Record type="HKQuantityTypeIdentifierActiveEnergyBurned" \
startDate="2024-03-15 07:30:00 -0700" \
value="350" unit="kcal" sourceName="iPhone"/>
<Workout workoutActivityType="HKWorkoutActivityTypeRunning" \
duration="1800" \
startDate="2024-03-15 07:00:00 -0700" \
endDate="2024-03-15 07:30:00 -0700" \
totalDistance="5.2" totalEnergyBurned="300"/>
</HealthData>
"""
def test_apple_health_registered():
"""AppleHealthConnector is discoverable via ConnectorRegistry."""
from openjarvis.connectors.apple_health import AppleHealthConnector
ConnectorRegistry.register_value("apple_health", AppleHealthConnector)
assert ConnectorRegistry.contains("apple_health")
cls = ConnectorRegistry.get("apple_health")
assert cls.connector_id == "apple_health"
assert cls.display_name == "Apple Health"
assert cls.auth_type == "local"
def test_not_connected_no_files(tmp_path):
"""is_connected() returns False when neither data source exists."""
from openjarvis.connectors.apple_health import AppleHealthConnector
connector = AppleHealthConnector(
export_path=str(tmp_path / "nope.xml"),
healthkit_db_path=str(tmp_path / "nope.sqlite"),
)
assert connector.is_connected() is False
def test_connected_with_export(tmp_path):
"""is_connected() returns True when the export XML exists."""
from openjarvis.connectors.apple_health import AppleHealthConnector
export_file = tmp_path / "export.xml"
export_file.write_text(_SAMPLE_EXPORT_XML, encoding="utf-8")
connector = AppleHealthConnector(
export_path=str(export_file),
healthkit_db_path=str(tmp_path / "nope.sqlite"),
)
assert connector.is_connected() is True
def test_sync_export_xml(tmp_path):
"""Sync from export XML yields correct Documents for each data type."""
from openjarvis.connectors.apple_health import AppleHealthConnector
export_file = tmp_path / "export.xml"
export_file.write_text(_SAMPLE_EXPORT_XML, encoding="utf-8")
connector = AppleHealthConnector(
export_path=str(export_file),
healthkit_db_path=str(tmp_path / "nope.sqlite"),
)
docs = list(connector.sync())
assert all(isinstance(d, Document) for d in docs)
by_type = {}
for d in docs:
by_type.setdefault(d.doc_type, []).append(d)
# Steps: two records on 2024-03-15 should be summed (1234 + 500 = 1734)
assert "steps" in by_type
step_doc = by_type["steps"][0]
assert step_doc.source == "apple_health"
step_data = json.loads(step_doc.content)
assert step_data["steps"] == 1734
assert "1,734" in step_doc.title
# Sleep
assert "sleep" in by_type
sleep_doc = by_type["sleep"][0]
sleep_data = json.loads(sleep_doc.content)
assert sleep_data["total_seconds"] == 28800.0 # 8 hours
assert "8h" in sleep_doc.title
# Heart rate: avg of 65 and 75 = 70
assert "heart_rate" in by_type
hr_doc = by_type["heart_rate"][0]
hr_data = json.loads(hr_doc.content)
assert hr_data["avg_bpm"] == 70.0
assert hr_data["min_bpm"] == 65.0
assert hr_data["max_bpm"] == 75.0
# Active energy
assert "active_energy" in by_type
energy_doc = by_type["active_energy"][0]
energy_data = json.loads(energy_doc.content)
assert energy_data["kcal"] == 350.0
# Workout
assert "workout" in by_type
workout_doc = by_type["workout"][0]
assert "Running" in workout_doc.title
assert "5.2" in workout_doc.title
workout_data = json.loads(workout_doc.content)
assert workout_data["duration_seconds"] == 1800.0
def test_sync_with_since_filter(tmp_path):
"""Only records after the ``since`` date are included."""
from openjarvis.connectors.apple_health import AppleHealthConnector
export_file = tmp_path / "export.xml"
export_file.write_text(_SAMPLE_EXPORT_XML, encoding="utf-8")
connector = AppleHealthConnector(
export_path=str(export_file),
healthkit_db_path=str(tmp_path / "nope.sqlite"),
)
# All sample data is from 2024-03-15; filtering after that should yield nothing
docs = list(connector.sync(since=datetime(2025, 1, 1)))
assert len(docs) == 0
def test_disconnect_is_noop(tmp_path):
"""disconnect() does not raise for a local connector."""
from openjarvis.connectors.apple_health import AppleHealthConnector
connector = AppleHealthConnector(
export_path=str(tmp_path / "nope.xml"),
healthkit_db_path=str(tmp_path / "nope.sqlite"),
)
connector.disconnect() # should not raise
tests/connectors/test_apple_music.py
+177
View File
@@ -0,0 +1,177 @@
"""Tests for AppleMusicConnector -- local Music.app via AppleScript.
All tests mock ``subprocess.run`` so no actual Music.app interaction is needed.
"""
from __future__ import annotations
import json
from datetime import datetime
from typing import List
from unittest.mock import MagicMock, patch
import pytest
from openjarvis.connectors._stubs import Document
from openjarvis.core.registry import ConnectorRegistry
# ---------------------------------------------------------------------------
# Sample AppleScript output
# ---------------------------------------------------------------------------
_SAMPLE_OUTPUT = (
"Bohemian Rhapsody|||Queen|||A Night at the Opera|||354.0|||Rock|||42|||"
"Saturday, March 15, 2026 at 2:30:00 PM\n"
"Blinding Lights|||The Weeknd|||After Hours|||200.0|||Pop|||18|||never\n"
)
# ---------------------------------------------------------------------------
# Fixtures
# ---------------------------------------------------------------------------
@pytest.fixture()
def connector():
"""Return a fresh AppleMusicConnector."""
from openjarvis.connectors.apple_music import AppleMusicConnector
return AppleMusicConnector()
# ---------------------------------------------------------------------------
# Test 1 -- registry
# ---------------------------------------------------------------------------
def test_apple_music_registered():
"""AppleMusicConnector is discoverable via ConnectorRegistry."""
from openjarvis.connectors.apple_music import AppleMusicConnector
ConnectorRegistry.register_value("apple_music", AppleMusicConnector)
assert ConnectorRegistry.contains("apple_music")
cls = ConnectorRegistry.get("apple_music")
assert cls.connector_id == "apple_music"
assert cls.display_name == "Apple Music"
assert cls.auth_type == "local"
# ---------------------------------------------------------------------------
# Test 2 -- is_connected on macOS (happy path)
# ---------------------------------------------------------------------------
def test_is_connected_on_macos(connector):
"""is_connected() returns True on macOS when Music.app responds."""
mock_result = MagicMock()
mock_result.returncode = 0
mock_result.stdout = "Library"
with (
patch("openjarvis.connectors.apple_music.sys") as mock_sys,
patch("openjarvis.connectors.apple_music.subprocess") as mock_subprocess,
):
mock_sys.platform = "darwin"
mock_subprocess.run.return_value = mock_result
mock_subprocess.TimeoutExpired = TimeoutError
assert connector.is_connected() is True
# ---------------------------------------------------------------------------
# Test 3 -- is_connected on Linux
# ---------------------------------------------------------------------------
def test_is_connected_on_linux(connector):
"""is_connected() returns False on non-macOS platforms."""
with patch("openjarvis.connectors.apple_music.sys") as mock_sys:
mock_sys.platform = "linux"
assert connector.is_connected() is False
# ---------------------------------------------------------------------------
# Test 4 -- sync yields Documents
# ---------------------------------------------------------------------------
def test_sync_yields_tracks(connector):
"""sync() parses AppleScript output and yields correct Documents."""
mock_result = MagicMock()
mock_result.returncode = 0
mock_result.stdout = _SAMPLE_OUTPUT
with patch(
"openjarvis.connectors.apple_music._run_osascript",
return_value=_SAMPLE_OUTPUT,
):
docs: List[Document] = list(connector.sync())
assert len(docs) == 2
assert all(isinstance(d, Document) for d in docs)
assert all(d.source == "apple_music" for d in docs)
assert all(d.doc_type == "track" for d in docs)
# First track
assert docs[0].author == "Queen"
assert "Bohemian Rhapsody" in docs[0].title
content0 = json.loads(docs[0].content)
assert content0["album"] == "A Night at the Opera"
assert content0["play_count"] == 42
assert docs[0].metadata["genre"] == "Rock"
assert docs[0].metadata["duration_s"] == 354.0
# Second track
assert docs[1].author == "The Weeknd"
assert "Blinding Lights" in docs[1].title
content1 = json.loads(docs[1].content)
assert content1["genre"] == "Pop"
assert docs[1].metadata["play_count"] == 18
# ---------------------------------------------------------------------------
# Test 5 -- sync with since filter
# ---------------------------------------------------------------------------
def test_sync_filters_by_since(connector):
"""sync(since=...) skips tracks with played_date before the threshold."""
with patch(
"openjarvis.connectors.apple_music._run_osascript",
return_value=_SAMPLE_OUTPUT,
):
# The first track was played 2026-03-15; filter after that
docs = list(connector.sync(since=datetime(2026, 3, 16)))
# Only tracks with played_date > since are included.
# "Blinding Lights" has played_date "never" so since filter is not applied
# (since filter only applies when played_date is not None).
# "Bohemian Rhapsody" was played 2026-03-15 which is <= 2026-03-16, so skipped.
assert len(docs) == 1
assert "Blinding Lights" in docs[0].title
# ---------------------------------------------------------------------------
# Test 6 -- sync handles Music.app failure gracefully
# ---------------------------------------------------------------------------
def test_sync_handles_failure(connector):
"""sync() returns empty when AppleScript fails."""
with patch(
"openjarvis.connectors.apple_music._run_osascript",
return_value=None,
):
docs = list(connector.sync())
assert len(docs) == 0
assert connector.sync_status().state == "error"
# ---------------------------------------------------------------------------
# Test 7 -- disconnect is a no-op
# ---------------------------------------------------------------------------
def test_disconnect_is_noop(connector):
"""disconnect() does nothing (local connector)."""
connector.disconnect() # should not raise
tests/connectors/test_connector_health.py
+12 -4
View File
@@ -35,7 +35,9 @@ _ALL_CONNECTORS = _LOCAL_CONNECTORS + _TOKEN_CONNECTORS
ids=[c[0] for c in _ALL_CONNECTORS],
)
def test_connector_instantiates(
connector_id: str, module_path: str, class_name: str,
connector_id: str,
module_path: str,
class_name: str,
) -> None:
"""Every connector can be instantiated without errors."""
import importlib
@@ -52,7 +54,9 @@ def test_connector_instantiates(
ids=[c[0] for c in _ALL_CONNECTORS],
)
def test_connector_has_required_methods(
connector_id: str, module_path: str, class_name: str,
connector_id: str,
module_path: str,
class_name: str,
) -> None:
"""Every connector implements the BaseConnector interface."""
import importlib
@@ -75,7 +79,9 @@ def test_connector_has_required_methods(
ids=[c[0] for c in _TOKEN_CONNECTORS],
)
def test_connector_not_connected_without_creds(
connector_id: str, module_path: str, class_name: str,
connector_id: str,
module_path: str,
class_name: str,
) -> None:
"""Token connectors report not connected without credentials."""
import importlib
@@ -96,7 +102,9 @@ def test_connector_not_connected_without_creds(
ids=[c[0] for c in _ALL_CONNECTORS],
)
def test_connector_has_metadata(
connector_id: str, module_path: str, class_name: str,
connector_id: str,
module_path: str,
class_name: str,
) -> None:
"""Every connector has connector_id, display_name, and auth_type."""
import importlib
tests/connectors/test_gcalendar.py
+7 -1
View File
@@ -57,10 +57,16 @@ def _make_credentials(tmp_path: Path) -> Path:
@pytest.fixture()
def connector(tmp_path: Path):
"""GCalendarConnector pointing at a tmp credentials path (no file yet)."""
from unittest.mock import patch
from openjarvis.connectors.gcalendar import GCalendarConnector # noqa: PLC0415
creds_path = str(tmp_path / "gcalendar.json")
return GCalendarConnector(credentials_path=creds_path)
with patch(
"openjarvis.connectors.oauth._SHARED_GOOGLE_CREDENTIALS_PATH",
str(tmp_path / "google_shared.json"),
):
yield GCalendarConnector(credentials_path=creds_path)
# ---------------------------------------------------------------------------
tests/connectors/test_gdrive.py
+6 -1
View File
@@ -55,7 +55,12 @@ def connector(tmp_path: Path):
from openjarvis.connectors.gdrive import GDriveConnector # noqa: PLC0415
creds_path = str(tmp_path / "gdrive.json")
return GDriveConnector(credentials_path=creds_path)
# Prevent fallback to shared google.json on machines with real credentials
with patch(
"openjarvis.connectors.oauth._SHARED_GOOGLE_CREDENTIALS_PATH",
str(tmp_path / "google_shared.json"),
):
yield GDriveConnector(credentials_path=creds_path)
# ---------------------------------------------------------------------------
tests/connectors/test_github_notifications.py
+105
View File
@@ -0,0 +1,105 @@
"""Tests for GitHubNotificationsConnector — GitHub REST API."""
from __future__ import annotations
from datetime import datetime
from unittest.mock import patch
import pytest
from openjarvis.connectors._stubs import Document
from openjarvis.core.registry import ConnectorRegistry
def test_github_notifications_registered():
"""GitHubNotificationsConnector is discoverable via ConnectorRegistry."""
from openjarvis.connectors.github_notifications import (
GitHubNotificationsConnector,
)
ConnectorRegistry.register_value(
"github_notifications", GitHubNotificationsConnector
)
assert ConnectorRegistry.contains("github_notifications")
cls = ConnectorRegistry.get("github_notifications")
assert cls.connector_id == "github_notifications"
assert cls.auth_type == "token"
_NOTIFICATIONS_RESPONSE = [
{
"id": "1001",
"reason": "review_requested",
"updated_at": "2026-04-01T10:00:00Z",
"subject": {
"title": "Add caching layer to inference engine",
"type": "PullRequest",
"url": "https://api.github.com/repos/org/repo/pulls/42",
},
"repository": {"full_name": "org/repo"},
},
{
"id": "1002",
"reason": "mention",
"updated_at": "2026-04-01T11:30:00Z",
"subject": {
"title": "Bug: memory leak in long sessions",
"type": "Issue",
"url": "https://api.github.com/repos/org/repo/issues/99",
},
"repository": {"full_name": "org/repo"},
},
]
@pytest.fixture()
def connector(tmp_path):
"""GitHubNotificationsConnector with fake token file."""
from openjarvis.connectors.github_notifications import (
GitHubNotificationsConnector,
)
token_path = tmp_path / "github.json"
token_path.write_text('{"token": "ghp_fake123"}', encoding="utf-8")
return GitHubNotificationsConnector(token_path=str(token_path))
def test_is_connected(connector):
assert connector.is_connected() is True
def test_is_connected_no_file(tmp_path):
from openjarvis.connectors.github_notifications import (
GitHubNotificationsConnector,
)
c = GitHubNotificationsConnector(token_path=str(tmp_path / "missing.json"))
assert c.is_connected() is False
def test_sync_yields_documents(connector):
"""Sync returns Documents for each notification."""
with patch(
"openjarvis.connectors.github_notifications._github_api_get",
return_value=_NOTIFICATIONS_RESPONSE,
):
docs = list(connector.sync(since=datetime(2026, 4, 1)))
assert len(docs) == 2
assert all(isinstance(d, Document) for d in docs)
pr_doc = docs[0]
assert pr_doc.source == "github_notifications"
assert pr_doc.title == "Add caching layer to inference engine"
assert "review_requested" in pr_doc.content
assert pr_doc.metadata["repo"] == "org/repo"
assert pr_doc.metadata["type"] == "PullRequest"
issue_doc = docs[1]
assert issue_doc.title == "Bug: memory leak in long sessions"
assert issue_doc.metadata["reason"] == "mention"
def test_disconnect(connector):
connector.disconnect()
assert connector.is_connected() is False

Some files were not shown because too many files have changed in this diff Show More