Auto-detects local sources (Apple Notes, iMessage, Obsidian), ingests
them into ~/.openjarvis/knowledge.db via IngestionPipeline + SyncEngine,
and launches an interactive Deep Research chat session with Qwen3.5 via
Ollama.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements SyncScheduler that runs a daemon background thread to call
SyncEngine.sync() on all registered connectors at a configurable interval.
Provides run_once() as a synchronous helper for testing. Disconnected
connectors are skipped each cycle; errors are logged without stopping the
loop. 4 tests covering run_once, disconnected skip, start/stop, and
per-connector chunk counts.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements WhatsAppConnector that reads exported .txt chat files from a
directory, parses the standard WhatsApp export line format with regex,
and yields one Document per chat file with participants, timestamps, and
since filtering. Registers under "whatsapp" in ConnectorRegistry with
MCP tools for whatsapp_search_messages and whatsapp_get_chat. 10 tests
covering parsing, multi-file, since filtering, is_connected, and registry.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove the ARG002 noqa suppression and translate the `since` datetime
parameter into a Gmail `after:<epoch>` query string passed to
`_gmail_api_list_messages`, so incremental syncs only fetch messages
newer than the requested cutoff.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add optional `attachment_store` parameter to IngestionPipeline. When
provided, attachments are stored as blobs in AttachmentStore and their
text (plain/markdown/csv via decode, PDF via pdfplumber) is chunked and
indexed in the KnowledgeStore with attachment provenance metadata.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add disk-persistent storage for ColBERT token-level embeddings so the
reranker can reuse pre-computed embeddings across queries instead of
re-encoding every document on every search.
- EmbeddingStore: individual .pt files per chunk with SQLite index for
O(1) lookup, graceful degradation when torch is not installed
- ColBERTReranker: checks EmbeddingStore for cached embeddings before
falling back to docFromText(), stores newly computed embeddings
- KnowledgeStore: ensures chunk_id is always present in retrieval
metadata (both at store time and as a backfill in retrieve)
- 18 tests covering round-trip persistence, deletion, torch-absent
degradation, and reranker cache-hit/miss behavior
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds a new POST /{connector_id}/sync endpoint alongside the existing GET
sync-status endpoint. The new endpoint validates the connector is registered
and connected, then runs SyncEngine.sync() and returns chunks_indexed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements AttachmentStore that writes blobs to {base_dir}/{sha[:2]}/{sha}
with a SQLite metadata index tracking filename, MIME type, size, and the
accumulated list of source_doc_ids for each content-identical blob.
Includes 7 unit tests covering SHA-256 return, file path layout,
idempotent dedup, multi-source tracking, metadata retrieval, content
round-trip, and missing-blob None return.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Parse the last_sync checkpoint timestamp into a datetime and pass it as
the `since` argument to connector.sync(), enabling subsequent syncs to
fetch only new items rather than a full re-fetch.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements create_connectors_router() with GET /connectors, GET
/connectors/{id}, POST /connectors/{id}/connect, POST
/connectors/{id}/disconnect, and GET /connectors/{id}/sync endpoints.
Includes 6 passing tests covering list, detail, 404, connect, disconnect,
and sync status.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds tests/agents/test_channel_agent_integration.py covering the full
stack from KnowledgeStore + IngestionPipeline through TwoStageRetriever,
KnowledgeSearchTool, and DeepResearchAgent into ChannelAgent/FakeChannel:
- test_quick_query_inline_response: asserts inline reply with meeting info
and no openjarvis:// escalation link
- test_deep_query_escalation_link: asserts openjarvis:// link is present
when engine issues a tool call and returns a long report
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds classify_query() heuristic (quick vs deep) and ChannelAgent that
bridges BaseChannel messages to any agent, delivering inline replies for
quick/short responses and preview+openjarvis:// escalation links for deep
or long responses. 18 tests cover classifier cases and agent behaviour
including non-blocking handler and error recovery.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Tests the full path from multi-source Document ingestion through
IngestionPipeline -> KnowledgeStore -> TwoStageRetriever ->
KnowledgeSearchTool -> DeepResearchAgent to a cited report, and
verifies cross-platform retrieval returns results from >= 2 sources.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Multi-hop research agent that uses native function calling (OpenAI
tool_calls format) to search personal data across sources via
KnowledgeSearchTool, cross-references results, and produces narrative
answers with inline source citations.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add optional retriever parameter to KnowledgeSearchTool so it can
delegate to TwoStageRetriever (BM25 + reranking) when supplied, falling
back to direct KnowledgeStore retrieval otherwise.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two bugs found via live API testing:
- List notes API returns "notes" key, not "data" (was returning 0 results)
- Transcript speaker field is a dict {"source": "microphone"}, not a string
Verified: 112 real meeting notes synced, 2,226 chunks indexed and searchable.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Slack connector now skips bot_id messages and non-content subtypes
(message_changed, message_deleted, bot_message, channel_join, channel_leave).
Found by comparing against hermes-agent reference implementation.
Live smoke test indexes 4,467 chunks from real docs/ markdown files and
verifies the full pipeline: Obsidian → SyncEngine → KnowledgeStore → search.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Break long lines in test_notion.py, test_store.py, and test_sync_engine.py
to comply with the 88-character E501 limit enforced by ruff.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements IMessageConnector (auth_type='local') that opens ~/Library/Messages/chat.db
read-only, converts Apple nanosecond timestamps to UTC datetimes, and yields one
Document per message with handle-based author resolution and chat display names.
Includes 8 tests using a temporary SQLite DB and 2 MCP tools (search + get conversation).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements GContactsConnector using the People API v1 to paginate
contacts (names, emails, phones, orgs) and yield them as Documents
with doc_type="contact". Includes 6 unit tests and auto-registration
in the connectors __init__.py.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements GCalendarConnector (registered as "gcalendar") following the
gmail.py pattern: module-level API functions for calendarList and events.list,
_format_event helper for human-readable content, paginated sync across all
calendars, and three MCP tools (get_events_today, search_events, next_meeting).
Includes 6 tests covering not_connected, auth_url scope, sync document fields,
disconnect, mcp_tools, and registry.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements GDriveConnector using the Drive API v3 with OAuth, paginated
file listing, Google Workspace export (Docs→text/plain, Sheets→text/csv,
Slides→text/plain), and 3 MCP tools. Adds auto-registration in __init__.py.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements SlackConnector that syncs channel message history via the Slack
Web API, with OAuth credential storage, user-map resolution, and 3 MCP tools.
Includes 7 tests covering connectivity, auth, sync document correctness,
disconnect, MCP tools, and registry registration.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements GranolaConnector that syncs meeting notes from the Granola
public API, combining AI-generated summaries with full speaker transcripts
into searchable Documents. Includes cursor-based pagination, created_after
filtering, 2 MCP tools, and 8 tests covering all connector behaviours.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements NotionConnector that syncs pages via the Notion REST API,
renders block content (paragraph, headings, lists, code, quote, divider,
to_do) to markdown, and registers with ConnectorRegistry. Includes 8
tests covering auth, sync, rendering, disconnect, MCP tools, and registry.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Wire up ObsidianConnector and GmailConnector via auto-import in the
connectors __init__.py so they register on package import. Add a full
end-to-end integration test covering the Obsidian vault → SyncEngine →
KnowledgeStore → knowledge_search pipeline.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds connect_cmd.py with --list, --disconnect, --sync, and positional source
routing (filesystem vs OAuth), registers it in the CLI, and covers all
behaviours with 5 passing unit tests.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements KnowledgeSearchTool registered under "knowledge_search" that wraps
KnowledgeStore with optional filters (source, doc_type, author, since, until,
top_k) and formats results with source attribution for agent consumption.
Includes 8 unit tests covering basic search, filter-by-source, filter-by-author,
no-results, empty-query, no-store, spec parameters, and registry checks.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements GmailConnector registered under 'gmail' in ConnectorRegistry,
a shared oauth.py helper (build_google_auth_url, load/save/delete_tokens)
reusable by Drive/Calendar/Contacts, and 7 fully mocked pytest tests
covering auth state, sync document extraction, disconnect, mcp_tools,
and registry lookup.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Implements ObsidianConnector (filesystem auth, no OAuth) that walks a vault
directory for .md/.markdown/.txt files, parses YAML frontmatter via a
dependency-free parser, skips hidden dirs and binary files, and yields
Document objects with doc_type="note" and obsidian:// deep-link URLs.
Exposes an obsidian_search_notes MCP ToolSpec. 9 tests cover connection
state, vault traversal, hidden-dir/binary filtering, frontmatter extraction,
and registry wiring.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Introduces SyncEngine that wraps IngestionPipeline with a SQLite state
database (sync_state.db) for checkpoint/resume: cursors and item counts
are persisted after every 100-document batch and on completion/error.
Adds 4 tests covering single-connector ingestion, checkpoint accuracy,
unsynced-connector None return, and multi-connector source filtering.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements IngestionPipeline that deduplicates Documents by doc_id (both
in-memory and by loading existing doc_ids from the store on init), chunks
content via SemanticChunker, and persists all chunks with full provenance
metadata to KnowledgeStore. 8 tests cover single-doc ingestion, dedup across
calls and batches, persistence across pipeline instances, long-doc multi-chunk
splitting, atomic event chunking, multi-source filtering, and return-value accuracy.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements SemanticChunker that splits text based on doc_type (event/contact
as single chunks, email on reply boundaries, message on double-newlines,
document/note on ## headings → paragraphs → sentences). ChunkResult carries
sequential 0-based indexes and inherits parent metadata; section headings are
added as chunk metadata. 16 tests covering all splitting strategies.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Implements KnowledgeStore (extends MemoryBackend) for Deep Research with a
rich SQLite/FTS5 schema supporting source, doc_type, author, participants,
timestamp, thread_id, url, and chunk_index columns; BM25 ranking via FTS5
with porter unicode61 tokenizer; filtered retrieval by source, doc_type,
author, since, and until; MEMORY_STORE/MEMORY_RETRIEVE event emission; WAL
journal mode; and 15 isolated tests using tmp_path fixtures.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Move mid-file imports to top of test file to resolve E402 violations and apply ruff formatting to both gemma_cpp.py and test_gemma_cpp.py.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
