Data collection:
- Restructure digest_collect output into priority-ordered sections
(HEALTH > MESSAGES > CALENDAR > MUSIC) with human-readable formatting
- Extract key metrics from Oura (HR, HRV, sleep duration, scores)
- Format Gmail as sender + subject + time ago
- Group music tracks into single lines per source
- Cap 15 items per source, truncate content to 500 chars
Calendar:
- Pass `since` parameter to Google Calendar API as `timeMin`
- Default to 24h lookback instead of dumping all-time events
Gmail:
- Re-enable gmail connector registration in __init__.py
Prompt:
- Add section summary + closing encouragement to Jarvis persona
- Reinforce no-markdown, no-hallucination rules in user message
- Strip markdown artifacts before sending to TTS
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Full morning digest system:
- MorningDigestAgent, DigestStore, digest_collect tool, TTS backends
- Connectors: Oura, Strava, Spotify, Google Tasks, Apple Health, Apple Music
- CLI `jarvis digest` command + FastAPI /api/digest endpoints
- Cartesia, Kokoro, OpenAI TTS backends with persona prompts
Unified OAuth setup across all surfaces:
- Generic OAuthProvider registry for Google, Strava, Spotify
- CLI auto-opens browser + catches callback (no more paste-code)
- Server /oauth/start + /oauth/callback endpoints for desktop/browser
- Frontend OAuthPanel with popup + polling
Apple connectors:
- Apple Health reads from HealthKit DB or iPhone export XML
- Apple Music queries Music.app via AppleScript
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- test_deep_research_prompt.py (8 tests): date injection, time, day of week,
dynamic generation, response types, tool descriptions, /no_think, Jarvis name
- test_slack_messaging.py (7 tests): connect with/without tokens, error state,
disconnect, handler registration, send without token
- test_connector_health.py (44 tests): all 11 connectors instantiate, have
required methods, report not-connected without creds, have metadata.
Plus KnowledgeStore data presence checks.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add exchange_google_token() and run_oauth_flow() to oauth.py for the
full authorization code exchange. Update gdrive, gcalendar, and
gcontacts connectors to trigger the browser-based OAuth flow when a
client_id:client_secret pair is provided, prefer access_token over raw
token, and require an actual access_token for is_connected(). auth_url()
now returns the Cloud Console credentials page when no client_id is
stored.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Implements SyncScheduler that runs a daemon background thread to call
SyncEngine.sync() on all registered connectors at a configurable interval.
Provides run_once() as a synchronous helper for testing. Disconnected
connectors are skipped each cycle; errors are logged without stopping the
loop. 4 tests covering run_once, disconnected skip, start/stop, and
per-connector chunk counts.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements WhatsAppConnector that reads exported .txt chat files from a
directory, parses the standard WhatsApp export line format with regex,
and yields one Document per chat file with participants, timestamps, and
since filtering. Registers under "whatsapp" in ConnectorRegistry with
MCP tools for whatsapp_search_messages and whatsapp_get_chat. 10 tests
covering parsing, multi-file, since filtering, is_connected, and registry.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove the ARG002 noqa suppression and translate the `since` datetime
parameter into a Gmail `after:<epoch>` query string passed to
`_gmail_api_list_messages`, so incremental syncs only fetch messages
newer than the requested cutoff.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add optional `attachment_store` parameter to IngestionPipeline. When
provided, attachments are stored as blobs in AttachmentStore and their
text (plain/markdown/csv via decode, PDF via pdfplumber) is chunked and
indexed in the KnowledgeStore with attachment provenance metadata.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add disk-persistent storage for ColBERT token-level embeddings so the
reranker can reuse pre-computed embeddings across queries instead of
re-encoding every document on every search.
- EmbeddingStore: individual .pt files per chunk with SQLite index for
O(1) lookup, graceful degradation when torch is not installed
- ColBERTReranker: checks EmbeddingStore for cached embeddings before
falling back to docFromText(), stores newly computed embeddings
- KnowledgeStore: ensures chunk_id is always present in retrieval
metadata (both at store time and as a backfill in retrieve)
- 18 tests covering round-trip persistence, deletion, torch-absent
degradation, and reranker cache-hit/miss behavior
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Implements AttachmentStore that writes blobs to {base_dir}/{sha[:2]}/{sha}
with a SQLite metadata index tracking filename, MIME type, size, and the
accumulated list of source_doc_ids for each content-identical blob.
Includes 7 unit tests covering SHA-256 return, file path layout,
idempotent dedup, multi-source tracking, metadata retrieval, content
round-trip, and missing-blob None return.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Parse the last_sync checkpoint timestamp into a datetime and pass it as
the `since` argument to connector.sync(), enabling subsequent syncs to
fetch only new items rather than a full re-fetch.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two bugs found via live API testing:
- List notes API returns "notes" key, not "data" (was returning 0 results)
- Transcript speaker field is a dict {"source": "microphone"}, not a string
Verified: 112 real meeting notes synced, 2,226 chunks indexed and searchable.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Slack connector now skips bot_id messages and non-content subtypes
(message_changed, message_deleted, bot_message, channel_join, channel_leave).
Found by comparing against hermes-agent reference implementation.
Live smoke test indexes 4,467 chunks from real docs/ markdown files and
verifies the full pipeline: Obsidian → SyncEngine → KnowledgeStore → search.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Break long lines in test_notion.py, test_store.py, and test_sync_engine.py
to comply with the 88-character E501 limit enforced by ruff.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements IMessageConnector (auth_type='local') that opens ~/Library/Messages/chat.db
read-only, converts Apple nanosecond timestamps to UTC datetimes, and yields one
Document per message with handle-based author resolution and chat display names.
Includes 8 tests using a temporary SQLite DB and 2 MCP tools (search + get conversation).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements GContactsConnector using the People API v1 to paginate
contacts (names, emails, phones, orgs) and yield them as Documents
with doc_type="contact". Includes 6 unit tests and auto-registration
in the connectors __init__.py.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements GCalendarConnector (registered as "gcalendar") following the
gmail.py pattern: module-level API functions for calendarList and events.list,
_format_event helper for human-readable content, paginated sync across all
calendars, and three MCP tools (get_events_today, search_events, next_meeting).
Includes 6 tests covering not_connected, auth_url scope, sync document fields,
disconnect, mcp_tools, and registry.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements GDriveConnector using the Drive API v3 with OAuth, paginated
file listing, Google Workspace export (Docs→text/plain, Sheets→text/csv,
Slides→text/plain), and 3 MCP tools. Adds auto-registration in __init__.py.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements SlackConnector that syncs channel message history via the Slack
Web API, with OAuth credential storage, user-map resolution, and 3 MCP tools.
Includes 7 tests covering connectivity, auth, sync document correctness,
disconnect, MCP tools, and registry registration.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements GranolaConnector that syncs meeting notes from the Granola
public API, combining AI-generated summaries with full speaker transcripts
into searchable Documents. Includes cursor-based pagination, created_after
filtering, 2 MCP tools, and 8 tests covering all connector behaviours.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements NotionConnector that syncs pages via the Notion REST API,
renders block content (paragraph, headings, lists, code, quote, divider,
to_do) to markdown, and registers with ConnectorRegistry. Includes 8
tests covering auth, sync, rendering, disconnect, MCP tools, and registry.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Wire up ObsidianConnector and GmailConnector via auto-import in the
connectors __init__.py so they register on package import. Add a full
end-to-end integration test covering the Obsidian vault → SyncEngine →
KnowledgeStore → knowledge_search pipeline.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements GmailConnector registered under 'gmail' in ConnectorRegistry,
a shared oauth.py helper (build_google_auth_url, load/save/delete_tokens)
reusable by Drive/Calendar/Contacts, and 7 fully mocked pytest tests
covering auth state, sync document extraction, disconnect, mcp_tools,
and registry lookup.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Implements ObsidianConnector (filesystem auth, no OAuth) that walks a vault
directory for .md/.markdown/.txt files, parses YAML frontmatter via a
dependency-free parser, skips hidden dirs and binary files, and yields
Document objects with doc_type="note" and obsidian:// deep-link URLs.
Exposes an obsidian_search_notes MCP ToolSpec. 9 tests cover connection
state, vault traversal, hidden-dir/binary filtering, frontmatter extraction,
and registry wiring.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Introduces SyncEngine that wraps IngestionPipeline with a SQLite state
database (sync_state.db) for checkpoint/resume: cursors and item counts
are persisted after every 100-document batch and on completion/error.
Adds 4 tests covering single-connector ingestion, checkpoint accuracy,
unsynced-connector None return, and multi-connector source filtering.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements IngestionPipeline that deduplicates Documents by doc_id (both
in-memory and by loading existing doc_ids from the store on init), chunks
content via SemanticChunker, and persists all chunks with full provenance
metadata to KnowledgeStore. 8 tests cover single-doc ingestion, dedup across
calls and batches, persistence across pipeline instances, long-doc multi-chunk
splitting, atomic event chunking, multi-source filtering, and return-value accuracy.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements SemanticChunker that splits text based on doc_type (event/contact
as single chunks, email on reply boundaries, message on double-newlines,
document/note on ## headings → paragraphs → sentences). ChunkResult carries
sequential 0-based indexes and inherits parent metadata; section headings are
added as chunk metadata. 16 tests covering all splitting strategies.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Implements KnowledgeStore (extends MemoryBackend) for Deep Research with a
rich SQLite/FTS5 schema supporting source, doc_type, author, participants,
timestamp, thread_id, url, and chunk_index columns; BM25 ranking via FTS5
with porter unicode61 tokenizer; filtered retrieval by source, doc_type,
author, since, and until; MEMORY_STORE/MEMORY_RETRIEVE event emission; WAL
journal mode; and 15 isolated tests using tmp_path fixtures.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
