Commit Graph
85 Commits
Author SHA1 Message Date
krypticmouseandClaude Sonnet 4.6 5cf2de5f74 feat: add end-to-end integration test for Deep Research pipeline
Tests the full path from multi-source Document ingestion through
IngestionPipeline -> KnowledgeStore -> TwoStageRetriever ->
KnowledgeSearchTool -> DeepResearchAgent to a cited report, and
verifies cross-platform retrieval returns results from >= 2 sources.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 03:11:53 +00:00
krypticmouseandClaude Opus 4.6 39a8117596 feat: add DeepResearchAgent with multi-hop retrieval and cited reports
Multi-hop research agent that uses native function calling (OpenAI
tool_calls format) to search personal data across sources via
KnowledgeSearchTool, cross-references results, and produces narrative
answers with inline source citations.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 03:09:23 +00:00
krypticmouseandClaude Sonnet 4.6 0917ebab25 feat: upgrade knowledge_search to support TwoStageRetriever
Add optional retriever parameter to KnowledgeSearchTool so it can
delegate to TwoStageRetriever (BM25 + reranking) when supplied, falling
back to direct KnowledgeStore retrieval otherwise.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 03:05:27 +00:00
krypticmouseandClaude Sonnet 4.6 2967419691 feat: add TwoStageRetriever with BM25 recall and pluggable ColBERT reranking
Introduces Reranker ABC, ColBERTReranker (lazy-loads colbert-ai with graceful
fallback), and TwoStageRetriever that composes KnowledgeStore BM25 recall with
optional semantic reranking. Includes 8 tests covering filters, top_k limits,
mock reranker invocation, and recall_k sizing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 03:03:48 +00:00
krypticmouseandClaude Opus 4.6 ad2f6cc03c fix: Granola connector uses correct API response key and speaker format
Two bugs found via live API testing:
- List notes API returns "notes" key, not "data" (was returning 0 results)
- Transcript speaker field is a dict {"source": "microphone"}, not a string

Verified: 112 real meeting notes synced, 2,226 chunks indexed and searchable.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 02:25:39 +00:00
krypticmouseandClaude Opus 4.6 20d132ff4a fix: add bot message filtering to Slack connector, add live smoke test
Slack connector now skips bot_id messages and non-content subtypes
(message_changed, message_deleted, bot_message, channel_join, channel_leave).
Found by comparing against hermes-agent reference implementation.

Live smoke test indexes 4,467 chunks from real docs/ markdown files and
verifies the full pipeline: Obsidian → SyncEngine → KnowledgeStore → search.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 02:06:43 +00:00
krypticmouseandClaude Sonnet 4.6 e3e9a70181 style: fix line length violations in connector tests
Break long lines in test_notion.py, test_store.py, and test_sync_engine.py
to comply with the 88-character E501 limit enforced by ruff.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 01:45:18 +00:00
krypticmouseandClaude Sonnet 4.6 506bcd88bd feat: add iMessage connector reading from macOS Messages database
Implements IMessageConnector (auth_type='local') that opens ~/Library/Messages/chat.db
read-only, converts Apple nanosecond timestamps to UTC datetimes, and yields one
Document per message with handle-based author resolution and chat display names.
Includes 8 tests using a temporary SQLite DB and 2 MCP tools (search + get conversation).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 01:43:02 +00:00
krypticmouseandClaude Sonnet 4.6 eb339294cb feat: add Google Contacts connector with People API sync
Implements GContactsConnector using the People API v1 to paginate
contacts (names, emails, phones, orgs) and yield them as Documents
with doc_type="contact". Includes 6 unit tests and auto-registration
in the connectors __init__.py.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 01:40:01 +00:00
krypticmouseandClaude Sonnet 4.6 79616c9da4 feat: add Google Calendar connector with event sync
Implements GCalendarConnector (registered as "gcalendar") following the
gmail.py pattern: module-level API functions for calendarList and events.list,
_format_event helper for human-readable content, paginated sync across all
calendars, and three MCP tools (get_events_today, search_events, next_meeting).
Includes 6 tests covering not_connected, auth_url scope, sync document fields,
disconnect, mcp_tools, and registry.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 01:37:28 +00:00
krypticmouseandClaude Sonnet 4.6 89e757adba feat: add Google Drive connector with file export and sync
Implements GDriveConnector using the Drive API v3 with OAuth, paginated
file listing, Google Workspace export (Docs→text/plain, Sheets→text/csv,
Slides→text/plain), and 3 MCP tools. Adds auto-registration in __init__.py.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 01:34:43 +00:00
krypticmouseandClaude Sonnet 4.6 0d9c25c6f9 feat: add Slack data source connector with channel history sync
Implements SlackConnector that syncs channel message history via the Slack
Web API, with OAuth credential storage, user-map resolution, and 3 MCP tools.
Includes 7 tests covering connectivity, auth, sync document correctness,
disconnect, MCP tools, and registry registration.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 01:32:05 +00:00
krypticmouseandClaude Sonnet 4.6 1257303cd4 feat: add Granola meeting notes connector with transcript support
Implements GranolaConnector that syncs meeting notes from the Granola
public API, combining AI-generated summaries with full speaker transcripts
into searchable Documents. Includes cursor-based pagination, created_after
filtering, 2 MCP tools, and 8 tests covering all connector behaviours.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 01:20:12 +00:00
krypticmouseandClaude Sonnet 4.6 547b6ba734 feat: add Notion connector with block-to-markdown rendering
Implements NotionConnector that syncs pages via the Notion REST API,
renders block content (paragraph, headings, lists, code, quote, divider,
to_do) to markdown, and registers with ConnectorRegistry. Includes 8
tests covering auth, sync, rendering, disconnect, MCP tools, and registry.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 01:17:16 +00:00
krypticmouseandClaude Sonnet 4.6 138fb8a3d0 feat: add integration test and auto-registration for built-in connectors
Wire up ObsidianConnector and GmailConnector via auto-import in the
connectors __init__.py so they register on package import. Add a full
end-to-end integration test covering the Obsidian vault → SyncEngine →
KnowledgeStore → knowledge_search pipeline.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 00:09:36 +00:00
krypticmouseandClaude Sonnet 4.6 a23db16a12 feat: add 'jarvis connect' CLI command for managing data source connections
Adds connect_cmd.py with --list, --disconnect, --sync, and positional source
routing (filesystem vs OAuth), registers it in the CLI, and covers all
behaviours with 5 passing unit tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 00:06:29 +00:00
krypticmouseandClaude Sonnet 4.6 ffc43cd7b5 feat: add knowledge_search tool with filtered BM25 retrieval and source attribution
Implements KnowledgeSearchTool registered under "knowledge_search" that wraps
KnowledgeStore with optional filters (source, doc_type, author, since, until,
top_k) and formats results with source attribution for agent consumption.
Includes 8 unit tests covering basic search, filter-by-source, filter-by-author,
no-results, empty-query, no-store, spec parameters, and registry checks.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 00:02:02 +00:00
krypticmouseandClaude Opus 4.6 56554c481a feat: add Gmail connector with OAuth and mocked API sync
Implements GmailConnector registered under 'gmail' in ConnectorRegistry,
a shared oauth.py helper (build_google_auth_url, load/save/delete_tokens)
reusable by Drive/Calendar/Contacts, and 7 fully mocked pytest tests
covering auth state, sync document extraction, disconnect, mcp_tools,
and registry lookup.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 23:56:29 +00:00
krypticmouseandClaude Opus 4.6 5e0e8965d7 feat: add Obsidian/Markdown vault connector with frontmatter parsing
Implements ObsidianConnector (filesystem auth, no OAuth) that walks a vault
directory for .md/.markdown/.txt files, parses YAML frontmatter via a
dependency-free parser, skips hidden dirs and binary files, and yields
Document objects with doc_type="note" and obsidian:// deep-link URLs.
Exposes an obsidian_search_notes MCP ToolSpec.  9 tests cover connection
state, vault traversal, hidden-dir/binary filtering, frontmatter extraction,
and registry wiring.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 23:52:49 +00:00
krypticmouseandClaude Sonnet 4.6 af7d0db518 feat: add SyncEngine with checkpoint/resume for connector orchestration
Introduces SyncEngine that wraps IngestionPipeline with a SQLite state
database (sync_state.db) for checkpoint/resume: cursors and item counts
are persisted after every 100-document batch and on completion/error.
Adds 4 tests covering single-connector ingestion, checkpoint accuracy,
unsynced-connector None return, and multi-connector source filtering.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 23:49:54 +00:00
krypticmouseandClaude Sonnet 4.6 1a713e66fe feat: add ingestion pipeline with dedup, type-aware chunking, and indexed storage
Implements IngestionPipeline that deduplicates Documents by doc_id (both
in-memory and by loading existing doc_ids from the store on init), chunks
content via SemanticChunker, and persists all chunks with full provenance
metadata to KnowledgeStore. 8 tests cover single-doc ingestion, dedup across
calls and batches, persistence across pipeline instances, long-doc multi-chunk
splitting, atomic event chunking, multi-source filtering, and return-value accuracy.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 23:47:59 +00:00
krypticmouseandClaude Opus 4.6 efc979edf0 feat: add type-aware semantic chunker with section/paragraph/sentence splitting
Implements SemanticChunker that splits text based on doc_type (event/contact
as single chunks, email on reply boundaries, message on double-newlines,
document/note on ## headings → paragraphs → sentences). ChunkResult carries
sequential 0-based indexes and inherits parent metadata; section headings are
added as chunk metadata. 16 tests covering all splitting strategies.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 23:45:38 +00:00
krypticmouseandClaude Sonnet 4.6 8d3505f8f3 feat: add KnowledgeStore with source-aware SQLite schema and filtered BM25 retrieval
Implements KnowledgeStore (extends MemoryBackend) for Deep Research with a
rich SQLite/FTS5 schema supporting source, doc_type, author, participants,
timestamp, thread_id, url, and chunk_index columns; BM25 ranking via FTS5
with porter unicode61 tokenizer; filtered retrieval by source, doc_type,
author, since, and until; MEMORY_STORE/MEMORY_RETRIEVE event emission; WAL
journal mode; and 15 isolated tests using tmp_path fixtures.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 23:42:39 +00:00
krypticmouseandClaude Sonnet 4.6 29d275fe4e feat: add ConnectorRegistry and base connector types (Document, SyncStatus, BaseConnector)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 23:35:01 +00:00
Jon Saad-FalconandGitHub c390c73f85 Merge pull request #123 from open-jarvis/feat/gemma-cpp-engine
feat: add gemma.cpp engine via pygemma pybind11 bindings
2026-03-25 14:57:03 -07:00
Jon Saad-FalconandClaude Sonnet 4.6 e4d036dd50 test: add live integration test stubs for gemma_cpp engine
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 13:18:13 -07:00
Jon Saad-FalconandClaude Sonnet 4.6 16025571fd style: fix lint issues in gemma_cpp engine
Move mid-file imports to top of test file to resolve E402 violations and apply ruff formatting to both gemma_cpp.py and test_gemma_cpp.py.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 13:17:16 -07:00
Jon Saad-FalconandClaude Sonnet 4.6 d08360ca21 feat: wire gemma_cpp engine into discovery and optional imports
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 13:15:06 -07:00
Jon Saad-FalconandGitHub 15dad86832 Merge pull request #121 from open-jarvis/feat/pinchbench-integration
feat: PinchBench benchmark integration
2026-03-25 12:51:01 -07:00
Jon Saad-FalconandClaude Sonnet 4.6 411e822916 test: add config resolution tests for gemma_cpp engine
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 12:40:24 -07:00
Jon Saad-FalconandClaude Sonnet 4.6 8b0fd702c4 feat: implement gemma_cpp health checks and model listing
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 12:39:38 -07:00
Jon Saad-FalconandClaude Opus 4.6 8c2ee6843a feat: implement gemma_cpp engine lifecycle and inference methods
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 12:38:25 -07:00
Jon Saad-FalconandClaude Sonnet 4.6 9cd822611e feat: add GemmaCppEngine skeleton with chat template formatting
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 12:36:35 -07:00
Jon Saad-FalconandClaude Sonnet 4.6 d589bfb55d feat: add GemmaCppEngineConfig and wire into EngineConfig
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 12:34:48 -07:00
Jon Saad-FalconandClaude Opus 4.6 784b880130 test(evals): add PinchBench integration tests — full grading pipeline
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 09:31:02 -07:00
Jon Saad-FalconandClaude Opus 4.6 73cb1f1bbd feat(evals): add PinchBench grading helpers — transcript translation, automated/LLM judge/hybrid scoring
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 09:26:12 -07:00
Jon Saad-FalconandClaude Opus 4.6 8e8d4a68e4 feat(evals): add PinchBench dataset provider — repo clone and task markdown parsing
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 09:25:39 -07:00
Jon Saad-FalconandClaude Sonnet 4.6 c5cb6a9e49 feat(evals): add tool_calls field to TurnTrace for rich tool data
Adds a `tool_calls` list field to `TurnTrace` capturing name, arguments,
and result for each tool invocation, enabling PinchBench transcript integration.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 09:20:25 -07:00
Jon Saad-FalconandGitHub 14c81bba98 Merge pull request #115 from open-jarvis/feat/init-onboarding-and-privacy-scanner
feat: interactive model download in init + privacy environment scanner
2026-03-25 08:12:45 -07:00
Jon Saad-FalconandClaude Opus 4.6 68a8f5646c fix: add --no-download and PrivacyScanner mock to test_cli init test
The test_init_creates_config test in test_cli.py was not updated
when the download prompt and privacy hook were added to jarvis init.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 08:07:35 -07:00
Jon Saad-FalconandClaude Opus 4.6 b2152fb40e style: fix E501 line-length and unused imports in test files
Wraps long CliRunner.invoke() calls, removes unused pytest imports,
fixes import ordering, and removes unused variables in test_scan.py.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 20:38:14 -07:00
Jon Saad-FalconandClaude Opus 4.6 a934977ba8 fix: address spec review findings for privacy scanner
- Add slots=True to ScanResult dataclass
- Fix extra space in IPv6 port f-string
- Add missing tests: check_remote_access, check_icloud_sync, run_quick

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 18:26:05 -07:00
Jon Saad-FalconandClaude Sonnet 4.6 02bc36c1f8 feat: register jarvis scan command and add init privacy hook
Registers the new scan command in the CLI. Adds a lightweight
privacy check at the end of jarvis init that runs disk encryption
and cloud sync checks, with pointer to jarvis scan for full audit.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-24 18:22:13 -07:00
Jon Saad-FalconandClaude Opus 4.6 e2376d29d1 feat: add interactive model download and empty-model fallback to init
Prompts user to download recommended model during jarvis init.
Adds --no-download flag for CI. Shows helpful message when no
model fits available memory. Adds exo/nexa next-steps text.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 18:16:59 -07:00
Jon Saad-Falcon 00f71ae504 fix: update energy scaling test for linear KV-cache FLOPs model
The FLOPs formula changed from quadratic (P*N*(N+1)) to linear
(2*P*T_evaluated) with KV-cache awareness. Update the test
expectation from ~100x to ~10x for 10x token increase.
2026-03-24 18:16:11 -07:00
Jon Saad-FalconandClaude Opus 4.6 a05c2b7df6 feat: extract ollama_pull helper and add multi-engine model pull
Refactors model pull into reusable ollama_pull() function. Adds
--engine flag to support llamacpp (GGUF) and mlx (HuggingFace)
downloads via huggingface-cli, with FileNotFoundError handling.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 18:12:55 -07:00
Jon Saad-FalconandClaude Opus 4.6 ced38bcba3 feat: add privacy environment scanner with platform-specific checks
New PrivacyScanner class with checks for disk encryption (FileVault/LUKS),
MDM profiles, cloud sync agents, network exposure, and screen recording.
Supports macOS and Linux with graceful skip on missing tools.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 18:07:54 -07:00
Jon Saad-FalconandClaude Opus 4.6 7d3b4e5ea3 fix: add MLX engine support to Qwen3.5 model catalog entries
Fixes recommend_model() returning empty string on Apple Silicon
when MLX is the recommended engine. Also adds gguf_file and
mlx_repo download metadata, and estimated_download_gb helper.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 18:04:25 -07:00
Jon Saad-FalconandClaude Opus 4.6 9dbd172950 fix: mock config in test_init_defaults to test class-level defaults
The test_init_defaults test expected OperativeAgent class defaults
(temperature=0.3) but didn't mock load_config(), so when config loaded
successfully it returned the global default (0.7) instead.

Fix: mock load_config to raise, so the test properly validates the
class-level _default_temperature/max_tokens/max_turns fallback path.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 17:30:46 -07:00
Jon Saad-FalconandGitHub 033ced67e6 Merge pull request #109 from open-jarvis/fix/agent-config-defaults-103
fix: agent constructor defaults resolve from config.toml
2026-03-24 16:54:44 -07:00