Commit Graph
196 Commits
Author SHA1 Message Date
krypticmouseandClaude Sonnet 4.6 ffc43cd7b5 feat: add knowledge_search tool with filtered BM25 retrieval and source attribution
Implements KnowledgeSearchTool registered under "knowledge_search" that wraps
KnowledgeStore with optional filters (source, doc_type, author, since, until,
top_k) and formats results with source attribution for agent consumption.
Includes 8 unit tests covering basic search, filter-by-source, filter-by-author,
no-results, empty-query, no-store, spec parameters, and registry checks.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 00:02:02 +00:00
krypticmouseandClaude Opus 4.6 56554c481a feat: add Gmail connector with OAuth and mocked API sync
Implements GmailConnector registered under 'gmail' in ConnectorRegistry,
a shared oauth.py helper (build_google_auth_url, load/save/delete_tokens)
reusable by Drive/Calendar/Contacts, and 7 fully mocked pytest tests
covering auth state, sync document extraction, disconnect, mcp_tools,
and registry lookup.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 23:56:29 +00:00
krypticmouseandClaude Opus 4.6 5e0e8965d7 feat: add Obsidian/Markdown vault connector with frontmatter parsing
Implements ObsidianConnector (filesystem auth, no OAuth) that walks a vault
directory for .md/.markdown/.txt files, parses YAML frontmatter via a
dependency-free parser, skips hidden dirs and binary files, and yields
Document objects with doc_type="note" and obsidian:// deep-link URLs.
Exposes an obsidian_search_notes MCP ToolSpec.  9 tests cover connection
state, vault traversal, hidden-dir/binary filtering, frontmatter extraction,
and registry wiring.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 23:52:49 +00:00
krypticmouseandClaude Sonnet 4.6 af7d0db518 feat: add SyncEngine with checkpoint/resume for connector orchestration
Introduces SyncEngine that wraps IngestionPipeline with a SQLite state
database (sync_state.db) for checkpoint/resume: cursors and item counts
are persisted after every 100-document batch and on completion/error.
Adds 4 tests covering single-connector ingestion, checkpoint accuracy,
unsynced-connector None return, and multi-connector source filtering.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 23:49:54 +00:00
krypticmouseandClaude Sonnet 4.6 1a713e66fe feat: add ingestion pipeline with dedup, type-aware chunking, and indexed storage
Implements IngestionPipeline that deduplicates Documents by doc_id (both
in-memory and by loading existing doc_ids from the store on init), chunks
content via SemanticChunker, and persists all chunks with full provenance
metadata to KnowledgeStore. 8 tests cover single-doc ingestion, dedup across
calls and batches, persistence across pipeline instances, long-doc multi-chunk
splitting, atomic event chunking, multi-source filtering, and return-value accuracy.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 23:47:59 +00:00
krypticmouseandClaude Opus 4.6 efc979edf0 feat: add type-aware semantic chunker with section/paragraph/sentence splitting
Implements SemanticChunker that splits text based on doc_type (event/contact
as single chunks, email on reply boundaries, message on double-newlines,
document/note on ## headings → paragraphs → sentences). ChunkResult carries
sequential 0-based indexes and inherits parent metadata; section headings are
added as chunk metadata. 16 tests covering all splitting strategies.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 23:45:38 +00:00
krypticmouseandClaude Sonnet 4.6 8d3505f8f3 feat: add KnowledgeStore with source-aware SQLite schema and filtered BM25 retrieval
Implements KnowledgeStore (extends MemoryBackend) for Deep Research with a
rich SQLite/FTS5 schema supporting source, doc_type, author, participants,
timestamp, thread_id, url, and chunk_index columns; BM25 ranking via FTS5
with porter unicode61 tokenizer; filtered retrieval by source, doc_type,
author, since, and until; MEMORY_STORE/MEMORY_RETRIEVE event emission; WAL
journal mode; and 15 isolated tests using tmp_path fixtures.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 23:42:39 +00:00
krypticmouseandClaude Sonnet 4.6 29d275fe4e feat: add ConnectorRegistry and base connector types (Document, SyncStatus, BaseConnector)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 23:35:01 +00:00
krypticmouseandClaude Opus 4.6 e9e4bb874b docs: add Phase 1 implementation plan for Deep Research connector foundation
10-task TDD plan covering: ConnectorRegistry, KnowledgeStore, semantic chunker,
ingestion pipeline, SyncEngine, OAuth helper, Obsidian connector, Gmail connector,
knowledge_search tool, and jarvis connect CLI.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 23:21:09 +00:00
krypticmouseandClaude Opus 4.6 59ac24205e docs: add Deep Research setup experience design spec
Comprehensive spec covering the end-to-end "killer install" experience:
- Desktop-first setup wizard with guided OAuth for 12 data sources
- BaseConnector abstraction with native bulk sync + MCP real-time tools
- Ingestion pipeline with semantic chunking and dual-write indexing
- BM25 → ColBERTv2 two-stage retrieval with agent multi-hop
- DeepResearchAgent with cross-platform citations
- Channel plugins (iMessage, WhatsApp, Slack) with auto-escalation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 22:52:10 +00:00
Jon Saad-FalconandGitHub c390c73f85 Merge pull request #123 from open-jarvis/feat/gemma-cpp-engine
feat: add gemma.cpp engine via pygemma pybind11 bindings
2026-03-25 14:57:03 -07:00
Jon Saad-FalconandClaude Opus 4.6 d44076db3f fix: add pygemma API comment and model-mismatch warning to stream()
- Document that pygemma v0.1.3 completion() does not accept
  temperature/max_tokens (params accepted for ABC compliance)
- Add model-mismatch warning to stream() for consistency with generate()

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 13:22:13 -07:00
Jon Saad-FalconandClaude Sonnet 4.6 e4d036dd50 test: add live integration test stubs for gemma_cpp engine
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 13:18:13 -07:00
Jon Saad-FalconandClaude Sonnet 4.6 16025571fd style: fix lint issues in gemma_cpp engine
Move mid-file imports to top of test file to resolve E402 violations and apply ruff formatting to both gemma_cpp.py and test_gemma_cpp.py.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 13:17:16 -07:00
Jon Saad-FalconandClaude Sonnet 4.6 d08360ca21 feat: wire gemma_cpp engine into discovery and optional imports
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 13:15:06 -07:00
Jon Saad-FalconandGitHub 15dad86832 Merge pull request #121 from open-jarvis/feat/pinchbench-integration
feat: PinchBench benchmark integration
2026-03-25 12:51:01 -07:00
Jon Saad-FalconandClaude Opus 4.6 ae12d925bb fix(traces): allow TraceStore SQLite access from worker threads
AgenticRunner dispatches _run_body() to a ThreadPoolExecutor when a
task environment is present (for Playwright compatibility). The
TraceStore connection was created on the main thread, causing
"SQLite objects created in a thread can only be used in that same
thread" on the first agentic eval query.

Pass check_same_thread=False to sqlite3.connect(), consistent with
SchedulerStore, AgentManager, SessionStore, and TelemetryStore which
already use this flag.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 12:46:06 -07:00
Jon Saad-FalconandClaude Sonnet 4.6 411e822916 test: add config resolution tests for gemma_cpp engine
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 12:40:24 -07:00
Jon Saad-FalconandClaude Sonnet 4.6 8b0fd702c4 feat: implement gemma_cpp health checks and model listing
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 12:39:38 -07:00
Jon Saad-FalconandClaude Opus 4.6 8c2ee6843a feat: implement gemma_cpp engine lifecycle and inference methods
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 12:38:25 -07:00
Jon Saad-FalconandClaude Opus 4.6 ed47fece39 fix: lint — shorten long lines in CLI and agentic runner
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 12:38:13 -07:00
Jon Saad-FalconandClaude Sonnet 4.6 9cd822611e feat: add GemmaCppEngine skeleton with chat template formatting
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 12:36:35 -07:00
Jon Saad-FalconandClaude Sonnet 4.6 d589bfb55d feat: add GemmaCppEngineConfig and wire into EngineConfig
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 12:34:48 -07:00
Jon Saad-FalconandClaude Opus 4.6 9b511e08f7 fix(evals): workspace path resolution and grading crash resilience
- Store workspace_path in record.metadata from AgenticRunner so task
  envs can reuse the agent's workspace instead of creating a separate
  temp dir the agent can't see
- PinchBenchTaskEnv now uses the existing workspace when available,
  copies fixtures there, and sets CWD so file_read/file_write resolve
  relative paths correctly
- Wrap grading in try/except so LLM judge failures don't crash the
  entire run (graceful degradation to score 0.0)
- Restore CWD on exit and only clean up self-owned workspaces

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 12:33:58 -07:00
Jon Saad-Falcon 3ec7e76215 feat: add inference-gemma optional dependency for pygemma 2026-03-25 12:33:46 -07:00
Jon Saad-FalconandClaude Opus 4.6 784b880130 test(evals): add PinchBench integration tests — full grading pipeline
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 09:31:02 -07:00
Jon Saad-FalconandClaude Sonnet 4.6 b4962482c0 feat(evals): register PinchBench in CLI and add default TOML config
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 09:28:29 -07:00
Jon Saad-Falcon 6ae720baa9 feat(evals): add PinchBench task environment — workspace setup and run_tests() grading 2026-03-25 09:27:18 -07:00
Jon Saad-FalconandClaude Opus 4.6 73cb1f1bbd feat(evals): add PinchBench grading helpers — transcript translation, automated/LLM judge/hybrid scoring
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 09:26:12 -07:00
Jon Saad-FalconandClaude Opus 4.6 8e8d4a68e4 feat(evals): add PinchBench dataset provider — repo clone and task markdown parsing
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 09:25:39 -07:00
Jon Saad-FalconandClaude Opus 4.6 ae0262da82 feat(evals): bridge EventBus to EventRecorder, capture tool_calls in traces
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 09:24:26 -07:00
Jon Saad-FalconandClaude Sonnet 4.6 e0ff20c473 feat(tools): include result text in TOOL_CALL_END event metadata
Adds a truncated (10KB max) string representation of the tool result's
content to the TOOL_CALL_END event payload so grading functions (e.g.
PinchBench) can inspect what each tool returned.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 09:22:13 -07:00
Jon Saad-FalconandClaude Sonnet 4.6 c5cb6a9e49 feat(evals): add tool_calls field to TurnTrace for rich tool data
Adds a `tool_calls` list field to `TurnTrace` capturing name, arguments,
and result for each tool invocation, enabling PinchBench transcript integration.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 09:20:25 -07:00
Jon Saad-FalconandGitHub a8569ec3a3 Merge pull request #120 from open-jarvis/feat/github-contribution-infra
feat: add take-bot workflow and update contribution docs
2026-03-25 08:49:45 -07:00
Jon Saad-FalconandClaude Opus 4.6 36554ab0d1 docs: consolidate README sections and add Contributing
- Merge Quick Start, Docker, and Development into single Quick Start
- Point to docs site for full documentation (Docker, cloud, dev setup)
- Add Contributing section with dev setup and roadmap pointer

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 15:46:12 +00:00
Jon Saad-FalconandGitHub 14c81bba98 Merge pull request #115 from open-jarvis/feat/init-onboarding-and-privacy-scanner
feat: interactive model download in init + privacy environment scanner
2026-03-25 08:12:45 -07:00
Jon Saad-FalconandClaude Opus 4.6 68a8f5646c fix: add --no-download and PrivacyScanner mock to test_cli init test
The test_init_creates_config test in test_cli.py was not updated
when the download prompt and privacy hook were added to jarvis init.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 08:07:35 -07:00
Jon Saad-FalconandClaude Opus 4.6 b2152fb40e style: fix E501 line-length and unused imports in test files
Wraps long CliRunner.invoke() calls, removes unused pytest imports,
fixes import ordering, and removes unused variables in test_scan.py.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 20:38:14 -07:00
Jon Saad-FalconandClaude Opus 4.6 cbf6fe3ea2 feat: add take-bot workflow and update contribution docs
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 03:35:53 +00:00
Jon Saad-FalconandClaude Opus 4.6 bc2a7e5595 chore: remove spec/plan files from tracked tree
These are Claude-generated working documents that should not
be checked into the repository.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 20:31:09 -07:00
Jon Saad-FalconandClaude Opus 4.6 a934977ba8 fix: address spec review findings for privacy scanner
- Add slots=True to ScanResult dataclass
- Fix extra space in IPv6 port f-string
- Add missing tests: check_remote_access, check_icloud_sync, run_quick

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 18:26:05 -07:00
Jon Saad-FalconandClaude Opus 4.6 d9faa4621e style: fix E501 line length in model pull --engine option
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 18:24:07 -07:00
Jon Saad-FalconandClaude Sonnet 4.6 02bc36c1f8 feat: register jarvis scan command and add init privacy hook
Registers the new scan command in the CLI. Adds a lightweight
privacy check at the end of jarvis init that runs disk encryption
and cloud sync checks, with pointer to jarvis scan for full audit.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-24 18:22:13 -07:00
Jon Saad-FalconandGitHub a5949052b6 Merge pull request #111 from open-jarvis/fix/token-counting-no-kv-cache
fix: count full prompt tokens
2026-03-24 18:21:03 -07:00
Jon Saad-FalconandClaude Opus 4.6 e2376d29d1 feat: add interactive model download and empty-model fallback to init
Prompts user to download recommended model during jarvis init.
Adds --no-download flag for CI. Shows helpful message when no
model fits available memory. Adds exo/nexa next-steps text.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 18:16:59 -07:00
Jon Saad-Falcon 00f71ae504 fix: update energy scaling test for linear KV-cache FLOPs model
The FLOPs formula changed from quadratic (P*N*(N+1)) to linear
(2*P*T_evaluated) with KV-cache awareness. Update the test
expectation from ~100x to ~10x for 10x token increase.
2026-03-24 18:16:11 -07:00
Jon Saad-FalconandClaude Opus 4.6 a05c2b7df6 feat: extract ollama_pull helper and add multi-engine model pull
Refactors model pull into reusable ollama_pull() function. Adds
--engine flag to support llamacpp (GGUF) and mlx (HuggingFace)
downloads via huggingface-cli, with FileNotFoundError handling.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 18:12:55 -07:00
Jon Saad-FalconandGitHub 5de72fb54e Merge pull request #112 from open-jarvis/revert/openai-compat-tool-call-fix
revert: remove _fix_tool_call_arguments from OpenAI-compatible engines
2026-03-24 18:12:29 -07:00
Jon Saad-Falcon 26a4e2e65e feat: KV-cache-aware FLOPs/energy with full-count dollar cost
Thread prompt_tokens_evaluated through the telemetry stack so
savings calculations use the right token count for each metric:

- Dollar cost: full prompt_tokens (what cloud providers charge)
- FLOPs/energy: prompt_tokens_evaluated (actual compute with KV
  cache — subsequent turns only re-evaluate new tokens)

Ollama reports both: prompt_eval_count (cache-aware) and we estimate
full count from messages. OpenAI-compat engines report full count
only (KV caching is transparent in their API).

Changes: TelemetryRecord, store schema, aggregator, engines,
savings calculation, /v1/savings route.
2026-03-24 18:12:13 -07:00
Jon Saad-FalconandClaude Opus 4.6 ced38bcba3 feat: add privacy environment scanner with platform-specific checks
New PrivacyScanner class with checks for disk encryption (FileVault/LUKS),
MDM profiles, cloud sync agents, network exposure, and screen recording.
Supports macOS and Linux with graceful skip on missing tools.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 18:07:54 -07:00