Initial commit: project vision, roadmap, and README

Establishes OpenJarvis as a modular AI assistant backend with five
composable pillars (Intelligence, Memory, Agents, Inference, Learning).
Documents the architecture, supported backends, and phased development plan.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

README.md

@@ -0,0 +1,45 @@
+# OpenJarvis
+
+**Your AI stack, your rules.**
+
+A modular, pluggable AI assistant backend. Compose your own stack across five pillars — Intelligence, Learning, Memory, Agents, and Inference — then swap any piece without touching the rest.
+
+> **Status: Early Development** — Interfaces are being defined. Not yet usable.
+
+## What is this?
+
+OpenJarvis lets you build a personal AI assistant from composable parts:
+
+- **Intelligence** — multi-model management with automatic routing (Qwen3, GPT OSS, Kimi-K2.5, Claude, GPT-5, Gemini)
+- **Memory** — persistent, searchable storage with multiple backends (SQLite, FAISS, ColBERTv2, BM25, hybrid)
+- **Agents** — pluggable reasoning and tool use (OpenClaw Pi agent, simple, orchestrator, custom)
+- **Inference** — hardware-aware engine selection (vLLM, SGLang, Ollama, llama.cpp, MLX)
+- **Learning** — router that improves over time (heuristic now, learned later)
+
+## Documentation
+
+- **[VISION.md](VISION.md)** — Project vision, architecture, design principles, and detailed pillar descriptions
+- **[ROADMAP.md](ROADMAP.md)** — Phased development plan with deliverables and version milestones
+
+## Quick orientation
+
+```
+src/openjarvis/
+├── core/          # Registry, types, config, event bus
+├── intelligence/  # Model management, routing
+├── memory/        # Storage backends (SQLite, FAISS, ColBERT, BM25, hybrid)
+├── agents/        # Agent implementations + tool system
+├── engine/        # Inference engine wrappers
+├── learning/      # Router policy (placeholder)
+└── cli/           # CLI entry points (jarvis ask, serve, model, memory)
+```
+
+## Requirements
+
+- Python 3.10+
+- An inference backend: [Ollama](https://ollama.com), [vLLM](https://github.com/vllm-project/vllm), or [llama.cpp](https://github.com/ggerganov/llama.cpp)
+- Node.js 22+ (only if using OpenClaw agent)
+
+## License
+
+TBD

ROADMAP.md

@@ -0,0 +1,345 @@
+# OpenJarvis Roadmap
+
+Phased development plan for OpenJarvis. Phases are ordered to maximize early usability: foundation first, then intelligence + inference (so you can ask questions), then memory (so it remembers), then agents (so it can act), then learning (so it improves).
+
+---
+
+## Phase 0 — Foundation (~2-3 weeks)
+
+**Goal:** Repository scaffolding, core abstractions, and CLI skeleton. Nothing runs yet, but all interfaces are defined.
+
+**Version milestone:** v0.1
+
+### Repository structure
+
+```
+OpenJarvis/
+├── pyproject.toml              # uv/hatchling, all deps + extras
+├── src/
+│   └── openjarvis/
+│       ├── __init__.py
+│       ├── core/
+│       │   ├── registry.py     # RegistryBase[T] + typed registries
+│       │   ├── types.py        # Message, Conversation, ModelSpec, ToolResult, TelemetryRecord
+│       │   ├── config.py       # JarvisConfig dataclass hierarchy, TOML loader
+│       │   └── events.py       # Event bus: pub/sub for inter-pillar telemetry
+│       ├── intelligence/       # Phase 1
+│       ├── memory/             # Phase 2
+│       ├── agents/             # Phase 3
+│       ├── engine/             # Phase 1
+│       ├── learning/           # Phase 4
+│       └── cli/                # CLI entry points
+├── tests/
+├── VISION.md
+├── ROADMAP.md
+└── README.md
+```
+
+### Deliverables
+
+- [ ] **Registry system** — `RegistryBase[T]` adapted from IPW's `registry.py`. Typed subclasses:
+  - `ModelRegistry` — model specs and metadata
+  - `EngineRegistry` — inference engine implementations
+  - `MemoryRegistry` — memory backend implementations
+  - `AgentRegistry` — agent implementations
+  - `ToolRegistry` — tools with `ToolSpec` metadata (category, cost, latency, capabilities)
+
+- [ ] **Core types** (`core/types.py`):
+  - `Message` — role + content + metadata (tool calls, images, etc.)
+  - `Conversation` — ordered list of messages with sliding window support
+  - `ModelSpec` — model ID, parameter count, quantization, context length, hardware compatibility
+  - `ToolResult` — tool name + output + usage + cost
+  - `TelemetryRecord` — timestamp, model, tokens, latency, energy (optional), cost
+
+- [ ] **Config system** (`core/config.py`):
+  - `JarvisConfig` dataclass hierarchy: `EngineConfig`, `IntelligenceConfig`, `MemoryConfig`, `AgentConfig`
+  - TOML config file at `~/.openjarvis/config.toml`
+  - Hardware auto-detection: GPU vendor/model/VRAM/platform → populate defaults
+
+- [ ] **Event bus** (`core/events.py`):
+  - Simple pub/sub for inter-pillar communication
+  - Telemetry events flow without tight coupling between pillars
+  - Synchronous dispatch (async optional later)
+
+- [ ] **CLI skeleton** (Click-based):
+  - `jarvis init` — create `~/.openjarvis/config.toml` with auto-detected defaults
+  - `jarvis ask` — placeholder (wired in Phase 1)
+  - `jarvis serve` — placeholder (wired in Phase 3)
+  - `jarvis model list|pull|info` — placeholder (wired in Phase 1)
+  - `jarvis memory index|search|stats` — placeholder (wired in Phase 2)
+
+---
+
+## Phase 1 — Intelligence + Inference Engine (~3-4 weeks)
+
+**Goal:** You can ask OpenJarvis a question and get an answer. Models run locally or via cloud APIs. Basic telemetry records every call.
+
+**Version milestone:** v0.2 — first usable version
+
+### Inference Engine
+
+- [ ] **`InferenceEngine` ABC:**
+  ```python
+  class InferenceEngine(ABC):
+      def generate(self, model: str, messages: list[Message], **params) -> Response: ...
+      def stream(self, model: str, messages: list[Message], **params) -> Iterator[ResponseChunk]: ...
+      def list_models(self) -> list[ModelSpec]: ...
+      def health(self) -> bool: ...
+  ```
+
+- [ ] **Engine implementations:**
+  - `OllamaEngine` — wraps Ollama HTTP API (`/api/chat`, `/api/tags`). Apple Silicon + NVIDIA.
+  - `VLLMEngine` — wraps vLLM OpenAI-compatible API. Multi-GPU, tensor parallelism.
+  - `LlamaCppEngine` — wraps `llama-cpp-python` or llama.cpp server. Maximum compatibility.
+  - `CloudEngine` — unified wrapper for OpenAI, Anthropic, and Google APIs. Key-based routing.
+
+- [ ] **Model management:**
+  - Auto-discovery from running engines (poll `/api/tags`, `/v1/models`)
+  - `ModelSpec` with hardware compatibility matrix (min VRAM, supported engines, quantization options)
+  - `jarvis model list` shows all available models across engines
+  - `jarvis model info <model>` shows spec, hardware requirements, estimated performance
+
+### Intelligence
+
+- [ ] **Hardware profiles:**
+  - Auto-detect: `nvidia-smi`, `rocm-smi`, `system_profiler` (macOS), `/proc/cpuinfo`
+  - Map GPU to capabilities: VRAM, compute capability, FP8/FP4 support, unified memory
+  - Recommend engine: Apple Silicon → Ollama/MLX, NVIDIA datacenter → vLLM, AMD → vLLM+ROCm, CPU → llama.cpp
+
+- [ ] **Heuristic Router V0:**
+  - Rule-based routing: short queries (< 50 tokens) → small model, complex (reasoning keywords, multi-step) → large model, code patterns → code specialist
+  - Fallback chains: if preferred model unavailable, try next in chain
+  - Configurable via `~/.openjarvis/config.toml`
+
+- [ ] **Basic telemetry:**
+  - Wrap every `generate()` / `stream()` call with timing + token counting
+  - Record to SQLite: model, prompt tokens, completion tokens, latency, cost estimate
+  - `TelemetryRecord` stored via event bus, accumulated for future learning phase
+
+### Wire-up
+
+- [ ] **`jarvis ask "What is X?"` works end-to-end:**
+  1. Parse query → detect complexity → route to model
+  2. Generate response via selected engine
+  3. Record telemetry
+  4. Print response (with optional `--json` output)
+
+---
+
+## Phase 2 — Memory / Storage (~3-4 weeks)
+
+**Goal:** OpenJarvis remembers conversations, can index your documents, and injects relevant context into prompts.
+
+**Version milestone:** v0.3
+
+### Memory backends
+
+- [ ] **`MemoryBackend` ABC:**
+  ```python
+  class MemoryBackend(ABC):
+      def store(self, content: str, metadata: dict) -> str: ...      # Returns doc ID
+      def retrieve(self, query: str, k: int = 10) -> list[Result]: ...
+      def delete(self, doc_id: str) -> bool: ...
+      def clear(self) -> None: ...
+  ```
+
+- [ ] **Memory subtypes:**
+  - `ConversationMemory` — sliding window (configurable size) + automatic summarization of older turns via LLM call
+  - `KnowledgeBase` — indexed document collection with multi-backend search
+
+- [ ] **Backend implementations:**
+
+  - **`SQLiteMemory`** — FTS5 full-text search, zero-config default. Always available, no extra dependencies.
+
+  - **`FAISSMemory`** — Dense neural retrieval. Encodes documents with `sentence-transformers`, builds FAISS index (IVF or flat depending on collection size). GPU-accelerated when available.
+
+  - **`ColBERTMemory`** — ColBERTv2 late interaction retrieval. Best retrieval quality.
+    - Package: `colbert-ai[torch,faiss-gpu]`
+    - Indexing: `Indexer(checkpoint="colbertv2.0", config=ColBERTConfig(nbits=2))` → `indexer.index(name, collection)`
+    - Search: `Searcher(index=name)` → `searcher.search(query, k=10)` returns `(passage_ids, ranks, scores)`
+    - Token-level MaxSim matching: each query token attends to each document token, max-pooled per query token, summed
+    - 2-bit residual compression keeps indexes compact (~50x smaller than full embeddings)
+    - Millisecond query latency, substantially better than single-vector methods on complex queries
+
+  - **`BM25Memory`** — Keyword search baseline using `rank-bm25`. No GPU, no embeddings. Fast and effective for keyword-heavy queries.
+
+  - **`HybridMemory`** — Combines BM25 with a dense backend (FAISS or ColBERT) using Reciprocal Rank Fusion (RRF):
+    ```
+    RRF_score(d) = sum(1 / (k + rank_i(d))) for each retriever i
+    ```
+    Configurable `k` parameter (default 60). Best overall retrieval when you don't know the query type.
+
+### Document pipeline
+
+- [ ] **Indexing pipeline:** PDF / Markdown / plain text / code → chunking (configurable size + overlap) → embedding (if using dense/ColBERT backend) → index
+- [ ] **Context injection:** auto-retrieve top-k relevant chunks before each LLM call, inject into prompt with source attribution (`[Source: filename:line]`)
+
+### CLI
+
+- [ ] `jarvis memory index <path>` — index a file or directory
+- [ ] `jarvis memory search <query>` — search across all memory backends
+- [ ] `jarvis memory stats` — show index sizes, document counts, backend status
+
+---
+
+## Phase 3 — Agentic Logic (~3-4 weeks)
+
+**Goal:** OpenJarvis can use tools, reason over multiple turns, and serve an OpenAI-compatible API. The default agent is OpenClaw's Pi.
+
+**Version milestone:** v0.4
+
+### Agent framework
+
+- [ ] **`BaseAgent` ABC:**
+  ```python
+  class BaseAgent(ABC):
+      def run(self, input: str, context: AgentContext) -> AgentResult: ...
+  ```
+  `AgentContext` carries: conversation history, memory handle, tool registry, telemetry recorder, model router.
+  `AgentResult` contains: response text, tool calls made, tokens used, telemetry data.
+
+- [ ] **Agent implementations:**
+
+  - **`OpenClawAgent`** (default) — wraps OpenClaw's Pi agent runtime (`@mariozechner/pi-coding-agent` v0.52.12+). Two modes:
+    1. **HTTP mode:** OpenClaw gateway running locally on `:18789`. Communicate via WebSocket. Best for persistent sessions.
+    2. **Subprocess mode:** invoke `node` with `runEmbeddedPiAgent()` call, JSON over stdin/stdout. No gateway needed.
+    - Capabilities: multi-turn reasoning, tool calling, streaming, skill composition, context compaction
+    - Requires Node.js 22+
+
+  - **`SimpleAgent`** — single-turn: parse query → call model → return response. No tool calling, no multi-turn. Works without Node.js. Good for testing and simple Q&A.
+
+  - **`OrchestratorAgent`** — multi-turn with model selection per step. Adapted from IPW's executor pattern. Each reasoning step can route to a different model (e.g., fast model for planning, large model for synthesis).
+
+  - **`CustomAgent`** — template class for user-defined agents. Subclass `BaseAgent`, implement `run()`, register with `@AgentRegistry.register("my-agent")`.
+
+### Tool system
+
+- [ ] **`BaseTool` ABC:**
+  ```python
+  class BaseTool(ABC):
+      name: str
+      spec: ToolSpec  # category, cost_estimate, latency_estimate, capabilities
+      def execute(self, input: str, **params) -> ToolResult: ...
+  ```
+
+- [ ] **Built-in tools:**
+  - `Calculator` — evaluate math expressions
+  - `WebSearch` — search the web (Tavily, SearXNG, or DuckDuckGo)
+  - `CodeInterpreter` — execute Python in sandboxed environment
+  - `FileRead` / `FileWrite` — local file operations
+  - `Think` — internal reasoning scratchpad (zero-cost tool for chain-of-thought)
+  - `Retrieval` — wired to memory backends, returns relevant documents
+  - `LLMTool` — call another LLM as a tool (for model composition)
+
+- [ ] **`ToolRegistry`** with discovery:
+  - `ToolSpec` metadata: category, estimated latency, estimated cost, required API keys, capabilities list
+  - Auto-discover available tools based on installed packages and environment
+
+### API server
+
+- [ ] **OpenAI-compatible API server:**
+  - `POST /v1/chat/completions` — standard chat completion with tool use
+  - `GET /v1/models` — list available models
+  - Streaming via Server-Sent Events (SSE)
+  - `jarvis serve --port 8000 --agent openclaw`
+
+### CLI
+
+- [ ] `jarvis serve --port 8000 --agent <agent>` — start API server
+- [ ] `jarvis ask` now supports `--agent <agent>` flag
+
+---
+
+## Phase 4 — Learning Approach (placeholder)
+
+**Goal:** Stub interfaces for the learned router. No ML training in this phase — just the contracts and telemetry plumbing so everything is ready when we build it.
+
+**Version milestone:** v0.5
+
+### Stubs
+
+- [ ] **`RouterPolicy` ABC:**
+  ```python
+  class RouterPolicy(ABC):
+      def select_model(self, query: str, context: RoutingContext) -> ModelSpec: ...
+  ```
+  The heuristic router from Phase 1 implements this as the default.
+
+- [ ] **`RewardFunction` ABC:**
+  ```python
+  class RewardFunction(ABC):
+      def compute(self, trajectory: Trajectory) -> float: ...
+  ```
+  Placeholder implementations: `QualityReward` (LLM-judge), `LatencyReward` (inverse latency), `EnergyReward` (inverse energy), `CostReward` (inverse cost), `CompositeReward` (weighted combination).
+
+- [ ] **`TelemetryAggregator`:**
+  - Reads `TelemetryRecord` entries from SQLite (accumulated since Phase 1)
+  - Computes per-model statistics: average latency, token throughput, cost, quality (when graded)
+  - Exports training-ready datasets for the future GRPO pipeline
+
+- [ ] **Design document:** `docs/learning-pipeline.md` describing the planned GRPO training pipeline:
+  - Trajectory generation from Phases 1-3 telemetry
+  - Reward model training
+  - Policy optimization with GRPO
+  - Online evaluation and rollout strategy
+
+---
+
+## Phase 5 — Integration & Polish (~3-4 weeks)
+
+**Goal:** Production-ready packaging, OpenClaw integration, benchmarking, SDK, and documentation.
+
+**Version milestone:** v1.0
+
+### OpenClaw integration
+
+- [ ] **`openjarvis-openclaw` plugin package:**
+  - `register()` hook implementing OpenClaw's plugin API
+  - `registerProvider()` — wraps OpenJarvis as an OpenClaw `ProviderPlugin` (routes through OpenJarvis intelligence + engine)
+  - `registerTool()` — exposes OpenJarvis tools to OpenClaw
+  - `MemorySearchManager` — implements OpenClaw's `search()` / `sync()` / `status()` interface, backed by OpenJarvis memory
+
+### Deployment
+
+- [ ] **Dockerfile** — multi-stage build with optional GPU support
+- [ ] **docker-compose.yml** — OpenJarvis + Ollama/vLLM + optional gateway
+- [ ] **Service files** — systemd (Linux) and launchd (macOS) for running as a system service
+
+### Python SDK
+
+- [ ] **Programmatic API:**
+  ```python
+  from openjarvis import Jarvis
+
+  j = Jarvis()                                        # Auto-loads config
+  response = await j.ask("Explain transformers")      # Uses router + engine
+  await j.memory.index("~/papers/")                   # Index documents
+  results = await j.memory.search("attention mechanism")
+  ```
+
+### Benchmarking
+
+- [ ] **`jarvis bench` CLI:**
+  - `BaseBenchmark` / `DatasetBenchmark` ABCs (adapted from IPW's `BenchmarkSuite`)
+  - Run benchmarks across models, measure accuracy + latency + energy
+  - Output JSONL results + summary JSON
+
+### Documentation
+
+- [ ] Documentation site (MkDocs or similar)
+- [ ] Getting started guide
+- [ ] Plugin development guide
+- [ ] API reference
+
+---
+
+## Version Summary
+
+| Version | Phase | What you get |
+|---------|-------|-------------|
+| **v0.1** | Phase 0 | Scaffolding, registries, config, CLI skeleton |
+| **v0.2** | Phase 1 | `jarvis ask` works — local & cloud inference with telemetry |
+| **v0.3** | Phase 2 | Memory — index docs, conversation history, context injection |
+| **v0.4** | Phase 3 | Agents + tools + OpenAI-compatible API server |
+| **v0.5** | Phase 4 | Learning stubs — router policy interface, telemetry aggregation |
+| **v1.0** | Phase 5 | Production — SDK, OpenClaw plugin, Docker, benchmarks, docs |

VISION.md

@@ -0,0 +1,299 @@
+# OpenJarvis
+
+**Your AI stack, your rules.**
+
+OpenJarvis is a modular, pluggable AI assistant backend. Instead of locking you into one model, one memory system, or one inference engine, OpenJarvis lets you compose your own stack across five pillars — then swap any piece without touching the rest.
+
+Built for developers who want full control and researchers who need reproducible, measurable AI systems.
+
+---
+
+## The Five Pillars
+
+OpenJarvis is organized around five composable pillars. Each pillar defines a clear interface; implementations are discovered at runtime via a decorator-based registry system.
+
+```
+┌─────────────────────────────────────────────────────────────────────┐
+│                          OpenJarvis                                 │
+│                                                                     │
+│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐                │
+│  │ Intelligence │  │  Learning   │  │   Memory /  │                │
+│  │   (Models)   │  │  Approach   │  │   Storage   │                │
+│  │             │  │  (Router)   │  │             │                │
+│  └──────┬──────┘  └──────┬──────┘  └──────┬──────┘                │
+│         │                │                │                        │
+│         ▼                ▼                ▼                        │
+│  ┌──────────────────────────────────────────────┐                  │
+│  │              Agentic Logic                    │                  │
+│  │     (Orchestration, Tools, Reasoning)         │                  │
+│  └──────────────────────┬───────────────────────┘                  │
+│                         │                                          │
+│                         ▼                                          │
+│  ┌──────────────────────────────────────────────┐                  │
+│  │            Inference Engine                   │                  │
+│  │    (vLLM, Ollama, llama.cpp, SGLang, MLX)     │                  │
+│  └──────────────────────────────────────────────┘                  │
+└─────────────────────────────────────────────────────────────────────┘
+```
+
+---
+
+### 1. Intelligence (Model Layer)
+
+**What it does:** Manages available language models — local and cloud — and routes queries to the best model for the task.
+
+**What's pluggable:** Models, model providers, routing heuristics.
+
+**Supported at launch:**
+
+| Category | Models |
+|----------|--------|
+| Open-source (local) | Qwen3 8B, Qwen3 32B, GPT OSS 120B, Kimi-K2.5, MiniMax-M2.5 |
+| Cloud APIs | Claude (Anthropic), GPT-4o / GPT-5 (OpenAI), Gemini (Google) |
+
+**Key components:**
+
+- **`ModelRegistry`** — decorator-based registry mapping model keys to `ModelSpec` objects (parameter count, quantization, hardware compatibility, context length)
+- **Heuristic Router (V0)** — rule-based routing: short queries → small model, complex reasoning → large model, code → code specialist, fallback chains for unavailable models
+- **Auto-discovery** — detects models from running inference engines (Ollama, vLLM) and available API keys
+
+---
+
+### 2. Learning Approach (Router Policy)
+
+**What it does:** Determines *which* model handles a given query. V1 is heuristic; future versions will learn from usage data.
+
+**What's pluggable:** Routing policy, reward functions, training pipeline.
+
+**V1 (Placeholder):**
+- Heuristic routing based on query characteristics (length, complexity keywords, domain detection)
+- Rule-based fallback chains
+- All telemetry logged to SQLite for future training
+
+**Future (Post-V1):**
+- Learned router via GRPO (Group Relative Policy Optimization)
+- Preference learning from user feedback
+- Continual fine-tuning on accumulated trajectories
+- Multi-objective optimization: quality vs. latency vs. energy vs. cost
+
+---
+
+### 3. Memory / Storage
+
+**What it does:** Provides persistent, searchable memory across conversations, documents, and personal notes. Memory is automatically injected into prompts with source attribution.
+
+**What's pluggable:** Storage backends, retrieval strategies, embedding models, chunking strategies.
+
+**Memory types:**
+- **Conversation Memory** — sliding window with automatic summarization of older turns
+- **Knowledge Base** — indexed documents (PDF, Markdown, code, text) with multi-backend search
+- **Personal Notes** — user-created persistent notes and preferences
+- **Episodic Memory** — records of past interactions, tool uses, and outcomes
+
+**Backend implementations:**
+
+| Backend | Type | Description |
+|---------|------|-------------|
+| **SQLite** (default) | Keyword + FTS | FTS5 full-text search. Zero dependencies, zero config. Always available. |
+| **FAISS** | Dense retrieval | Neural semantic search via `sentence-transformers` + FAISS indexes. |
+| **ColBERTv2** | Late interaction | Token-level MaxSim matching with 2-bit residual compression. Best retrieval quality. Uses `colbert-ai` package with `Indexer` for offline indexing and `Searcher` for millisecond-latency queries. |
+| **BM25** | Sparse retrieval | Classic keyword search baseline. Fast, no GPU needed. |
+| **Hybrid** | Fusion | BM25 + dense (or ColBERT) with Reciprocal Rank Fusion (RRF). Best of both worlds. |
+| **Vector DB adapters** | Dense retrieval | Qdrant, ChromaDB connectors for users with existing vector infrastructure. |
+
+**ColBERTv2 details:**
+- Late interaction model: queries and documents are encoded independently, then matched at the token level via MaxSim
+- 2-bit residual compression keeps indexes compact while preserving quality
+- Offline indexing via `Indexer(checkpoint="colbertv2.0", config=ColBERTConfig(nbits=2))`
+- Millisecond query latency via `Searcher(index=name).search(query, k=10)`
+- Substantially better retrieval quality than single-vector dense methods on complex queries
+
+---
+
+### 4. Agentic Logic
+
+**What it does:** Orchestrates multi-turn reasoning, tool calling, and task execution. The agent layer sits between the user and the model, managing context, tools, and conversation flow.
+
+**What's pluggable:** Agent implementations, tools, tool registries, execution strategies.
+
+**Agent implementations:**
+
+| Agent | Description |
+|-------|-------------|
+| **`OpenClawAgent`** (default) | Wraps OpenClaw's Pi agent runtime. Multi-turn reasoning, tool calling, streaming responses, skill composition, context compaction. Two modes: **HTTP** (WebSocket to OpenClaw gateway on `:18789`) or **subprocess** (invoke `node` with `runEmbeddedPiAgent()`, JSON over stdin/stdout). Requires Node.js 22+. |
+| **`SimpleAgent`** | Single-turn: query → model → response. No tool calling. Works without Node.js. Good for quick answers and testing. |
+| **`OrchestratorAgent`** | Multi-turn with per-step model selection. Adapted from IPW's executor pattern. Routes each reasoning step to the optimal model. |
+| **`CustomAgent`** | Template for user-defined agent logic. Subclass `BaseAgent`, implement `run()`, register with `AgentRegistry`. |
+
+**Tool system:**
+- `BaseTool` ABC with `ToolSpec` metadata (category, cost estimate, latency estimate, capabilities)
+- `ToolRegistry` — runtime-discoverable tool catalog
+- Built-in tools: Calculator, WebSearch, CodeInterpreter, FileRead/Write, Think, Retrieval (wired to memory backends), LLM-as-tool
+- MCP (Model Context Protocol) compatible
+
+**API server:**
+- OpenAI-compatible `/v1/chat/completions` and `/v1/models` endpoints
+- Streaming via Server-Sent Events (SSE)
+- Drop-in replacement for any OpenAI-compatible client
+
+---
+
+### 5. Inference Engine
+
+**What it does:** Manages the actual LLM inference runtime — loading models, generating tokens, managing GPU memory.
+
+**What's pluggable:** Engine backends, hardware profiles, quantization strategies.
+
+**Supported engines:**
+
+| Engine | Best for | GPU | CPU |
+|--------|----------|-----|-----|
+| **vLLM** | High-throughput server, multi-GPU, production | NVIDIA, AMD | — |
+| **SGLang** | Structured generation, constrained decoding | NVIDIA, AMD | — |
+| **Ollama** | Easy setup, Apple Silicon, single-model | NVIDIA, Apple | Yes |
+| **llama.cpp** | Maximum hardware compatibility, GGUF models | NVIDIA, AMD, Apple | Yes |
+| **MLX** | Apple Silicon native, Metal acceleration | Apple | Apple |
+
+**Hardware auto-detection:**
+- Detects GPU vendor (NVIDIA/AMD/Apple), model, VRAM, compute capability
+- Recommends the best engine for detected hardware
+- Apple Silicon → Ollama or MLX; NVIDIA datacenter → vLLM; AMD → vLLM with ROCm; CPU-only → llama.cpp
+
+---
+
+## Query Flow
+
+```
+User query
+    │
+    ▼
+┌──────────┐     ┌──────────┐     ┌──────────┐
+│  Agentic │────▶│ Memory   │────▶│ Context  │
+│  Logic   │     │ Retrieve │     │ Inject   │
+└────┬─────┘     └──────────┘     └────┬─────┘
+     │                                  │
+     ▼                                  ▼
+┌──────────┐     ┌──────────┐     ┌──────────┐
+│ Learning │────▶│  Model   │────▶│ Inference│
+│ (Router) │     │ Select   │     │ Engine   │
+└──────────┘     └──────────┘     └────┬─────┘
+                                       │
+                                       ▼
+                                  ┌──────────┐
+                                  │ Response │
+                                  │ + Telem. │
+                                  └──────────┘
+```
+
+1. **Agentic Logic** receives the user query, determines if tools or memory are needed
+2. **Memory** retrieves relevant context (conversation history, documents, notes)
+3. **Context Injection** assembles the full prompt with retrieved content and source attribution
+4. **Learning/Router** selects the best model for this query based on routing policy
+5. **Inference Engine** runs the selected model and streams the response
+6. **Telemetry** records timing, token counts, energy (if available), and cost
+
+---
+
+## User Scenarios
+
+### Developer on M4 Max MacBook Pro (128 GB unified memory)
+
+```toml
+# ~/.openjarvis/config.toml
+[engine]
+backend = "ollama"          # Native Apple Silicon support
+
+[intelligence]
+default_model = "qwen3-32b" # Fits in 128 GB unified memory
+fallback = "qwen3-8b"
+
+[memory]
+backend = "sqlite"          # Zero-config, always works
+retrieval = "hybrid"        # BM25 + FAISS for local docs
+
+[agent]
+type = "openclaw"           # Full agent capabilities
+mode = "subprocess"         # No separate gateway needed
+```
+
+Day-to-day: codes with `jarvis ask`, indexes project docs with `jarvis memory index`, runs a local OpenAI-compatible server with `jarvis serve` for editor integration.
+
+### Researcher on DGX Spark (2x B200, 384 GB GPU memory)
+
+```toml
+[engine]
+backend = "vllm"
+tensor_parallel = 2
+
+[intelligence]
+default_model = "qwen3-235b-a22b"
+router = "heuristic"        # Route small queries to 8B, large to 235B
+
+[memory]
+backend = "colbert"         # Best retrieval quality for papers
+knowledge_base = "~/papers/"
+
+[agent]
+type = "orchestrator"       # Multi-model orchestration
+```
+
+Running benchmarks with `jarvis bench`, profiling energy per query, comparing model efficiency across hardware configurations.
+
+### Privacy-Focused Offline Setup
+
+```toml
+[engine]
+backend = "llamacpp"        # No server needed
+network = "offline"
+
+[intelligence]
+default_model = "qwen3-8b-q4"  # Quantized to fit available RAM
+
+[memory]
+backend = "sqlite"          # Everything local
+retrieval = "bm25"          # No neural models needed
+
+[agent]
+type = "simple"             # No external dependencies
+```
+
+Fully air-gapped. No cloud APIs, no network calls, no telemetry export. All data stays on the machine.
+
+---
+
+## Comparison
+
+| Feature | OpenJarvis | Ollama | LangChain | OpenClaw | vLLM |
+|---------|-----------|--------|-----------|----------|------|
+| **Focus** | Composable AI backend | Model runner | LLM app framework | AI coding assistant | Inference server |
+| **Model management** | Multi-engine, auto-detect | Single engine | Bring your own | Cloud-first | Single engine |
+| **Memory** | Multi-backend retrieval | None | Vector store wrappers | Conversation only | None |
+| **Agents** | Pluggable (Pi, custom) | None | Chain-based | Pi agent (built-in) | None |
+| **Inference** | vLLM/SGLang/Ollama/llama.cpp/MLX | Ollama only | External | External | vLLM only |
+| **Hardware-aware** | Auto-detect + recommend | Manual | No | No | Manual |
+| **Telemetry** | Energy, latency, cost | None | Callbacks | Basic | Metrics |
+| **Offline** | Full support | Full support | Partial | No | Full support |
+| **API** | OpenAI-compatible | OpenAI-compatible | Custom | Custom | OpenAI-compatible |
+| **Language** | Python | Go | Python | TypeScript | Python |
+
+OpenJarvis is **not** a replacement for these tools — it *composes* them. Ollama and vLLM are inference engine options. OpenClaw's Pi agent is the default agentic logic. LangChain-style chains can be implemented as custom agents.
+
+---
+
+## Design Principles
+
+1. **Pluggable everything** — every component is registered and discoverable at runtime. Swap models, engines, memory backends, and agents without code changes.
+
+2. **Registry-driven** — `RegistryBase[T]` pattern (adapted from IPW) provides type-safe, decorator-based registration for all extensible components: `ModelRegistry`, `EngineRegistry`, `MemoryRegistry`, `AgentRegistry`, `ToolRegistry`.
+
+3. **Offline-first** — works without network access. Cloud APIs are optional enhancements, never requirements.
+
+4. **Telemetry-native** — every inference call records timing, token counts, and (when hardware supports it) energy consumption. Data lands in SQLite for analysis.
+
+5. **Hardware-aware** — auto-detects GPU vendor, model, VRAM, and platform. Recommends the best engine and model configuration for your hardware.
+
+6. **Python-first** — core is pure Python (3.10+). Node.js required only for OpenClaw agent integration. No Java, no JVM, no heavy runtimes.
+
+7. **OpenAI-compatible API** — `jarvis serve` exposes `/v1/chat/completions` and `/v1/models`. Any client that speaks OpenAI protocol works out of the box.
+
+8. **Standalone** — OpenJarvis is a self-contained backend. OpenClaw is one possible frontend; so is `curl`, a Python SDK call, or any OpenAI-compatible client.