mirror of
https://github.com/open-jarvis/OpenJarvis.git
synced 2026-07-30 02:42:16 +00:00
Add MkDocs Material documentation site with 40 pages and auto-generated API reference
Sets up a complete documentation website with 7 navigable sections (Home, Getting Started, User Guide, Architecture, API Reference, Deployment, Development), light/dark mode, search, code copy, and Mermaid diagram support. API reference pages use mkdocstrings to auto-generate docs from source docstrings. GitHub Actions workflow deploys to GitHub Pages on push to main. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Jon Saad-Falcon
co-authored by
Claude Opus 4.6
Claude Opus 4.6
parent
990d7d8a79
commit
f75afefcfb
@@ -0,0 +1,227 @@
|
||||
# Changelog
|
||||
|
||||
All notable changes to OpenJarvis are documented in this file.
|
||||
|
||||
---
|
||||
|
||||
## v1.0.0
|
||||
|
||||
*Phase 5 -- SDK, Production Readiness, and Documentation*
|
||||
|
||||
### Added
|
||||
|
||||
- **Python SDK** -- `Jarvis` class providing a high-level sync API for
|
||||
programmatic use
|
||||
- `ask()` / `ask_full()` methods for direct engine and agent mode queries
|
||||
- `MemoryHandle` proxy for lazy memory backend initialization
|
||||
- `list_models()` and `list_engines()` for runtime introspection
|
||||
- Router policy selection via config (`learning.default_policy`)
|
||||
- Lazy engine initialization with automatic discovery and health probing
|
||||
- Resource cleanup via `close()`
|
||||
- **OpenClaw agent infrastructure**
|
||||
- `OpenClawAgent` with HTTP and subprocess transports
|
||||
- `ProtocolMessage` dataclass with JSON-line serialization/deserialization
|
||||
- `MessageType` enum for structured agent communication
|
||||
- `HttpTransport` for HTTP POST-based communication with OpenClaw servers
|
||||
- `SubprocessTransport` for Node.js stdin/stdout communication
|
||||
- `ProviderPlugin` wrapping inference engines for OpenClaw
|
||||
- `MemorySearchManager` wrapping memory backends for OpenClaw
|
||||
- **Benchmarking framework**
|
||||
- `BaseBenchmark` ABC and `BenchmarkSuite` runner
|
||||
- `LatencyBenchmark` measuring per-call latency (mean, p50, p95, min, max)
|
||||
- `ThroughputBenchmark` measuring tokens-per-second throughput
|
||||
- `BenchmarkResult` dataclass with JSONL export
|
||||
- `jarvis bench run` CLI with options for model, engine, sample count,
|
||||
benchmark selection, and JSON/JSONL output
|
||||
- **Docker deployment**
|
||||
- `Dockerfile` -- Multi-stage Python 3.12-slim build with `[server]` extra
|
||||
- `Dockerfile.gpu` -- NVIDIA CUDA 12.4 runtime variant
|
||||
- `docker-compose.yml` -- Services for `jarvis` (port 8000) and `ollama`
|
||||
(port 11434)
|
||||
- `deploy/systemd/openjarvis.service` -- systemd unit file for Linux
|
||||
- `deploy/launchd/com.openjarvis.plist` -- launchd plist for macOS
|
||||
- **Documentation site** -- MkDocs Material with mkdocstrings, covering
|
||||
getting started, user guide, architecture, API reference, deployment, and
|
||||
development
|
||||
|
||||
---
|
||||
|
||||
## v0.5.0
|
||||
|
||||
*Phase 4 -- Learning, Telemetry, and Router Policies*
|
||||
|
||||
### Added
|
||||
|
||||
- **Learning system**
|
||||
- `RouterPolicy` ABC and `RoutingContext` dataclass
|
||||
- `RewardFunction` ABC for scoring inference results
|
||||
- `HeuristicRewardFunction` scoring on latency, cost, and efficiency
|
||||
- `RouterPolicyRegistry` for pluggable routing strategies
|
||||
- `HeuristicRouter` registered as `"heuristic"` policy (6 priority rules:
|
||||
code detection, math detection, short/long queries, urgency override,
|
||||
default fallback)
|
||||
- `TraceDrivenPolicy` registered as `"learned"` policy with batch updates
|
||||
via `update_from_traces()` and online updates via `observe()`
|
||||
- `GRPORouterPolicy` stub registered as `"grpo"` for future RL training
|
||||
- `ensure_registered()` pattern for lazy, test-safe registration
|
||||
- **Telemetry aggregation**
|
||||
- `TelemetryAggregator` with `per_model_stats()`, `per_engine_stats()`,
|
||||
`top_models()`, `summary()`, `export_records()`, and `clear()` methods
|
||||
- Time-range filtering via `since` / `until` parameters
|
||||
- `ModelStats` and `EngineStats` dataclasses
|
||||
- `AggregatedStats` summary dataclass
|
||||
- **CLI enhancements**
|
||||
- `--router` flag on `jarvis ask` for explicit policy selection
|
||||
- `jarvis telemetry stats` -- display aggregated telemetry statistics
|
||||
- `jarvis telemetry export --format json|csv` -- export telemetry records
|
||||
- `jarvis telemetry clear --yes` -- delete all telemetry records
|
||||
|
||||
---
|
||||
|
||||
## v0.4.0
|
||||
|
||||
*Phase 3 -- Agents, Tools, and API Server*
|
||||
|
||||
### Added
|
||||
|
||||
- **Agent system**
|
||||
- `BaseAgent` ABC with `run()` method returning `AgentResult`
|
||||
- `AgentContext` dataclass with conversation, tools, and memory results
|
||||
- `AgentResult` dataclass with content, tool results, turns, and metadata
|
||||
- `AgentRegistry` for pluggable agent implementations
|
||||
- `SimpleAgent` -- single-turn query-to-response, no tool calling
|
||||
- `OrchestratorAgent` -- multi-turn tool-calling loop with `ToolExecutor`,
|
||||
configurable `max_turns`
|
||||
- `CustomAgent` -- template for user-defined agent behavior
|
||||
- `OpenClawAgent` -- transport-based agent with tool-call loop and event
|
||||
bus integration
|
||||
- **Tool system**
|
||||
- `BaseTool` ABC with `spec` property and `execute()` method
|
||||
- `ToolSpec` dataclass describing tool interface and characteristics
|
||||
- `ToolExecutor` dispatch engine with JSON argument parsing, latency
|
||||
tracking, and event bus integration (`TOOL_CALL_START` / `TOOL_CALL_END`)
|
||||
- `ToolRegistry` for tool discovery
|
||||
- `to_openai_function()` method for OpenAI function calling format
|
||||
- Built-in tools:
|
||||
- `CalculatorTool` -- safe math evaluation via AST parsing
|
||||
- `ThinkTool` -- reasoning scratchpad for chain-of-thought
|
||||
- `RetrievalTool` -- memory search integration
|
||||
- `LLMTool` -- sub-model calls within agent loops
|
||||
- `FileReadTool` -- safe file reading with path validation
|
||||
- **OpenAI-compatible API server** (`jarvis serve`)
|
||||
- FastAPI + Uvicorn with optional `[server]` extra
|
||||
- `POST /v1/chat/completions` -- non-streaming and SSE streaming
|
||||
- `GET /v1/models` -- list available models
|
||||
- `GET /health` -- health check endpoint
|
||||
- Pydantic request/response models matching OpenAI API format
|
||||
|
||||
---
|
||||
|
||||
## v0.3.0
|
||||
|
||||
*Phase 2 -- Memory System*
|
||||
|
||||
### Added
|
||||
|
||||
- **Memory backends**
|
||||
- `MemoryBackend` ABC with `store()`, `retrieve()`, `delete()`, `clear()`
|
||||
- `RetrievalResult` dataclass with content, score, source, and metadata
|
||||
- `MemoryRegistry` for backend discovery
|
||||
- `SQLiteMemory` -- zero-dependency default using SQLite FTS5 with BM25
|
||||
ranking and FTS5 query escaping
|
||||
- `FAISSMemory` -- vector search using FAISS with sentence-transformers
|
||||
embeddings (optional `[memory-faiss]` extra)
|
||||
- `ColBERTMemory` -- ColBERTv2 neural retrieval backend (optional
|
||||
`[memory-colbert]` extra)
|
||||
- `BM25Memory` -- BM25 ranking backend using rank-bm25 (optional
|
||||
`[memory-bm25]` extra)
|
||||
- `HybridMemory` -- Reciprocal Rank Fusion combining multiple backends
|
||||
- **Document processing**
|
||||
- `ChunkConfig` dataclass for chunk size and overlap settings
|
||||
- `chunk_text()` for splitting documents into overlapping chunks
|
||||
- `ingest_path()` for recursively indexing files and directories
|
||||
- `read_document()` with support for plain text, Markdown, and PDF
|
||||
(optional `[memory-pdf]` extra)
|
||||
- **Context injection**
|
||||
- `ContextConfig` with top-k, minimum score, and max context token settings
|
||||
- `inject_context()` for prepending memory results as system messages with
|
||||
source attribution
|
||||
- `--no-context` flag on `jarvis ask` to disable injection
|
||||
- **CLI commands**
|
||||
- `jarvis memory index <path>` -- index documents into memory
|
||||
- `jarvis memory search <query>` -- search memory for relevant chunks
|
||||
- `jarvis memory stats` -- show backend statistics
|
||||
- **Event bus integration** -- `MEMORY_STORE` and `MEMORY_RETRIEVE` events
|
||||
|
||||
---
|
||||
|
||||
## v0.2.0
|
||||
|
||||
*Phase 1 -- Intelligence and Inference*
|
||||
|
||||
### Added
|
||||
|
||||
- **Intelligence pillar**
|
||||
- `ModelSpec` dataclass with parameter count, context length, quantization,
|
||||
VRAM requirements, and supported engines
|
||||
- `ModelRegistry` for model metadata storage
|
||||
- `BUILTIN_MODELS` catalog with pre-defined model specifications
|
||||
- `register_builtin_models()` and `merge_discovered_models()` helpers
|
||||
- `HeuristicRouter` with rule-based model selection
|
||||
- `build_routing_context()` for query analysis (code detection, math
|
||||
detection, length classification)
|
||||
- **Inference engines**
|
||||
- `InferenceEngine` ABC with `generate()`, `stream()`, `list_models()`,
|
||||
and `health()` methods
|
||||
- `EngineRegistry` for engine discovery
|
||||
- `OllamaEngine` -- Ollama backend via native HTTP API with tool call
|
||||
extraction
|
||||
- `VllmEngine` -- vLLM backend via OpenAI-compatible API
|
||||
- `LlamaCppEngine` -- llama.cpp server backend
|
||||
- `EngineConnectionError` for unreachable engines
|
||||
- `messages_to_dicts()` for Message-to-OpenAI-format conversion
|
||||
- **Engine discovery**
|
||||
- `discover_engines()` -- probe all registered engines for health
|
||||
- `discover_models()` -- aggregate model lists across engines
|
||||
- `get_engine()` -- get configured default with automatic fallback
|
||||
- **Hardware detection**
|
||||
- NVIDIA GPU detection via `nvidia-smi`
|
||||
- AMD GPU detection via `rocm-smi`
|
||||
- Apple Silicon detection via `system_profiler`
|
||||
- CPU brand detection via `/proc/cpuinfo` and `sysctl`
|
||||
- `recommend_engine()` mapping hardware to best engine
|
||||
- **Telemetry**
|
||||
- `TelemetryRecord` dataclass with timing, tokens, energy, and cost
|
||||
- `TelemetryStore` with SQLite persistence and EventBus subscription
|
||||
- `instrumented_generate()` wrapper for automatic telemetry recording
|
||||
- **CLI**
|
||||
- `jarvis ask <query>` -- query via discovered engine
|
||||
- `jarvis ask --agent simple <query>` -- route through SimpleAgent
|
||||
- `jarvis model list` -- list models from running engines
|
||||
- `jarvis model info <model>` -- show model details
|
||||
|
||||
---
|
||||
|
||||
## v0.1.0
|
||||
|
||||
*Phase 0 -- Project Scaffolding*
|
||||
|
||||
### Added
|
||||
|
||||
- **Project structure** -- `hatchling` build backend, `uv` package manager,
|
||||
`pyproject.toml` with extras for optional backends
|
||||
- **Registry system** -- `RegistryBase[T]` generic base class with
|
||||
class-specific entry isolation, `register()` decorator, `get()`, `create()`,
|
||||
`items()`, `keys()`, `contains()`, `clear()` methods
|
||||
- **Typed registries** -- `ModelRegistry`, `EngineRegistry`, `MemoryRegistry`,
|
||||
`AgentRegistry`, `ToolRegistry`, `RouterPolicyRegistry`, `BenchmarkRegistry`
|
||||
- **Core types** -- `Role` enum, `Message`, `Conversation` (with sliding
|
||||
window), `ModelSpec`, `Quantization` enum, `ToolCall`, `ToolResult`,
|
||||
`TelemetryRecord`, `StepType` enum, `TraceStep`, `Trace`
|
||||
- **Configuration** -- `JarvisConfig` dataclass hierarchy, TOML loader with
|
||||
overlay semantics, hardware auto-detection, `generate_default_toml()` for
|
||||
`jarvis init`
|
||||
- **Event bus** -- Synchronous pub/sub `EventBus` with `EventType` enum for
|
||||
inter-pillar communication
|
||||
- **CLI skeleton** -- Click-based `jarvis` command group with `--version`,
|
||||
`--help`, and `init` subcommand
|
||||
@@ -0,0 +1,417 @@
|
||||
# Contributing Guide
|
||||
|
||||
This guide covers how to set up a development environment, run tests, and
|
||||
contribute code to OpenJarvis.
|
||||
|
||||
---
|
||||
|
||||
## Development Setup
|
||||
|
||||
### Prerequisites
|
||||
|
||||
| Requirement | Version | Notes |
|
||||
|---|---|---|
|
||||
| Python | 3.10+ | Required |
|
||||
| [uv](https://docs.astral.sh/uv/) | Latest | Package manager |
|
||||
| Node.js | 22+ | Only needed for OpenClaw agent |
|
||||
|
||||
### Clone and Install
|
||||
|
||||
```bash
|
||||
git clone https://github.com/jonsaadfalcon/OpenJarvis.git
|
||||
cd OpenJarvis
|
||||
uv sync --extra dev
|
||||
```
|
||||
|
||||
This installs the package in editable mode along with all development
|
||||
dependencies (pytest, ruff, respx, pytest-asyncio, pytest-cov).
|
||||
|
||||
!!! tip "Optional extras"
|
||||
Install additional extras for specific backends you want to work on:
|
||||
|
||||
```bash
|
||||
# Memory backends
|
||||
uv sync --extra dev --extra memory-faiss --extra memory-colbert --extra memory-bm25
|
||||
|
||||
# Cloud inference
|
||||
uv sync --extra dev --extra inference-cloud --extra inference-google
|
||||
|
||||
# API server
|
||||
uv sync --extra dev --extra server
|
||||
|
||||
# Documentation
|
||||
uv sync --extra dev --extra docs
|
||||
```
|
||||
|
||||
### Verify Installation
|
||||
|
||||
```bash
|
||||
uv run jarvis --version # Should print 1.0.0
|
||||
uv run jarvis --help # Show all subcommands
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Running Tests
|
||||
|
||||
OpenJarvis uses [pytest](https://docs.pytest.org/) with approximately 1,000+
|
||||
tests organized by module.
|
||||
|
||||
### Full Test Suite
|
||||
|
||||
```bash
|
||||
uv run pytest tests/ -v
|
||||
```
|
||||
|
||||
### Run a Specific Test File
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_registry.py -v
|
||||
uv run pytest tests/engine/test_ollama.py -v
|
||||
uv run pytest tests/memory/test_sqlite.py -v
|
||||
```
|
||||
|
||||
### Run a Specific Test
|
||||
|
||||
```bash
|
||||
uv run pytest tests/core/test_registry.py::test_register_and_get -v
|
||||
```
|
||||
|
||||
### Run Tests by Module
|
||||
|
||||
```bash
|
||||
uv run pytest tests/agents/ -v # All agent tests
|
||||
uv run pytest tests/tools/ -v # All tool tests
|
||||
uv run pytest tests/learning/ -v # All learning tests
|
||||
```
|
||||
|
||||
### Test Coverage
|
||||
|
||||
```bash
|
||||
uv run pytest tests/ --cov=openjarvis --cov-report=html
|
||||
```
|
||||
|
||||
### Test Markers
|
||||
|
||||
Tests that require specific hardware or running services are gated behind
|
||||
pytest markers. By default, these tests are collected but will skip
|
||||
gracefully if the requirement is not met.
|
||||
|
||||
| Marker | Description | Example |
|
||||
|---|---|---|
|
||||
| `live` | Requires a running inference engine (Ollama, vLLM, etc.) | `@pytest.mark.live` |
|
||||
| `cloud` | Requires cloud API keys (`OPENAI_API_KEY`, etc.) | `@pytest.mark.cloud` |
|
||||
| `nvidia` | Requires an NVIDIA GPU | `@pytest.mark.nvidia` |
|
||||
| `amd` | Requires an AMD GPU with ROCm | `@pytest.mark.amd` |
|
||||
| `apple` | Requires Apple Silicon | `@pytest.mark.apple` |
|
||||
| `slow` | Long-running test | `@pytest.mark.slow` |
|
||||
|
||||
Run only tests matching a specific marker:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/ -m live -v # Only live engine tests
|
||||
uv run pytest tests/ -m "not slow" -v # Skip slow tests
|
||||
uv run pytest tests/ -m "not cloud" -v # Skip cloud tests
|
||||
```
|
||||
|
||||
!!! info "Registry isolation in tests"
|
||||
The test `conftest.py` includes an `autouse` fixture that clears all
|
||||
registries and resets the event bus before every test. This ensures
|
||||
complete isolation between tests. Modules that need their registrations
|
||||
to survive clearing use the `ensure_registered()` pattern described
|
||||
below.
|
||||
|
||||
---
|
||||
|
||||
## Linting
|
||||
|
||||
OpenJarvis uses [Ruff](https://docs.astral.sh/ruff/) for linting, configured
|
||||
in `pyproject.toml`:
|
||||
|
||||
```bash
|
||||
uv run ruff check src/ tests/
|
||||
```
|
||||
|
||||
The Ruff configuration targets Python 3.10 and enables the following rule sets:
|
||||
|
||||
- **E** -- pycodestyle errors
|
||||
- **F** -- Pyflakes
|
||||
- **I** -- isort (import ordering)
|
||||
- **W** -- pycodestyle warnings
|
||||
|
||||
Fix auto-fixable issues:
|
||||
|
||||
```bash
|
||||
uv run ruff check src/ tests/ --fix
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Building Documentation
|
||||
|
||||
The documentation site uses [MkDocs Material](https://squidfunnel.com/mkdocs-material/).
|
||||
|
||||
```bash
|
||||
# Install docs dependencies
|
||||
uv sync --extra docs
|
||||
|
||||
# Serve locally with hot reload
|
||||
uv run mkdocs serve --dev-addr 127.0.0.1:8001
|
||||
|
||||
# Build static site
|
||||
uv run mkdocs build
|
||||
```
|
||||
|
||||
The site configuration lives in `mkdocs.yml`. API reference pages use
|
||||
[mkdocstrings](https://mkdocstrings.github.io/) to auto-generate from
|
||||
docstrings with the NumPy docstring style.
|
||||
|
||||
---
|
||||
|
||||
## Project Structure
|
||||
|
||||
The source code is organized under `src/openjarvis/`:
|
||||
|
||||
```
|
||||
src/openjarvis/
|
||||
__init__.py # Package root, __version__
|
||||
sdk.py # Jarvis class — high-level Python SDK
|
||||
|
||||
core/ # Shared infrastructure
|
||||
config.py # JarvisConfig, hardware detection, TOML loader
|
||||
events.py # EventBus pub/sub system
|
||||
registry.py # RegistryBase[T] and all typed registries
|
||||
types.py # Message, ModelSpec, ToolResult, Trace, etc.
|
||||
|
||||
intelligence/ # Model management and query routing
|
||||
model_catalog.py # BUILTIN_MODELS, register/merge helpers
|
||||
router.py # HeuristicRouter, build_routing_context
|
||||
|
||||
engine/ # Inference engine backends
|
||||
_stubs.py # InferenceEngine ABC
|
||||
_base.py # EngineConnectionError, messages_to_dicts
|
||||
_discovery.py # discover_engines, discover_models, get_engine
|
||||
_openai_compat.py # OpenAI-compatible wrapper
|
||||
ollama.py # OllamaEngine
|
||||
vllm.py # VllmEngine
|
||||
llamacpp.py # LlamaCppEngine
|
||||
sglang.py # SGLangEngine
|
||||
cloud.py # CloudEngine (OpenAI/Anthropic/Google)
|
||||
|
||||
agents/ # Agent implementations
|
||||
_stubs.py # BaseAgent ABC, AgentContext, AgentResult
|
||||
simple.py # SimpleAgent — single-turn, no tools
|
||||
orchestrator.py # OrchestratorAgent — multi-turn tool calling
|
||||
custom.py # CustomAgent — user template
|
||||
react.py # ReActAgent
|
||||
openhands.py # OpenHands agent
|
||||
openclaw.py # OpenClawAgent — HTTP/subprocess transport
|
||||
openclaw_protocol.py # OpenClaw message protocol
|
||||
openclaw_transport.py # OpenClaw transports (HTTP, subprocess)
|
||||
openclaw_plugin.py # OpenClaw provider/memory plugins
|
||||
|
||||
memory/ # Memory / retrieval backends
|
||||
_stubs.py # MemoryBackend ABC, RetrievalResult
|
||||
sqlite.py # SQLiteMemory — FTS5 default backend
|
||||
faiss_backend.py # FAISS vector backend
|
||||
colbert_backend.py # ColBERTv2 backend
|
||||
bm25.py # BM25 backend
|
||||
hybrid.py # Hybrid (RRF fusion) backend
|
||||
chunking.py # ChunkConfig, chunk_text
|
||||
context.py # ContextConfig, inject_context
|
||||
ingest.py # ingest_path, read_document
|
||||
|
||||
tools/ # Tool system
|
||||
_stubs.py # BaseTool ABC, ToolSpec, ToolExecutor
|
||||
calculator.py # CalculatorTool — safe AST math
|
||||
think.py # ThinkTool — reasoning scratchpad
|
||||
retrieval.py # RetrievalTool — memory search
|
||||
llm_tool.py # LLMTool — sub-model calls
|
||||
file_read.py # FileReadTool — safe file reading
|
||||
web_search.py # WebSearchTool
|
||||
code_interpreter.py # CodeInterpreterTool
|
||||
|
||||
learning/ # Router policies and reward functions
|
||||
_stubs.py # RouterPolicy ABC, RewardFunction ABC
|
||||
heuristic_policy.py # Wire HeuristicRouter to registry
|
||||
trace_policy.py # TraceDrivenPolicy — learns from traces
|
||||
grpo_policy.py # GRPORouterPolicy — RL training stub
|
||||
heuristic_reward.py # HeuristicRewardFunction
|
||||
|
||||
traces/ # Full interaction recording
|
||||
store.py # TraceStore — SQLite persistence
|
||||
collector.py # TraceCollector — wraps agents
|
||||
analyzer.py # TraceAnalyzer — aggregated queries
|
||||
|
||||
telemetry/ # Inference telemetry
|
||||
store.py # TelemetryStore — SQLite persistence
|
||||
aggregator.py # TelemetryAggregator — per-model/engine stats
|
||||
wrapper.py # instrumented_generate() wrapper
|
||||
|
||||
bench/ # Benchmarking framework
|
||||
_stubs.py # BaseBenchmark ABC, BenchmarkSuite
|
||||
latency.py # LatencyBenchmark
|
||||
throughput.py # ThroughputBenchmark
|
||||
|
||||
server/ # OpenAI-compatible API server
|
||||
app.py # FastAPI application factory
|
||||
routes.py # /v1/chat/completions, /v1/models, /health
|
||||
|
||||
mcp/ # MCP (Model Context Protocol) layer
|
||||
|
||||
cli/ # Click CLI commands
|
||||
__init__.py # main group
|
||||
ask.py # jarvis ask
|
||||
init_cmd.py # jarvis init
|
||||
model.py # jarvis model list/info
|
||||
memory_cmd.py # jarvis memory index/search/stats
|
||||
telemetry_cmd.py # jarvis telemetry stats/export/clear
|
||||
bench_cmd.py # jarvis bench run
|
||||
serve.py # jarvis serve
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Code Conventions
|
||||
|
||||
### File Naming
|
||||
|
||||
| Pattern | Purpose | Examples |
|
||||
|---|---|---|
|
||||
| `_stubs.py` | ABC definitions and dataclasses | `engine/_stubs.py`, `agents/_stubs.py`, `tools/_stubs.py` |
|
||||
| `_discovery.py` | Auto-detection and probing logic | `engine/_discovery.py` |
|
||||
| `_base.py` | Shared utilities and re-exports | `engine/_base.py` |
|
||||
| `*_cmd.py` | CLI command modules | `init_cmd.py`, `memory_cmd.py`, `bench_cmd.py` |
|
||||
|
||||
### Registry Pattern
|
||||
|
||||
All extensible components use the decorator-based registry pattern. New
|
||||
implementations are added by decorating a class -- no factory modifications
|
||||
needed:
|
||||
|
||||
```python
|
||||
from openjarvis.core.registry import EngineRegistry
|
||||
|
||||
@EngineRegistry.register("my_engine")
|
||||
class MyEngine(InferenceEngine):
|
||||
...
|
||||
```
|
||||
|
||||
Available registries:
|
||||
|
||||
| Registry | Stores | Key examples |
|
||||
|---|---|---|
|
||||
| `ModelRegistry` | `ModelSpec` objects | `"qwen3:8b"`, `"llama3.1:70b"` |
|
||||
| `EngineRegistry` | `InferenceEngine` classes | `"ollama"`, `"vllm"`, `"llamacpp"` |
|
||||
| `MemoryRegistry` | `MemoryBackend` classes | `"sqlite"`, `"faiss"`, `"bm25"` |
|
||||
| `AgentRegistry` | `BaseAgent` classes | `"simple"`, `"orchestrator"` |
|
||||
| `ToolRegistry` | `BaseTool` classes | `"calculator"`, `"think"`, `"retrieval"` |
|
||||
| `RouterPolicyRegistry` | `RouterPolicy` classes | `"heuristic"`, `"learned"` |
|
||||
| `BenchmarkRegistry` | `BaseBenchmark` classes | `"latency"`, `"throughput"` |
|
||||
|
||||
### Optional Dependencies
|
||||
|
||||
Backends that depend on optional packages use the `try/except ImportError`
|
||||
pattern to fail gracefully when deps are not installed:
|
||||
|
||||
```python
|
||||
# In __init__.py — import to trigger registration
|
||||
try:
|
||||
import openjarvis.memory.faiss_backend # noqa: F401
|
||||
except ImportError:
|
||||
pass
|
||||
```
|
||||
|
||||
This ensures the package always loads, even if `faiss-cpu` or other optional
|
||||
dependencies are not installed.
|
||||
|
||||
### The `ensure_registered()` Pattern
|
||||
|
||||
Benchmark and learning modules use lazy registration so that their entries
|
||||
survive registry clearing in tests:
|
||||
|
||||
```python
|
||||
def ensure_registered() -> None:
|
||||
"""Register the latency benchmark if not already present."""
|
||||
if not BenchmarkRegistry.contains("latency"):
|
||||
BenchmarkRegistry.register_value("latency", LatencyBenchmark)
|
||||
```
|
||||
|
||||
This pattern checks `contains()` before registering, making it safe to call
|
||||
multiple times without raising a duplicate-key error.
|
||||
|
||||
### Dataclass Conventions
|
||||
|
||||
- Use `slots=True` on all dataclasses for memory efficiency:
|
||||
|
||||
```python
|
||||
@dataclass(slots=True)
|
||||
class BenchmarkResult:
|
||||
benchmark_name: str
|
||||
model: str
|
||||
...
|
||||
```
|
||||
|
||||
### Type Hints
|
||||
|
||||
- All function signatures must have type annotations
|
||||
- Use `from __future__ import annotations` at the top of every module
|
||||
- Use `Optional[X]` for nullable types
|
||||
- Use `Sequence` for read-only collections, `List` for mutable ones
|
||||
|
||||
### Import Style
|
||||
|
||||
- Absolute imports only (`from openjarvis.core.registry import ...`)
|
||||
- Sort imports with `ruff` (isort rules enabled)
|
||||
- Place `from __future__ import annotations` as the first import
|
||||
|
||||
---
|
||||
|
||||
## PR Guidelines
|
||||
|
||||
### Before Submitting
|
||||
|
||||
1. **Run the full test suite** and verify no regressions:
|
||||
```bash
|
||||
uv run pytest tests/ -v
|
||||
```
|
||||
|
||||
2. **Run the linter** and fix all issues:
|
||||
```bash
|
||||
uv run ruff check src/ tests/
|
||||
```
|
||||
|
||||
3. **Add tests** for new functionality. Place them in the corresponding
|
||||
`tests/` subdirectory (e.g., new engine tests go in `tests/engine/`).
|
||||
|
||||
4. **Follow the registry pattern** for any new extensible component.
|
||||
|
||||
### Commit Messages
|
||||
|
||||
- Use the imperative mood (e.g., "Add FAISS memory backend")
|
||||
- Keep the first line under 72 characters
|
||||
- Reference relevant issues or PRs
|
||||
|
||||
### What Makes a Good PR
|
||||
|
||||
- **Focused**: One feature, fix, or refactor per PR
|
||||
- **Tested**: Include unit tests that cover the new code paths
|
||||
- **Documented**: Update docstrings and documentation pages if adding
|
||||
public API
|
||||
- **Backwards compatible**: Avoid breaking existing interfaces without
|
||||
discussion
|
||||
|
||||
### Adding a New Pillar Component
|
||||
|
||||
When adding a new engine, memory backend, agent, tool, benchmark, or router
|
||||
policy:
|
||||
|
||||
1. Implement the corresponding ABC
|
||||
2. Register with the appropriate `@XRegistry.register("key")` decorator
|
||||
3. Add an import in the module's `__init__.py` (with `try/except ImportError`
|
||||
if the component has optional deps)
|
||||
4. Add tests in the matching `tests/` subdirectory
|
||||
5. Add an entry in `pyproject.toml` under `[project.optional-dependencies]`
|
||||
if the component requires new packages
|
||||
|
||||
See the [Extending OpenJarvis](extending.md) guide for complete examples.
|
||||
@@ -0,0 +1,855 @@
|
||||
# Extending OpenJarvis
|
||||
|
||||
OpenJarvis is designed to be extended through its registry pattern. Every
|
||||
major subsystem defines an abstract base class (ABC) and uses a typed registry
|
||||
for runtime discovery. To add a new component, implement the ABC, decorate
|
||||
it with the registry, and import it in the module's `__init__.py`.
|
||||
|
||||
This guide provides complete, working code examples for each extension point.
|
||||
|
||||
---
|
||||
|
||||
## Adding a New Inference Engine
|
||||
|
||||
Inference engines connect OpenJarvis to an LLM runtime. All engines implement
|
||||
the `InferenceEngine` ABC defined in `engine/_stubs.py`.
|
||||
|
||||
### Step 1: Create the Engine Module
|
||||
|
||||
Create `src/openjarvis/engine/my_engine.py`:
|
||||
|
||||
```python
|
||||
"""My custom inference engine backend."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import AsyncIterator, Sequence
|
||||
from typing import Any, Dict, List
|
||||
|
||||
import httpx
|
||||
|
||||
from openjarvis.core.registry import EngineRegistry
|
||||
from openjarvis.core.types import Message
|
||||
from openjarvis.engine._base import (
|
||||
EngineConnectionError,
|
||||
InferenceEngine,
|
||||
messages_to_dicts,
|
||||
)
|
||||
|
||||
|
||||
@EngineRegistry.register("my_engine") # (1)!
|
||||
class MyEngine(InferenceEngine):
|
||||
"""Custom inference engine backend."""
|
||||
|
||||
engine_id = "my_engine" # (2)!
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
host: str = "http://localhost:9000",
|
||||
*,
|
||||
timeout: float = 120.0,
|
||||
) -> None:
|
||||
self._host = host.rstrip("/")
|
||||
self._client = httpx.Client(base_url=self._host, timeout=timeout)
|
||||
|
||||
def generate(
|
||||
self,
|
||||
messages: Sequence[Message],
|
||||
*,
|
||||
model: str,
|
||||
temperature: float = 0.7,
|
||||
max_tokens: int = 1024,
|
||||
**kwargs: Any,
|
||||
) -> Dict[str, Any]:
|
||||
"""Synchronous completion."""
|
||||
payload = {
|
||||
"model": model,
|
||||
"messages": messages_to_dicts(messages), # (3)!
|
||||
"temperature": temperature,
|
||||
"max_tokens": max_tokens,
|
||||
}
|
||||
# Pass tools if provided
|
||||
tools = kwargs.get("tools")
|
||||
if tools:
|
||||
payload["tools"] = tools
|
||||
|
||||
try:
|
||||
resp = self._client.post("/v1/chat/completions", json=payload)
|
||||
resp.raise_for_status()
|
||||
except (httpx.ConnectError, httpx.TimeoutException) as exc:
|
||||
raise EngineConnectionError(
|
||||
f"Engine not reachable at {self._host}"
|
||||
) from exc
|
||||
|
||||
data = resp.json()
|
||||
choice = data.get("choices", [{}])[0]
|
||||
message = choice.get("message", {})
|
||||
usage = data.get("usage", {})
|
||||
|
||||
result: Dict[str, Any] = {
|
||||
"content": message.get("content", ""),
|
||||
"usage": {
|
||||
"prompt_tokens": usage.get("prompt_tokens", 0),
|
||||
"completion_tokens": usage.get("completion_tokens", 0),
|
||||
"total_tokens": usage.get("total_tokens", 0),
|
||||
},
|
||||
"model": data.get("model", model),
|
||||
"finish_reason": choice.get("finish_reason", "stop"),
|
||||
}
|
||||
|
||||
# Extract tool calls if present
|
||||
raw_tool_calls = message.get("tool_calls", [])
|
||||
if raw_tool_calls:
|
||||
result["tool_calls"] = [
|
||||
{
|
||||
"id": tc.get("id", f"call_{i}"),
|
||||
"name": tc["function"]["name"],
|
||||
"arguments": tc["function"]["arguments"],
|
||||
}
|
||||
for i, tc in enumerate(raw_tool_calls)
|
||||
]
|
||||
return result
|
||||
|
||||
async def stream(
|
||||
self,
|
||||
messages: Sequence[Message],
|
||||
*,
|
||||
model: str,
|
||||
temperature: float = 0.7,
|
||||
max_tokens: int = 1024,
|
||||
**kwargs: Any,
|
||||
) -> AsyncIterator[str]:
|
||||
"""Yield token strings as they are generated."""
|
||||
# Implement SSE or WebSocket streaming for your engine
|
||||
result = self.generate(
|
||||
messages, model=model, temperature=temperature,
|
||||
max_tokens=max_tokens, **kwargs,
|
||||
)
|
||||
yield result.get("content", "")
|
||||
|
||||
def list_models(self) -> List[str]:
|
||||
"""Return identifiers of models available on this engine."""
|
||||
try:
|
||||
resp = self._client.get("/v1/models")
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
return [m["id"] for m in data.get("data", [])]
|
||||
except Exception:
|
||||
return []
|
||||
|
||||
def health(self) -> bool:
|
||||
"""Return True when the engine is reachable and healthy."""
|
||||
try:
|
||||
resp = self._client.get("/health", timeout=2.0)
|
||||
return resp.status_code == 200
|
||||
except Exception:
|
||||
return False
|
||||
```
|
||||
|
||||
1. The `@EngineRegistry.register("my_engine")` decorator makes this engine
|
||||
discoverable by key at runtime.
|
||||
2. The `engine_id` class attribute is used in telemetry and benchmark results.
|
||||
3. `messages_to_dicts()` converts `Message` objects to OpenAI-format dicts.
|
||||
|
||||
### Step 2: Register in `__init__.py`
|
||||
|
||||
Add your engine import to `src/openjarvis/engine/__init__.py`:
|
||||
|
||||
```python
|
||||
import openjarvis.engine.my_engine # noqa: F401
|
||||
```
|
||||
|
||||
If your engine requires optional dependencies, wrap the import:
|
||||
|
||||
```python
|
||||
try:
|
||||
import openjarvis.engine.my_engine # noqa: F401
|
||||
except ImportError:
|
||||
pass
|
||||
```
|
||||
|
||||
### Step 3: Add Optional Dependencies
|
||||
|
||||
If your engine needs extra packages, add them to `pyproject.toml`:
|
||||
|
||||
```toml
|
||||
[project.optional-dependencies]
|
||||
inference-myengine = [
|
||||
"my-engine-sdk>=1.0",
|
||||
]
|
||||
```
|
||||
|
||||
### Required ABC Methods
|
||||
|
||||
| Method | Signature | Returns | Description |
|
||||
|---|---|---|---|
|
||||
| `generate` | `(messages, *, model, temperature, max_tokens, **kwargs)` | `Dict[str, Any]` | Synchronous completion with `content` and `usage` keys |
|
||||
| `stream` | `(messages, *, model, temperature, max_tokens, **kwargs)` | `AsyncIterator[str]` | Yields token strings as they are generated |
|
||||
| `list_models` | `()` | `List[str]` | Model identifiers available on this engine |
|
||||
| `health` | `()` | `bool` | `True` when the engine is reachable |
|
||||
|
||||
The `generate` return dict must include at minimum:
|
||||
|
||||
```python
|
||||
{
|
||||
"content": "The response text",
|
||||
"usage": {
|
||||
"prompt_tokens": 10,
|
||||
"completion_tokens": 20,
|
||||
"total_tokens": 30,
|
||||
},
|
||||
"model": "model-name",
|
||||
"finish_reason": "stop", # or "tool_calls"
|
||||
}
|
||||
```
|
||||
|
||||
!!! tip "Tool call support"
|
||||
If your engine supports tool/function calling, include a `"tool_calls"`
|
||||
key in the return dict. Each tool call should have `id`, `name`, and
|
||||
`arguments` (JSON string) keys.
|
||||
|
||||
---
|
||||
|
||||
## Adding a New Memory Backend
|
||||
|
||||
Memory backends provide persistent, searchable storage. All backends implement
|
||||
the `MemoryBackend` ABC defined in `memory/_stubs.py`.
|
||||
|
||||
### Complete Example
|
||||
|
||||
Create `src/openjarvis/memory/my_backend.py`:
|
||||
|
||||
```python
|
||||
"""Custom memory backend example."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
from openjarvis.core.registry import MemoryRegistry
|
||||
from openjarvis.memory._stubs import MemoryBackend, RetrievalResult
|
||||
|
||||
|
||||
@MemoryRegistry.register("my_backend")
|
||||
class MyMemoryBackend(MemoryBackend):
|
||||
"""Custom memory backend implementation."""
|
||||
|
||||
backend_id = "my_backend"
|
||||
|
||||
def __init__(self, **kwargs: Any) -> None:
|
||||
# Initialize your storage (database, index, etc.)
|
||||
self._store: Dict[str, Dict[str, Any]] = {}
|
||||
|
||||
def store(
|
||||
self,
|
||||
content: str,
|
||||
*,
|
||||
source: str = "",
|
||||
metadata: Optional[Dict[str, Any]] = None,
|
||||
) -> str:
|
||||
"""Persist content and return a unique document id."""
|
||||
import uuid
|
||||
|
||||
doc_id = uuid.uuid4().hex
|
||||
self._store[doc_id] = {
|
||||
"content": content,
|
||||
"source": source,
|
||||
"metadata": metadata or {},
|
||||
}
|
||||
return doc_id
|
||||
|
||||
def retrieve(
|
||||
self,
|
||||
query: str,
|
||||
*,
|
||||
top_k: int = 5,
|
||||
**kwargs: Any,
|
||||
) -> List[RetrievalResult]:
|
||||
"""Search for query and return the top-k results."""
|
||||
results: List[RetrievalResult] = []
|
||||
for doc_id, doc in self._store.items():
|
||||
# Implement your search/ranking logic here
|
||||
if query.lower() in doc["content"].lower():
|
||||
results.append(RetrievalResult(
|
||||
content=doc["content"],
|
||||
score=1.0,
|
||||
source=doc["source"],
|
||||
metadata=doc["metadata"],
|
||||
))
|
||||
return results[:top_k]
|
||||
|
||||
def delete(self, doc_id: str) -> bool:
|
||||
"""Delete a document by id. Return True if it existed."""
|
||||
return self._store.pop(doc_id, None) is not None
|
||||
|
||||
def clear(self) -> None:
|
||||
"""Remove all stored documents."""
|
||||
self._store.clear()
|
||||
```
|
||||
|
||||
### Register in `__init__.py`
|
||||
|
||||
Add to `src/openjarvis/memory/__init__.py`:
|
||||
|
||||
```python
|
||||
try:
|
||||
import openjarvis.memory.my_backend # noqa: F401
|
||||
except ImportError:
|
||||
pass
|
||||
```
|
||||
|
||||
### Required ABC Methods
|
||||
|
||||
| Method | Signature | Returns | Description |
|
||||
|---|---|---|---|
|
||||
| `store` | `(content, *, source, metadata)` | `str` | Persist content, return document ID |
|
||||
| `retrieve` | `(query, *, top_k, **kwargs)` | `List[RetrievalResult]` | Search and return ranked results |
|
||||
| `delete` | `(doc_id)` | `bool` | Delete by ID, return whether it existed |
|
||||
| `clear` | `()` | `None` | Remove all stored documents |
|
||||
|
||||
The `RetrievalResult` dataclass has these fields:
|
||||
|
||||
```python
|
||||
@dataclass(slots=True)
|
||||
class RetrievalResult:
|
||||
content: str # The retrieved text
|
||||
score: float = 0.0 # Relevance score
|
||||
source: str = "" # Source identifier
|
||||
metadata: Dict[str, Any] = field(default_factory=dict)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Adding a New Agent
|
||||
|
||||
Agents implement the logic for handling queries, calling tools, and managing
|
||||
multi-turn interactions. All agents implement the `BaseAgent` ABC from
|
||||
`agents/_stubs.py`.
|
||||
|
||||
### Complete Example
|
||||
|
||||
Create `src/openjarvis/agents/my_agent.py`:
|
||||
|
||||
```python
|
||||
"""Custom agent implementation."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any, Optional
|
||||
|
||||
from openjarvis.agents._stubs import AgentContext, AgentResult, BaseAgent
|
||||
from openjarvis.core.events import EventBus, EventType
|
||||
from openjarvis.core.registry import AgentRegistry
|
||||
from openjarvis.core.types import Message, Role
|
||||
from openjarvis.engine._stubs import InferenceEngine
|
||||
from openjarvis.telemetry.wrapper import instrumented_generate
|
||||
|
||||
|
||||
@AgentRegistry.register("my_agent")
|
||||
class MyAgent(BaseAgent):
|
||||
"""Custom agent with specialized behavior."""
|
||||
|
||||
agent_id = "my_agent"
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
engine: InferenceEngine,
|
||||
model: str,
|
||||
*,
|
||||
bus: Optional[EventBus] = None,
|
||||
temperature: float = 0.7,
|
||||
max_tokens: int = 1024,
|
||||
) -> None:
|
||||
self._engine = engine
|
||||
self._model = model
|
||||
self._bus = bus
|
||||
self._temperature = temperature
|
||||
self._max_tokens = max_tokens
|
||||
|
||||
def run(
|
||||
self,
|
||||
input: str,
|
||||
context: Optional[AgentContext] = None,
|
||||
**kwargs: Any,
|
||||
) -> AgentResult:
|
||||
"""Execute the agent on input and return an AgentResult."""
|
||||
# Emit turn start event
|
||||
if self._bus:
|
||||
self._bus.publish(EventType.AGENT_TURN_START, {
|
||||
"agent": self.agent_id,
|
||||
"input": input,
|
||||
})
|
||||
|
||||
# Build messages from context + user input
|
||||
messages: list[Message] = []
|
||||
|
||||
# Add a system prompt for your agent's personality
|
||||
messages.append(Message(
|
||||
role=Role.SYSTEM,
|
||||
content="You are a helpful assistant with specialized knowledge.",
|
||||
))
|
||||
|
||||
# Include any prior conversation from context
|
||||
if context and context.conversation.messages:
|
||||
messages.extend(context.conversation.messages)
|
||||
|
||||
messages.append(Message(role=Role.USER, content=input))
|
||||
|
||||
# Generate via instrumented path for telemetry
|
||||
if self._bus:
|
||||
result = instrumented_generate(
|
||||
self._engine,
|
||||
messages,
|
||||
model=self._model,
|
||||
bus=self._bus,
|
||||
temperature=self._temperature,
|
||||
max_tokens=self._max_tokens,
|
||||
)
|
||||
else:
|
||||
result = self._engine.generate(
|
||||
messages,
|
||||
model=self._model,
|
||||
temperature=self._temperature,
|
||||
max_tokens=self._max_tokens,
|
||||
)
|
||||
|
||||
content = result.get("content", "")
|
||||
|
||||
# Emit turn end event
|
||||
if self._bus:
|
||||
self._bus.publish(EventType.AGENT_TURN_END, {
|
||||
"agent": self.agent_id,
|
||||
"content_length": len(content),
|
||||
})
|
||||
|
||||
return AgentResult(content=content, turns=1)
|
||||
```
|
||||
|
||||
### Register in `__init__.py`
|
||||
|
||||
Add to `src/openjarvis/agents/__init__.py`:
|
||||
|
||||
```python
|
||||
try:
|
||||
import openjarvis.agents.my_agent # noqa: F401
|
||||
except ImportError:
|
||||
pass
|
||||
```
|
||||
|
||||
### Key Types
|
||||
|
||||
=== "AgentContext"
|
||||
|
||||
```python
|
||||
@dataclass(slots=True)
|
||||
class AgentContext:
|
||||
conversation: Conversation = field(default_factory=Conversation)
|
||||
tools: List[str] = field(default_factory=list)
|
||||
memory_results: List[Any] = field(default_factory=list)
|
||||
metadata: Dict[str, Any] = field(default_factory=dict)
|
||||
```
|
||||
|
||||
=== "AgentResult"
|
||||
|
||||
```python
|
||||
@dataclass(slots=True)
|
||||
class AgentResult:
|
||||
content: str
|
||||
tool_results: List[ToolResult] = field(default_factory=list)
|
||||
turns: int = 0
|
||||
metadata: Dict[str, Any] = field(default_factory=dict)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Adding a New Tool
|
||||
|
||||
Tools are callable capabilities that agents can invoke during multi-turn
|
||||
reasoning. All tools implement the `BaseTool` ABC from `tools/_stubs.py`.
|
||||
|
||||
### Complete Example
|
||||
|
||||
Create `src/openjarvis/tools/my_tool.py`:
|
||||
|
||||
```python
|
||||
"""Custom tool implementation."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from openjarvis.core.registry import ToolRegistry
|
||||
from openjarvis.core.types import ToolResult
|
||||
from openjarvis.tools._stubs import BaseTool, ToolSpec
|
||||
|
||||
|
||||
@ToolRegistry.register("my_tool")
|
||||
class MyTool(BaseTool):
|
||||
"""A custom tool that does something useful."""
|
||||
|
||||
tool_id = "my_tool"
|
||||
|
||||
@property
|
||||
def spec(self) -> ToolSpec:
|
||||
"""Return the tool specification."""
|
||||
return ToolSpec(
|
||||
name="my_tool",
|
||||
description="Does something useful with the provided input.",
|
||||
parameters={ # (1)!
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"query": {
|
||||
"type": "string",
|
||||
"description": "The input to process",
|
||||
},
|
||||
"max_results": {
|
||||
"type": "integer",
|
||||
"description": "Maximum number of results to return",
|
||||
"default": 5,
|
||||
},
|
||||
},
|
||||
"required": ["query"],
|
||||
},
|
||||
category="utility",
|
||||
cost_estimate=0.001, # Estimated cost in USD per call
|
||||
latency_estimate=0.5, # Estimated latency in seconds
|
||||
requires_confirmation=False,
|
||||
)
|
||||
|
||||
def execute(self, **params: Any) -> ToolResult:
|
||||
"""Execute the tool with the given parameters."""
|
||||
query = params.get("query", "")
|
||||
max_results = params.get("max_results", 5)
|
||||
|
||||
if not query:
|
||||
return ToolResult(
|
||||
tool_name="my_tool",
|
||||
content="No query provided.",
|
||||
success=False,
|
||||
)
|
||||
|
||||
try:
|
||||
# Your tool logic here
|
||||
result_text = f"Processed '{query}' (max_results={max_results})"
|
||||
|
||||
return ToolResult(
|
||||
tool_name="my_tool",
|
||||
content=result_text,
|
||||
success=True,
|
||||
)
|
||||
except Exception as exc:
|
||||
return ToolResult(
|
||||
tool_name="my_tool",
|
||||
content=f"Error: {exc}",
|
||||
success=False,
|
||||
)
|
||||
```
|
||||
|
||||
1. The `parameters` dict follows the [JSON Schema](https://json-schema.org/)
|
||||
format used by OpenAI function calling. The `ToolExecutor` will parse
|
||||
incoming JSON arguments and pass them as keyword arguments to `execute()`.
|
||||
|
||||
### Register in `__init__.py`
|
||||
|
||||
Add to `src/openjarvis/tools/__init__.py`:
|
||||
|
||||
```python
|
||||
try:
|
||||
import openjarvis.tools.my_tool # noqa: F401
|
||||
except ImportError:
|
||||
pass
|
||||
```
|
||||
|
||||
### How Tools Are Invoked
|
||||
|
||||
The `ToolExecutor` handles the dispatch loop:
|
||||
|
||||
1. The agent's LLM generates a `tool_calls` response with tool name and
|
||||
JSON arguments
|
||||
2. `ToolExecutor.execute()` parses the JSON arguments
|
||||
3. The matching tool's `execute(**params)` is called
|
||||
4. The `ToolResult` is returned to the agent for the next turn
|
||||
|
||||
```python
|
||||
from openjarvis.tools._stubs import ToolExecutor
|
||||
|
||||
executor = ToolExecutor(
|
||||
tools=[MyTool()],
|
||||
bus=event_bus, # Optional — enables TOOL_CALL_START/END events
|
||||
)
|
||||
|
||||
# Dispatch a tool call
|
||||
from openjarvis.core.types import ToolCall
|
||||
|
||||
call = ToolCall(id="call_1", name="my_tool", arguments='{"query": "test"}')
|
||||
result = executor.execute(call)
|
||||
```
|
||||
|
||||
The `to_openai_function()` method converts a tool's spec to OpenAI function
|
||||
calling format, which is sent to the LLM alongside the conversation:
|
||||
|
||||
```python
|
||||
tool = MyTool()
|
||||
openai_format = tool.to_openai_function()
|
||||
# {
|
||||
# "type": "function",
|
||||
# "function": {
|
||||
# "name": "my_tool",
|
||||
# "description": "Does something useful...",
|
||||
# "parameters": { ... }
|
||||
# }
|
||||
# }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Adding a New Benchmark
|
||||
|
||||
Benchmarks measure engine performance. All benchmarks implement the
|
||||
`BaseBenchmark` ABC from `bench/_stubs.py` and use the `ensure_registered()`
|
||||
pattern for lazy registration.
|
||||
|
||||
### Complete Example
|
||||
|
||||
Create `src/openjarvis/bench/my_benchmark.py`:
|
||||
|
||||
```python
|
||||
"""Custom benchmark — measures time to first token."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
|
||||
from openjarvis.bench._stubs import BaseBenchmark, BenchmarkResult
|
||||
from openjarvis.core.registry import BenchmarkRegistry
|
||||
from openjarvis.core.types import Message, Role
|
||||
from openjarvis.engine._stubs import InferenceEngine
|
||||
|
||||
|
||||
class TTFTBenchmark(BaseBenchmark):
|
||||
"""Measures time-to-first-token across multiple samples."""
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return "ttft"
|
||||
|
||||
@property
|
||||
def description(self) -> str:
|
||||
return "Measures time-to-first-token latency"
|
||||
|
||||
def run(
|
||||
self,
|
||||
engine: InferenceEngine,
|
||||
model: str,
|
||||
*,
|
||||
num_samples: int = 10,
|
||||
) -> BenchmarkResult:
|
||||
ttft_values: list[float] = []
|
||||
errors = 0
|
||||
|
||||
for _ in range(num_samples):
|
||||
messages = [Message(role=Role.USER, content="Hello")]
|
||||
t0 = time.time()
|
||||
try:
|
||||
engine.generate(messages, model=model)
|
||||
ttft_values.append(time.time() - t0)
|
||||
except Exception:
|
||||
errors += 1
|
||||
|
||||
if not ttft_values:
|
||||
return BenchmarkResult(
|
||||
benchmark_name=self.name,
|
||||
model=model,
|
||||
engine=engine.engine_id,
|
||||
metrics={},
|
||||
samples=num_samples,
|
||||
errors=errors,
|
||||
)
|
||||
|
||||
return BenchmarkResult(
|
||||
benchmark_name=self.name,
|
||||
model=model,
|
||||
engine=engine.engine_id,
|
||||
metrics={
|
||||
"mean_ttft": sum(ttft_values) / len(ttft_values),
|
||||
"min_ttft": min(ttft_values),
|
||||
"max_ttft": max(ttft_values),
|
||||
},
|
||||
samples=num_samples,
|
||||
errors=errors,
|
||||
)
|
||||
|
||||
|
||||
def ensure_registered() -> None: # (1)!
|
||||
"""Register the TTFT benchmark if not already present."""
|
||||
if not BenchmarkRegistry.contains("ttft"):
|
||||
BenchmarkRegistry.register_value("ttft", TTFTBenchmark)
|
||||
```
|
||||
|
||||
1. The `ensure_registered()` function uses `contains()` before
|
||||
`register_value()` so it can be called multiple times safely. This is
|
||||
required because tests clear all registries between runs.
|
||||
|
||||
### Register in `__init__.py`
|
||||
|
||||
Update `src/openjarvis/bench/__init__.py` to call `ensure_registered()`:
|
||||
|
||||
```python
|
||||
from openjarvis.bench.my_benchmark import ensure_registered as _reg_ttft
|
||||
_reg_ttft()
|
||||
```
|
||||
|
||||
### BenchmarkResult Fields
|
||||
|
||||
```python
|
||||
@dataclass(slots=True)
|
||||
class BenchmarkResult:
|
||||
benchmark_name: str # e.g. "latency", "throughput"
|
||||
model: str # Model identifier
|
||||
engine: str # Engine identifier
|
||||
metrics: Dict[str, float] = ... # Measured values
|
||||
metadata: Dict[str, Any] = ... # Extra info
|
||||
samples: int = 0 # Number of samples run
|
||||
errors: int = 0 # Number of failed samples
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Adding a New Router Policy
|
||||
|
||||
Router policies determine which model handles a given query. All policies
|
||||
implement the `RouterPolicy` ABC from `learning/_stubs.py`.
|
||||
|
||||
### Complete Example
|
||||
|
||||
Create `src/openjarvis/learning/my_policy.py`:
|
||||
|
||||
```python
|
||||
"""Custom router policy — selects model based on query length."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import List, Optional
|
||||
|
||||
from openjarvis.core.registry import RouterPolicyRegistry
|
||||
from openjarvis.learning._stubs import RouterPolicy, RoutingContext
|
||||
|
||||
|
||||
class QueryLengthPolicy(RouterPolicy):
|
||||
"""Routes queries to models based on query length.
|
||||
|
||||
Short queries go to a fast, small model. Long or complex queries
|
||||
go to a larger, more capable model.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
available_models: Optional[List[str]] = None,
|
||||
*,
|
||||
default_model: str = "",
|
||||
fallback_model: str = "",
|
||||
short_threshold: int = 100,
|
||||
long_threshold: int = 500,
|
||||
) -> None:
|
||||
self._available = available_models or []
|
||||
self._default = default_model
|
||||
self._fallback = fallback_model
|
||||
self._short_threshold = short_threshold
|
||||
self._long_threshold = long_threshold
|
||||
|
||||
def select_model(self, context: RoutingContext) -> str:
|
||||
"""Return the model registry key best suited for this context."""
|
||||
available = self._available
|
||||
|
||||
if not available:
|
||||
return self._default or self._fallback or ""
|
||||
|
||||
if context.query_length < self._short_threshold:
|
||||
# Prefer the first (presumably smallest) available model
|
||||
return available[0]
|
||||
elif context.query_length > self._long_threshold:
|
||||
# Prefer the last (presumably largest) available model
|
||||
return available[-1]
|
||||
|
||||
# Default to configured model
|
||||
if self._default and self._default in available:
|
||||
return self._default
|
||||
return available[0]
|
||||
|
||||
|
||||
def ensure_registered() -> None:
|
||||
"""Register QueryLengthPolicy if not already present."""
|
||||
if not RouterPolicyRegistry.contains("query_length"):
|
||||
RouterPolicyRegistry.register_value("query_length", QueryLengthPolicy)
|
||||
|
||||
|
||||
ensure_registered()
|
||||
```
|
||||
|
||||
### Register in `__init__.py`
|
||||
|
||||
Update `src/openjarvis/learning/__init__.py`:
|
||||
|
||||
```python
|
||||
from openjarvis.learning.my_policy import ensure_registered as _reg_ql
|
||||
_reg_ql()
|
||||
```
|
||||
|
||||
### Using Your Policy
|
||||
|
||||
Once registered, your policy can be selected via the config file or CLI:
|
||||
|
||||
=== "Config (TOML)"
|
||||
|
||||
```toml
|
||||
[learning]
|
||||
default_policy = "query_length"
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
uv run jarvis ask --router query_length "Hello"
|
||||
```
|
||||
|
||||
### The RoutingContext
|
||||
|
||||
The `RoutingContext` dataclass provides all the information a router needs:
|
||||
|
||||
```python
|
||||
@dataclass(slots=True)
|
||||
class RoutingContext:
|
||||
query: str = ""
|
||||
query_length: int = 0
|
||||
has_code: bool = False
|
||||
has_math: bool = False
|
||||
language: str = "en"
|
||||
urgency: float = 0.5 # 0 = low priority, 1 = real-time
|
||||
metadata: Dict[str, Any] = field(default_factory=dict)
|
||||
```
|
||||
|
||||
The `build_routing_context()` helper in `intelligence/router.py` populates
|
||||
this from a raw query string, detecting code and math patterns automatically.
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
| Component | ABC | Registry | Key location |
|
||||
|---|---|---|---|
|
||||
| Inference Engine | `InferenceEngine` | `EngineRegistry` | `engine/_stubs.py` |
|
||||
| Memory Backend | `MemoryBackend` | `MemoryRegistry` | `memory/_stubs.py` |
|
||||
| Agent | `BaseAgent` | `AgentRegistry` | `agents/_stubs.py` |
|
||||
| Tool | `BaseTool` | `ToolRegistry` | `tools/_stubs.py` |
|
||||
| Benchmark | `BaseBenchmark` | `BenchmarkRegistry` | `bench/_stubs.py` |
|
||||
| Router Policy | `RouterPolicy` | `RouterPolicyRegistry` | `learning/_stubs.py` |
|
||||
|
||||
The general pattern for all extension points:
|
||||
|
||||
1. Implement the ABC in a new module
|
||||
2. Decorate the class with `@XRegistry.register("key")` or use
|
||||
`ensure_registered()` for lazy registration
|
||||
3. Import the module in the package's `__init__.py` (with `try/except
|
||||
ImportError` if optional deps are involved)
|
||||
4. Add tests in `tests/<module>/`
|
||||
5. Add optional dependencies to `pyproject.toml` if needed
|
||||
@@ -0,0 +1,85 @@
|
||||
# Roadmap
|
||||
|
||||
OpenJarvis development follows a phased approach, with each version adding
|
||||
a major pillar or cross-cutting capability to the framework.
|
||||
|
||||
---
|
||||
|
||||
## Development Phases
|
||||
|
||||
| Version | Phase | Status | Delivers |
|
||||
|---|---|---|---|
|
||||
| **v0.1** | Phase 0 -- Scaffolding | :material-check-circle:{ .green } Complete | Project scaffolding, registry system (`RegistryBase[T]`), core types (`Message`, `ModelSpec`, `Conversation`, `ToolResult`), configuration loader with hardware detection, Click CLI skeleton |
|
||||
| **v0.2** | Phase 1 -- Intelligence + Inference | :material-check-circle:{ .green } Complete | Intelligence pillar (model catalog, heuristic router), inference engines (Ollama, vLLM, llama.cpp), engine discovery and health probing, `jarvis ask` command working end-to-end |
|
||||
| **v0.3** | Phase 2 -- Memory | :material-check-circle:{ .green } Complete | Memory backends (SQLite/FTS5, FAISS, ColBERTv2, BM25, Hybrid/RRF), document chunking and ingestion pipeline, context injection with source attribution, `jarvis memory` commands |
|
||||
| **v0.4** | Phase 3 -- Agents + Tools + Server | :material-check-circle:{ .green } Complete | Agent system (SimpleAgent, OrchestratorAgent, OpenClawAgent, CustomAgent), tool system (Calculator, Think, Retrieval, LLM, FileRead), ToolExecutor dispatch engine, OpenAI-compatible API server (`jarvis serve`) |
|
||||
| **v0.5** | Phase 4 -- Learning + Telemetry | :material-check-circle:{ .green } Complete | Learning system (HeuristicRouter policy, TraceDrivenPolicy, GRPO stub), reward functions, telemetry aggregation (per-model/engine stats, export), `--router` CLI flag, `jarvis telemetry` commands |
|
||||
| **v1.0** | Phase 5 -- SDK + Production | :material-check-circle:{ .green } Complete | Python SDK (`Jarvis` class, `MemoryHandle`), OpenClaw agent infrastructure (protocol, transports, plugins), benchmarking framework (latency, throughput), Docker deployment (CPU + GPU), MkDocs documentation site |
|
||||
| **v1.1** | Phase 6 -- Traces + Learning | :material-progress-clock:{ .amber } In Progress | Trace system (`TraceStore`, `TraceCollector`, `TraceAnalyzer`), trace-driven learning, pluggable agent architectures (ReAct, OpenHands), MCP integration layer |
|
||||
|
||||
---
|
||||
|
||||
## Current Status
|
||||
|
||||
OpenJarvis v1.0 is complete. The framework provides:
|
||||
|
||||
- **Four core abstractions** -- Intelligence, Engine, Agentic Logic, Memory -- each with an ABC interface and registry-based discovery
|
||||
- **Five inference engines** -- Ollama, vLLM, llama.cpp, SGLang, Cloud (OpenAI/Anthropic/Google)
|
||||
- **Five memory backends** -- SQLite/FTS5, FAISS, ColBERTv2, BM25, Hybrid (RRF fusion)
|
||||
- **Multiple agent types** -- Simple, Orchestrator, Custom, OpenClaw, ReAct, OpenHands
|
||||
- **Seven built-in tools** -- Calculator, Think, Retrieval, LLM, FileRead, WebSearch, CodeInterpreter
|
||||
- **Python SDK** -- `Jarvis` class for programmatic use
|
||||
- **OpenAI-compatible API server** -- `POST /v1/chat/completions`, `GET /v1/models`
|
||||
- **Benchmarking framework** -- Latency and throughput measurements
|
||||
- **Telemetry and traces** -- SQLite-backed recording and aggregation
|
||||
- **Docker deployment** -- CPU and GPU images with docker-compose
|
||||
|
||||
Phase 6 is actively in progress, adding the trace system and trace-driven
|
||||
learning capabilities.
|
||||
|
||||
---
|
||||
|
||||
## Phase 6 Details
|
||||
|
||||
Phase 6 focuses on closing the loop between execution and learning:
|
||||
|
||||
### Trace System
|
||||
|
||||
- **TraceStore** -- Persists complete `Trace` objects to SQLite, capturing the
|
||||
full sequence of steps (route, retrieve, generate, tool_call, respond) with
|
||||
timing, inputs, outputs, and outcomes
|
||||
- **TraceCollector** -- Wraps any `BaseAgent` to automatically record traces
|
||||
during execution via EventBus subscription
|
||||
- **TraceAnalyzer** -- Read-only query layer providing aggregated statistics
|
||||
(per-route, per-tool, by query type, time-range filtering)
|
||||
|
||||
### Trace-Driven Learning
|
||||
|
||||
- **TraceDrivenPolicy** -- A router policy that learns from historical trace
|
||||
outcomes to improve model selection over time
|
||||
- Query classification groups traces by type (code, math, short, long, general)
|
||||
- Per-model scoring combines success rate and user feedback
|
||||
- Online updates via `observe()` for incremental learning
|
||||
|
||||
### Pluggable Agents
|
||||
|
||||
- **ReActAgent** -- Reasoning + Acting pattern for systematic tool use
|
||||
- **OpenHands** -- Integration with the OpenHands agent framework
|
||||
|
||||
---
|
||||
|
||||
## Future Directions
|
||||
|
||||
Beyond Phase 6, areas of ongoing exploration include:
|
||||
|
||||
- **GRPO training** -- Reinforcement learning from trace data to train the
|
||||
routing policy, moving beyond heuristics and simple statistics
|
||||
- **Streaming telemetry** -- Real-time performance dashboards and alerting
|
||||
- **Multi-model orchestration** -- Coordinating multiple models within a
|
||||
single query pipeline (e.g., small model for classification, large model
|
||||
for generation)
|
||||
- **Federated memory** -- Memory backends that synchronize across devices
|
||||
- **Plugin ecosystem** -- Community-contributed engines, tools, and agents
|
||||
distributed as Python packages
|
||||
- **Energy-aware routing** -- Using power consumption data from telemetry to
|
||||
optimize for energy efficiency alongside latency and quality
|
||||
Reference in New Issue
Block a user