mirror of
https://github.com/open-jarvis/OpenJarvis.git
synced 2026-07-31 03:12:16 +00:00
Add MkDocs Material documentation site with 40 pages and auto-generated API reference
Sets up a complete documentation website with 7 navigable sections (Home, Getting Started, User Guide, Architecture, API Reference, Deployment, Development), light/dark mode, search, code copy, and Mermaid diagram support. API reference pages use mkdocstrings to auto-generate docs from source docstrings. GitHub Actions workflow deploys to GitHub Pages on push to main. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Jon Saad-Falcon
co-authored by
Claude Opus 4.6
Claude Opus 4.6
parent
990d7d8a79
commit
f75afefcfb
@@ -0,0 +1,269 @@
|
||||
# Agents
|
||||
|
||||
Agents are the agentic logic layer of OpenJarvis. They determine how a query is processed -- whether it goes directly to a model, through a tool-calling loop, or via an external agent runtime. All agents implement the `BaseAgent` ABC and are registered via the `AgentRegistry`.
|
||||
|
||||
## Overview
|
||||
|
||||
| Agent | Registry Key | Tools | Multi-turn | Description |
|
||||
|------------------|-----------------|-------|------------|----------------------------------------------|
|
||||
| `SimpleAgent` | `simple` | No | No | Single-turn query-to-response |
|
||||
| `OrchestratorAgent` | `orchestrator` | Yes | Yes | Multi-turn tool-calling loop |
|
||||
| `OpenClawAgent` | `openclaw` | Yes | Yes | External agent via HTTP or subprocess |
|
||||
| `CustomAgent` | `custom` | -- | -- | Template for user-defined agents |
|
||||
|
||||
---
|
||||
|
||||
## BaseAgent ABC
|
||||
|
||||
All agents extend the abstract `BaseAgent` class.
|
||||
|
||||
```python
|
||||
from abc import ABC, abstractmethod
|
||||
from openjarvis.agents._stubs import AgentContext, AgentResult
|
||||
|
||||
class BaseAgent(ABC):
|
||||
agent_id: str
|
||||
|
||||
@abstractmethod
|
||||
def run(
|
||||
self,
|
||||
input: str,
|
||||
context: AgentContext | None = None,
|
||||
**kwargs,
|
||||
) -> AgentResult:
|
||||
"""Execute the agent on the given input."""
|
||||
```
|
||||
|
||||
### AgentContext
|
||||
|
||||
The runtime context handed to an agent on each invocation.
|
||||
|
||||
| Field | Type | Description |
|
||||
|------------------|--------------------|------------------------------------------------|
|
||||
| `conversation` | `Conversation` | Message history (pre-filled with context if memory injection is active) |
|
||||
| `tools` | `list[str]` | Tool names available to the agent |
|
||||
| `memory_results` | `list[Any]` | Pre-fetched memory retrieval results |
|
||||
| `metadata` | `dict[str, Any]` | Arbitrary metadata for the run |
|
||||
|
||||
### AgentResult
|
||||
|
||||
The result returned after an agent completes a run.
|
||||
|
||||
| Field | Type | Description |
|
||||
|----------------|--------------------|------------------------------------------------|
|
||||
| `content` | `str` | The final response text |
|
||||
| `tool_results` | `list[ToolResult]` | Results from tool executions during the run |
|
||||
| `turns` | `int` | Number of turns (inference calls) taken |
|
||||
| `metadata` | `dict[str, Any]` | Arbitrary metadata about the run |
|
||||
|
||||
---
|
||||
|
||||
## SimpleAgent
|
||||
|
||||
The `SimpleAgent` is a single-turn agent that sends the query directly to the inference engine and returns the response. It does not support tool calling.
|
||||
|
||||
**How it works:**
|
||||
|
||||
1. Builds a message list from the conversation context (if provided) plus the user query.
|
||||
2. Calls the inference engine via `instrumented_generate()` for telemetry tracking.
|
||||
3. Returns the response as an `AgentResult` with `turns=1`.
|
||||
|
||||
**Constructor parameters:**
|
||||
|
||||
| Parameter | Type | Default | Description |
|
||||
|---------------|-------------------|---------|------------------------------------|
|
||||
| `engine` | `InferenceEngine` | -- | The inference engine to use |
|
||||
| `model` | `str` | -- | Model identifier |
|
||||
| `bus` | `EventBus` | `None` | Event bus for telemetry |
|
||||
| `temperature` | `float` | `0.7` | Sampling temperature |
|
||||
| `max_tokens` | `int` | `1024` | Maximum tokens to generate |
|
||||
|
||||
**When to use:** For straightforward question-answering without tool calling or multi-turn reasoning.
|
||||
|
||||
---
|
||||
|
||||
## OrchestratorAgent
|
||||
|
||||
The `OrchestratorAgent` is a multi-turn agent that implements a tool-calling loop. It is the primary agent for queries that require computation, knowledge retrieval, or structured reasoning.
|
||||
|
||||
**How it works:**
|
||||
|
||||
1. Builds the initial message list from context and the user query.
|
||||
2. Sends messages with tool definitions (OpenAI function-calling format) to the engine.
|
||||
3. If the engine responds with `tool_calls`, the `ToolExecutor` dispatches each call.
|
||||
4. Tool results are appended as `TOOL` messages and the loop continues.
|
||||
5. If no `tool_calls` are returned, the response is treated as the final answer.
|
||||
6. The loop stops after `max_turns` iterations (default: 10), returning whatever content is available along with a `max_turns_exceeded` metadata flag.
|
||||
|
||||
**Constructor parameters:**
|
||||
|
||||
| Parameter | Type | Default | Description |
|
||||
|---------------|-------------------|---------|------------------------------------|
|
||||
| `engine` | `InferenceEngine` | -- | The inference engine to use |
|
||||
| `model` | `str` | -- | Model identifier |
|
||||
| `tools` | `list[BaseTool]` | `[]` | Tool instances to make available |
|
||||
| `bus` | `EventBus` | `None` | Event bus for telemetry |
|
||||
| `max_turns` | `int` | `10` | Maximum number of tool-calling turns |
|
||||
| `temperature` | `float` | `0.7` | Sampling temperature |
|
||||
| `max_tokens` | `int` | `1024` | Maximum tokens to generate |
|
||||
|
||||
**When to use:** For queries that need calculation, memory search, sub-model calls, file reading, or multi-step reasoning.
|
||||
|
||||
!!! info "Tool-Calling Loop"
|
||||
The orchestrator follows the OpenAI function-calling convention. The engine must support returning `tool_calls` in its response for the loop to engage. If tools are provided but the engine does not return any tool calls, the agent behaves like a single-turn agent.
|
||||
|
||||
---
|
||||
|
||||
## OpenClawAgent
|
||||
|
||||
The `OpenClawAgent` wraps the OpenClaw Pi agent runtime, communicating via either HTTP or subprocess transport. It supports tool calling through the OpenClaw protocol.
|
||||
|
||||
**How it works:**
|
||||
|
||||
1. Checks transport health.
|
||||
2. Sends a `QUERY` protocol message through the transport.
|
||||
3. If the response is a `TOOL_CALL`, dispatches the tool locally via `ToolExecutor`.
|
||||
4. Sends the tool result back as a `TOOL_RESULT` message.
|
||||
5. Continues the tool-call loop until the response is a final answer or error (up to 10 turns).
|
||||
|
||||
**Constructor parameters:**
|
||||
|
||||
| Parameter | Type | Default | Description |
|
||||
|--------------|----------------------|-----------|-------------------------------------------|
|
||||
| `engine` | `Any` | `None` | Inference engine (fallback/provider) |
|
||||
| `model` | `str` | `""` | Model identifier |
|
||||
| `transport` | `OpenClawTransport` | `None` | Pre-configured transport (overrides mode) |
|
||||
| `mode` | `str` | `"http"` | Transport mode: `"http"` or `"subprocess"` |
|
||||
| `bus` | `EventBus` | `None` | Event bus for telemetry |
|
||||
|
||||
**Transport modes:**
|
||||
|
||||
- **HTTP** (`HttpTransport`): Sends HTTP POST requests to an OpenClaw server.
|
||||
- **Subprocess** (`SubprocessTransport`): Spawns a Node.js process and communicates via stdin/stdout using JSON-line protocol.
|
||||
|
||||
!!! warning "Node.js Requirement"
|
||||
The subprocess transport mode requires Node.js 22+ to be installed on the system.
|
||||
|
||||
---
|
||||
|
||||
## CustomAgent
|
||||
|
||||
The `CustomAgent` is a template for building user-defined agents. It raises `NotImplementedError` by default -- subclass it and override `run()` to implement your logic.
|
||||
|
||||
```python
|
||||
from openjarvis.agents._stubs import AgentContext, AgentResult, BaseAgent
|
||||
from openjarvis.core.registry import AgentRegistry
|
||||
|
||||
|
||||
@AgentRegistry.register("my-agent")
|
||||
class MyAgent(BaseAgent):
|
||||
agent_id = "my-agent"
|
||||
|
||||
def __init__(self, engine, model, **kwargs):
|
||||
self._engine = engine
|
||||
self._model = model
|
||||
|
||||
def run(self, input: str, context: AgentContext | None = None, **kwargs) -> AgentResult:
|
||||
# Your custom logic here
|
||||
result = self._engine.generate(
|
||||
[{"role": "user", "content": input}],
|
||||
model=self._model,
|
||||
)
|
||||
return AgentResult(
|
||||
content=result.get("content", ""),
|
||||
turns=1,
|
||||
)
|
||||
```
|
||||
|
||||
After registration, you can use your custom agent via the CLI or SDK:
|
||||
|
||||
```bash
|
||||
jarvis ask --agent my-agent "Hello"
|
||||
```
|
||||
|
||||
```python
|
||||
response = j.ask("Hello", agent="my-agent")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Using Agents
|
||||
|
||||
### Via CLI
|
||||
|
||||
```bash
|
||||
# Simple agent
|
||||
jarvis ask --agent simple "What is the capital of France?"
|
||||
|
||||
# Orchestrator with tools
|
||||
jarvis ask --agent orchestrator --tools calculator,think "What is sqrt(256)?"
|
||||
|
||||
# OpenClaw agent
|
||||
jarvis ask --agent openclaw "Tell me a story"
|
||||
```
|
||||
|
||||
### Via Python SDK
|
||||
|
||||
```python
|
||||
from openjarvis import Jarvis
|
||||
|
||||
j = Jarvis()
|
||||
|
||||
# Simple agent
|
||||
response = j.ask("Hello", agent="simple")
|
||||
|
||||
# Orchestrator with tools
|
||||
response = j.ask(
|
||||
"Calculate 15% of 340",
|
||||
agent="orchestrator",
|
||||
tools=["calculator"],
|
||||
)
|
||||
|
||||
# Full result with tool details
|
||||
result = j.ask_full(
|
||||
"What is the square root of 144?",
|
||||
agent="orchestrator",
|
||||
tools=["calculator", "think"],
|
||||
)
|
||||
print(result["content"])
|
||||
print(result["turns"])
|
||||
print(result["tool_results"])
|
||||
|
||||
j.close()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Agent Registration
|
||||
|
||||
Agents are registered via the `@AgentRegistry.register()` decorator. This makes them discoverable by name at runtime:
|
||||
|
||||
```python
|
||||
from openjarvis.core.registry import AgentRegistry
|
||||
|
||||
# Check if an agent is registered
|
||||
AgentRegistry.contains("orchestrator") # True
|
||||
|
||||
# Get the agent class
|
||||
agent_cls = AgentRegistry.get("orchestrator")
|
||||
|
||||
# List all registered agent keys
|
||||
AgentRegistry.keys() # ["simple", "orchestrator", "openclaw", "custom"]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Event Bus Integration
|
||||
|
||||
All agents publish events on the `EventBus` when a bus is provided:
|
||||
|
||||
| Event | When |
|
||||
|-------------------------|---------------------------------------------|
|
||||
| `AGENT_TURN_START` | At the beginning of a run |
|
||||
| `AGENT_TURN_END` | At the end of a run (includes turn count) |
|
||||
| `INFERENCE_START` | Before each engine call (orchestrator) |
|
||||
| `INFERENCE_END` | After each engine call (orchestrator) |
|
||||
| `TOOL_CALL_START` | Before each tool execution (openclaw) |
|
||||
| `TOOL_CALL_END` | After each tool execution (openclaw) |
|
||||
|
||||
These events enable the telemetry and trace systems to record detailed interaction data automatically.
|
||||
@@ -0,0 +1,337 @@
|
||||
# Benchmarks
|
||||
|
||||
The benchmarking framework measures inference engine performance with reproducible, standardized tests. It includes built-in benchmarks for latency and throughput, a suite runner for batch execution, and support for custom benchmarks.
|
||||
|
||||
## Overview
|
||||
|
||||
OpenJarvis ships with two benchmarks:
|
||||
|
||||
| Benchmark | Registry Key | Measures |
|
||||
|---------------|----------------|-----------------------------------------------|
|
||||
| **Latency** | `latency` | Per-call inference latency (mean, p50, p95, min, max) |
|
||||
| **Throughput**| `throughput` | Tokens per second throughput |
|
||||
|
||||
---
|
||||
|
||||
## BaseBenchmark ABC
|
||||
|
||||
All benchmarks implement the `BaseBenchmark` abstract base class.
|
||||
|
||||
```python
|
||||
from abc import ABC, abstractmethod
|
||||
from openjarvis.bench._stubs import BenchmarkResult
|
||||
from openjarvis.engine._stubs import InferenceEngine
|
||||
|
||||
class BaseBenchmark(ABC):
|
||||
|
||||
@property
|
||||
@abstractmethod
|
||||
def name(self) -> str:
|
||||
"""Short identifier for this benchmark."""
|
||||
|
||||
@property
|
||||
@abstractmethod
|
||||
def description(self) -> str:
|
||||
"""Human-readable description of what this benchmark measures."""
|
||||
|
||||
@abstractmethod
|
||||
def run(
|
||||
self,
|
||||
engine: InferenceEngine,
|
||||
model: str,
|
||||
*,
|
||||
num_samples: int = 10,
|
||||
) -> BenchmarkResult:
|
||||
"""Execute the benchmark and return results."""
|
||||
```
|
||||
|
||||
### BenchmarkResult
|
||||
|
||||
Each benchmark run produces a `BenchmarkResult`:
|
||||
|
||||
| Field | Type | Description |
|
||||
|------------------|------------------|------------------------------------------|
|
||||
| `benchmark_name` | `str` | Name of the benchmark |
|
||||
| `model` | `str` | Model used |
|
||||
| `engine` | `str` | Engine backend used |
|
||||
| `metrics` | `dict[str, float]` | Key-value pairs of measured metrics |
|
||||
| `metadata` | `dict[str, Any]` | Additional metadata |
|
||||
| `samples` | `int` | Number of samples run |
|
||||
| `errors` | `int` | Number of errors encountered |
|
||||
|
||||
---
|
||||
|
||||
## Built-in Benchmarks
|
||||
|
||||
### Latency Benchmark
|
||||
|
||||
Measures per-call inference latency using short, fixed prompts. Each sample sends a simple prompt to the engine and measures wall-clock time.
|
||||
|
||||
**Prompts used:** The benchmark rotates through a set of short canned prompts ("Hello", "What is 2+2?", "Explain gravity in one sentence") to keep input variation consistent across runs.
|
||||
|
||||
**Metrics produced:**
|
||||
|
||||
| Metric | Description |
|
||||
|-----------------|-----------------------------------------------------|
|
||||
| `mean_latency` | Average latency across all successful samples |
|
||||
| `p50_latency` | Median latency (50th percentile) |
|
||||
| `p95_latency` | 95th percentile latency (tail performance) |
|
||||
| `min_latency` | Fastest single call |
|
||||
| `max_latency` | Slowest single call |
|
||||
|
||||
**Example output:**
|
||||
|
||||
```
|
||||
latency (10 samples, 0 errors)
|
||||
mean_latency: 0.2345
|
||||
p50_latency: 0.2100
|
||||
p95_latency: 0.3800
|
||||
min_latency: 0.1500
|
||||
max_latency: 0.4200
|
||||
```
|
||||
|
||||
### Throughput Benchmark
|
||||
|
||||
Measures inference throughput in tokens per second. Each sample sends a longer prompt ("Write a short paragraph about artificial intelligence") and measures both the time taken and the number of completion tokens generated.
|
||||
|
||||
**Metrics produced:**
|
||||
|
||||
| Metric | Description |
|
||||
|-----------------------|------------------------------------------------|
|
||||
| `tokens_per_second` | Total completion tokens / total time |
|
||||
| `total_tokens` | Total completion tokens across all samples |
|
||||
| `total_time_seconds` | Total wall-clock time across all samples |
|
||||
|
||||
**Example output:**
|
||||
|
||||
```
|
||||
throughput (10 samples, 0 errors)
|
||||
tokens_per_second: 45.6789
|
||||
total_tokens: 1250.0000
|
||||
total_time_seconds: 27.3600
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Interpreting Results
|
||||
|
||||
### Latency Metrics
|
||||
|
||||
- **mean_latency:** The average response time. Use this for general performance comparison.
|
||||
- **p50_latency (median):** The typical response time. Less affected by outliers than the mean.
|
||||
- **p95_latency:** The worst-case response time for 95% of requests. Critical for user experience -- if this is too high, some users will experience noticeable delays.
|
||||
- **min/max_latency:** The best and worst individual calls. A large gap between min and max indicates inconsistent performance.
|
||||
|
||||
!!! tip "What to look for"
|
||||
A healthy setup has `p95 / p50 < 2`. If the p95 is much higher than the median, investigate whether the engine is experiencing contention, thermal throttling, or memory pressure.
|
||||
|
||||
### Throughput Metrics
|
||||
|
||||
- **tokens_per_second:** The main throughput indicator. Higher is better. Typical ranges:
|
||||
- CPU-only: 5-20 tokens/second
|
||||
- Consumer GPU (RTX 3060-4090): 30-100 tokens/second
|
||||
- Data-center GPU (A100, H100): 100-500+ tokens/second
|
||||
- **total_tokens / total_time:** The raw data behind the throughput calculation. Useful for verifying that the engine is generating meaningful output (not returning empty responses).
|
||||
|
||||
---
|
||||
|
||||
## BenchmarkSuite
|
||||
|
||||
The `BenchmarkSuite` class runs a collection of benchmarks and provides aggregation and serialization utilities.
|
||||
|
||||
```python
|
||||
from openjarvis.bench._stubs import BenchmarkSuite
|
||||
from openjarvis.bench.latency import LatencyBenchmark
|
||||
from openjarvis.bench.throughput import ThroughputBenchmark
|
||||
|
||||
suite = BenchmarkSuite([LatencyBenchmark(), ThroughputBenchmark()])
|
||||
|
||||
# Run all benchmarks
|
||||
results = suite.run_all(engine, model, num_samples=20)
|
||||
|
||||
# Serialize to JSONL (one JSON object per line)
|
||||
jsonl = suite.to_jsonl(results)
|
||||
|
||||
# Get a summary dict
|
||||
summary = suite.summary(results)
|
||||
```
|
||||
|
||||
### Methods
|
||||
|
||||
| Method | Returns | Description |
|
||||
|-------------------------|--------------------|--------------------------------------------|
|
||||
| `run_all(engine, model, num_samples=10)` | `list[BenchmarkResult]` | Run all benchmarks sequentially |
|
||||
| `to_jsonl(results)` | `str` | Serialize results to JSONL format |
|
||||
| `summary(results)` | `dict[str, Any]` | Create a summary dictionary |
|
||||
|
||||
### JSONL Format
|
||||
|
||||
Each line in the JSONL output is a JSON object:
|
||||
|
||||
```json
|
||||
{"benchmark_name": "latency", "model": "qwen3:8b", "engine": "ollama", "metrics": {"mean_latency": 0.234, "p50_latency": 0.21, "p95_latency": 0.38, "min_latency": 0.15, "max_latency": 0.42}, "metadata": {}, "samples": 10, "errors": 0}
|
||||
{"benchmark_name": "throughput", "model": "qwen3:8b", "engine": "ollama", "metrics": {"tokens_per_second": 45.67, "total_tokens": 1250.0, "total_time_seconds": 27.36}, "metadata": {}, "samples": 10, "errors": 0}
|
||||
```
|
||||
|
||||
### Summary Format
|
||||
|
||||
```json
|
||||
{
|
||||
"benchmark_count": 2,
|
||||
"benchmarks": [
|
||||
{
|
||||
"name": "latency",
|
||||
"model": "qwen3:8b",
|
||||
"engine": "ollama",
|
||||
"metrics": {"mean_latency": 0.234, ...},
|
||||
"samples": 10,
|
||||
"errors": 0
|
||||
},
|
||||
{
|
||||
"name": "throughput",
|
||||
"model": "qwen3:8b",
|
||||
"engine": "ollama",
|
||||
"metrics": {"tokens_per_second": 45.67, ...},
|
||||
"samples": 10,
|
||||
"errors": 0
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## CLI Usage
|
||||
|
||||
```bash
|
||||
# Run all benchmarks with default settings (10 samples)
|
||||
jarvis bench run
|
||||
|
||||
# Run with more samples for better statistical accuracy
|
||||
jarvis bench run -n 50
|
||||
|
||||
# Run only the latency benchmark
|
||||
jarvis bench run -b latency
|
||||
|
||||
# Run only the throughput benchmark with 20 samples
|
||||
jarvis bench run -b throughput -n 20
|
||||
|
||||
# Specify model and engine
|
||||
jarvis bench run -m qwen3:8b -e ollama
|
||||
|
||||
# Output JSON summary to stdout
|
||||
jarvis bench run --json
|
||||
|
||||
# Write JSONL results to a file
|
||||
jarvis bench run -o results.jsonl
|
||||
|
||||
# Combine options
|
||||
jarvis bench run -b latency -n 100 -m qwen3:8b --json -o latency.jsonl
|
||||
```
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|----------------------------|--------|---------|------------------------------------------|
|
||||
| `-m`, `--model MODEL` | string | auto | Model to benchmark |
|
||||
| `-e`, `--engine ENGINE` | string | auto | Engine backend |
|
||||
| `-n`, `--samples N` | int | `10` | Number of samples per benchmark |
|
||||
| `-b`, `--benchmark NAME` | string | all | Specific benchmark to run (`latency` or `throughput`) |
|
||||
| `-o`, `--output PATH` | path | none | Write JSONL results to file |
|
||||
| `--json` | flag | off | Output JSON summary to stdout |
|
||||
|
||||
---
|
||||
|
||||
## Adding Custom Benchmarks
|
||||
|
||||
Create a custom benchmark by subclassing `BaseBenchmark` and registering it with the `BenchmarkRegistry`.
|
||||
|
||||
### Step 1: Implement the Benchmark
|
||||
|
||||
```python
|
||||
import time
|
||||
from openjarvis.bench._stubs import BaseBenchmark, BenchmarkResult
|
||||
from openjarvis.core.registry import BenchmarkRegistry
|
||||
from openjarvis.core.types import Message, Role
|
||||
from openjarvis.engine._stubs import InferenceEngine
|
||||
|
||||
|
||||
class ContextLengthBenchmark(BaseBenchmark):
|
||||
"""Measures how latency scales with input length."""
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return "context_length"
|
||||
|
||||
@property
|
||||
def description(self) -> str:
|
||||
return "Measures latency scaling with increasing input length"
|
||||
|
||||
def run(
|
||||
self,
|
||||
engine: InferenceEngine,
|
||||
model: str,
|
||||
*,
|
||||
num_samples: int = 10,
|
||||
) -> BenchmarkResult:
|
||||
latencies = {}
|
||||
errors = 0
|
||||
|
||||
for length in [100, 500, 1000, 2000]:
|
||||
prompt = "x " * length
|
||||
messages = [Message(role=Role.USER, content=prompt)]
|
||||
|
||||
t0 = time.time()
|
||||
try:
|
||||
engine.generate(messages, model=model)
|
||||
latencies[f"latency_{length}_tokens"] = time.time() - t0
|
||||
except Exception:
|
||||
errors += 1
|
||||
|
||||
return BenchmarkResult(
|
||||
benchmark_name=self.name,
|
||||
model=model,
|
||||
engine=engine.engine_id,
|
||||
metrics=latencies,
|
||||
samples=len(latencies),
|
||||
errors=errors,
|
||||
)
|
||||
```
|
||||
|
||||
### Step 2: Register the Benchmark
|
||||
|
||||
Use the `ensure_registered()` pattern to survive registry clearing in tests:
|
||||
|
||||
```python
|
||||
def ensure_registered() -> None:
|
||||
"""Register the benchmark if not already present."""
|
||||
if not BenchmarkRegistry.contains("context_length"):
|
||||
BenchmarkRegistry.register_value("context_length", ContextLengthBenchmark)
|
||||
```
|
||||
|
||||
Alternatively, use the decorator at class definition time:
|
||||
|
||||
```python
|
||||
@BenchmarkRegistry.register("context_length")
|
||||
class ContextLengthBenchmark(BaseBenchmark):
|
||||
...
|
||||
```
|
||||
|
||||
!!! info "The `ensure_registered()` Pattern"
|
||||
The `ensure_registered()` function is preferred over the decorator for benchmark modules because it survives registry clearing during testing. The built-in `latency` and `throughput` benchmarks both use this pattern. The benchmark CLI command calls `ensure_registered()` before looking up benchmarks.
|
||||
|
||||
### Step 3: Use Your Benchmark
|
||||
|
||||
Once registered, your benchmark is available through the CLI:
|
||||
|
||||
```bash
|
||||
jarvis bench run -b context_length
|
||||
```
|
||||
|
||||
And through the `BenchmarkSuite`:
|
||||
|
||||
```python
|
||||
from openjarvis.core.registry import BenchmarkRegistry
|
||||
|
||||
bench_cls = BenchmarkRegistry.get("context_length")
|
||||
bench = bench_cls()
|
||||
result = bench.run(engine, model, num_samples=5)
|
||||
```
|
||||
@@ -0,0 +1,395 @@
|
||||
# CLI Reference
|
||||
|
||||
OpenJarvis provides a command-line interface through the `jarvis` command. Built on [Click](https://click.palletsprojects.com/), it offers subcommands for querying models, managing memory, running benchmarks, and serving an OpenAI-compatible API.
|
||||
|
||||
## Global Options
|
||||
|
||||
```bash
|
||||
jarvis --version # Print the OpenJarvis version
|
||||
jarvis --help # Show top-level help with all subcommands
|
||||
```
|
||||
|
||||
## `jarvis init`
|
||||
|
||||
Detect local hardware (CPU, GPU, RAM) and generate a configuration file at `~/.openjarvis/config.toml`.
|
||||
|
||||
```bash
|
||||
jarvis init # Interactive — refuses to overwrite existing config
|
||||
jarvis init --force # Overwrite existing config without prompting
|
||||
```
|
||||
|
||||
| Option | Description |
|
||||
|-----------|-----------------------------------------------|
|
||||
| `--force` | Overwrite existing configuration without prompting |
|
||||
|
||||
The `init` command auto-detects:
|
||||
|
||||
- **Platform** (Linux, macOS, Windows)
|
||||
- **CPU** brand and core count
|
||||
- **RAM** in GB
|
||||
- **GPU** vendor, model, VRAM, and count (via `nvidia-smi`, `rocm-smi`, or `system_profiler`)
|
||||
|
||||
Based on the detected hardware, it recommends an appropriate inference engine and writes a pre-configured TOML file.
|
||||
|
||||
**Example output:**
|
||||
|
||||
```
|
||||
Detecting hardware...
|
||||
Platform : linux
|
||||
CPU : AMD Ryzen 9 7950X (32 cores)
|
||||
RAM : 64 GB
|
||||
GPU : NVIDIA RTX 4090 (24.0 GB VRAM, x1)
|
||||
|
||||
Config written successfully.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## `jarvis ask`
|
||||
|
||||
Send a query to the inference engine (directly or through an agent) and print the response.
|
||||
|
||||
```bash
|
||||
jarvis ask "What is the capital of France?"
|
||||
```
|
||||
|
||||
### Options
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|-------------------------------|---------|------------|-------------------------------------------------------|
|
||||
| `-m`, `--model MODEL` | string | auto | Model to use for inference |
|
||||
| `-e`, `--engine ENGINE` | string | auto | Engine backend (ollama, vllm, llamacpp, etc.) |
|
||||
| `-t`, `--temperature TEMP` | float | `0.7` | Sampling temperature |
|
||||
| `--max-tokens N` | int | `1024` | Maximum tokens to generate |
|
||||
| `--json` | flag | off | Output raw JSON result instead of plain text |
|
||||
| `--no-stream` | flag | off | Disable streaming (synchronous mode) |
|
||||
| `--no-context` | flag | off | Disable memory context injection |
|
||||
| `-a`, `--agent AGENT` | string | none | Agent to use (`simple`, `orchestrator`) |
|
||||
| `--tools TOOLS` | string | none | Comma-separated tool names to enable |
|
||||
| `--router POLICY` | string | config | Router policy for model selection (`heuristic`, `grpo`)|
|
||||
|
||||
### Direct Mode vs Agent Mode
|
||||
|
||||
**Direct mode** (default) sends the query straight to the inference engine:
|
||||
|
||||
```bash
|
||||
jarvis ask "Explain quantum computing"
|
||||
```
|
||||
|
||||
**Agent mode** routes the query through an agent that can use tools and manage multi-turn interactions:
|
||||
|
||||
```bash
|
||||
jarvis ask --agent orchestrator "What is 2+2?"
|
||||
jarvis ask --agent orchestrator --tools calculator,think "Calculate sqrt(144) + 3^2"
|
||||
jarvis ask --agent simple "Hello"
|
||||
```
|
||||
|
||||
### Usage Examples
|
||||
|
||||
```bash
|
||||
# Basic query
|
||||
jarvis ask "What is machine learning?"
|
||||
|
||||
# Specify a model
|
||||
jarvis ask -m qwen3:8b "Summarize this concept"
|
||||
|
||||
# Use the orchestrator agent with tools
|
||||
jarvis ask --agent orchestrator --tools calculator "What is 15% of 340?"
|
||||
|
||||
# Get JSON output
|
||||
jarvis ask --json "Hello"
|
||||
|
||||
# Disable memory context injection
|
||||
jarvis ask --no-context "Tell me about Python"
|
||||
|
||||
# Specify router policy and temperature
|
||||
jarvis ask --router heuristic -t 0.3 "Write a haiku"
|
||||
|
||||
# Set maximum token generation
|
||||
jarvis ask --max-tokens 2048 "Write a detailed essay about AI"
|
||||
```
|
||||
|
||||
### JSON Output Format
|
||||
|
||||
When using `--json` in **direct mode**, the output includes:
|
||||
|
||||
```json
|
||||
{
|
||||
"content": "The response text...",
|
||||
"usage": {
|
||||
"prompt_tokens": 12,
|
||||
"completion_tokens": 85,
|
||||
"total_tokens": 97
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
When using `--json` in **agent mode**, the output includes:
|
||||
|
||||
```json
|
||||
{
|
||||
"content": "The response text...",
|
||||
"turns": 3,
|
||||
"tool_results": [
|
||||
{
|
||||
"tool_name": "calculator",
|
||||
"content": "51.0",
|
||||
"success": true
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## `jarvis model`
|
||||
|
||||
Manage and inspect language models available on running engines.
|
||||
|
||||
### `jarvis model list`
|
||||
|
||||
List all models available from running inference engines, displayed as a Rich table with model parameters, context length, and VRAM requirements.
|
||||
|
||||
```bash
|
||||
jarvis model list
|
||||
```
|
||||
|
||||
**Example output:**
|
||||
|
||||
```
|
||||
Available Models
|
||||
┌─────────┬────────────────┬────────┬─────────┬──────┐
|
||||
│ Engine │ Model │ Params │ Context │ VRAM │
|
||||
├─────────┼────────────────┼────────┼─────────┼──────┤
|
||||
│ ollama │ qwen3:8b │ 8B │ 32,768 │ 6GB │
|
||||
│ ollama │ llama3.2:3b │ 3B │ 8,192 │ 3GB │
|
||||
└─────────┴────────────────┴────────┴─────────┴──────┘
|
||||
```
|
||||
|
||||
### `jarvis model info <model>`
|
||||
|
||||
Show detailed information about a specific model.
|
||||
|
||||
```bash
|
||||
jarvis model info qwen3:8b
|
||||
```
|
||||
|
||||
**Example output:**
|
||||
|
||||
```
|
||||
┌─ Qwen 3 8B ──────────────────────────────┐
|
||||
│ Model ID: qwen3:8b │
|
||||
│ Name: Qwen 3 8B │
|
||||
│ Parameters: 8B │
|
||||
│ Context: 32,768 │
|
||||
│ Quantization: none │
|
||||
│ Min VRAM: 6GB │
|
||||
│ Engines: ollama, vllm │
|
||||
│ Provider: Alibaba │
|
||||
│ API Key: not required │
|
||||
└───────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### `jarvis model pull <model>`
|
||||
|
||||
Download a model via Ollama. Shows a progress bar during download.
|
||||
|
||||
```bash
|
||||
jarvis model pull qwen3:8b
|
||||
```
|
||||
|
||||
!!! note
|
||||
The `pull` command requires a running Ollama instance. It connects to the Ollama API at the host configured in your `config.toml`.
|
||||
|
||||
---
|
||||
|
||||
## `jarvis memory`
|
||||
|
||||
Manage the document memory store for retrieval-augmented generation.
|
||||
|
||||
### `jarvis memory index <path>`
|
||||
|
||||
Index documents from a file or directory into the memory store.
|
||||
|
||||
```bash
|
||||
jarvis memory index ./docs/
|
||||
jarvis memory index ./notes.md
|
||||
jarvis memory index ./data/ --chunk-size 256 --chunk-overlap 32
|
||||
jarvis memory index ./docs/ --backend sqlite
|
||||
```
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|-----------------------------|--------|---------|--------------------------------------|
|
||||
| `--backend`, `-b` | string | config | Override the default memory backend |
|
||||
| `--chunk-size` | int | `512` | Chunk size in tokens |
|
||||
| `--chunk-overlap` | int | `64` | Overlap between chunks in tokens |
|
||||
|
||||
The ingestion pipeline supports text, markdown, code files, and PDF (with `pdfplumber` installed). Binary files and hidden directories are automatically skipped.
|
||||
|
||||
### `jarvis memory search <query>`
|
||||
|
||||
Search the memory store for relevant document chunks.
|
||||
|
||||
```bash
|
||||
jarvis memory search "machine learning basics"
|
||||
jarvis memory search -k 10 "neural networks"
|
||||
jarvis memory search --backend faiss "embeddings"
|
||||
```
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|--------------------|--------|---------|--------------------------------------|
|
||||
| `--top-k`, `-k` | int | `5` | Number of results to return |
|
||||
| `--backend`, `-b` | string | config | Override the default memory backend |
|
||||
|
||||
Results are displayed in a table with rank, score, source file, and a content preview.
|
||||
|
||||
### `jarvis memory stats`
|
||||
|
||||
Show memory store statistics including document count and database size.
|
||||
|
||||
```bash
|
||||
jarvis memory stats
|
||||
jarvis memory stats --backend sqlite
|
||||
```
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|--------------------|--------|---------|--------------------------------------|
|
||||
| `--backend`, `-b` | string | config | Override the default memory backend |
|
||||
|
||||
---
|
||||
|
||||
## `jarvis telemetry`
|
||||
|
||||
Query and manage inference telemetry data stored in SQLite.
|
||||
|
||||
### `jarvis telemetry stats`
|
||||
|
||||
Show aggregated telemetry statistics including total calls, tokens, cost, and latency, broken down by model and engine.
|
||||
|
||||
```bash
|
||||
jarvis telemetry stats
|
||||
jarvis telemetry stats -n 5 # Show top 5 models
|
||||
```
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|-----------------|------|---------|-------------------------------|
|
||||
| `-n`, `--top` | int | `10` | Number of top models to show |
|
||||
|
||||
### `jarvis telemetry export`
|
||||
|
||||
Export raw telemetry records in JSON or CSV format.
|
||||
|
||||
```bash
|
||||
jarvis telemetry export # JSON to stdout
|
||||
jarvis telemetry export --format csv # CSV to stdout
|
||||
jarvis telemetry export --format json -o data.json # JSON to file
|
||||
jarvis telemetry export -f csv -o metrics.csv # CSV to file
|
||||
```
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|-----------------------|--------|----------|---------------------------------|
|
||||
| `-f`, `--format` | choice | `json` | Output format: `json` or `csv` |
|
||||
| `-o`, `--output` | path | stdout | Output file path |
|
||||
|
||||
### `jarvis telemetry clear`
|
||||
|
||||
Delete all telemetry records from the database.
|
||||
|
||||
```bash
|
||||
jarvis telemetry clear # Interactive confirmation
|
||||
jarvis telemetry clear --yes # Skip confirmation
|
||||
```
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|----------------|------|---------|-------------------------------|
|
||||
| `-y`, `--yes` | flag | off | Skip confirmation prompt |
|
||||
|
||||
!!! warning
|
||||
This permanently deletes all stored telemetry data. Use `--yes` to skip the confirmation prompt in automated scripts.
|
||||
|
||||
---
|
||||
|
||||
## `jarvis bench`
|
||||
|
||||
Run inference benchmarks against a running engine.
|
||||
|
||||
### `jarvis bench run`
|
||||
|
||||
Execute benchmarks and report results.
|
||||
|
||||
```bash
|
||||
jarvis bench run # Run all benchmarks, 10 samples
|
||||
jarvis bench run -n 20 # 20 samples per benchmark
|
||||
jarvis bench run -b latency # Only the latency benchmark
|
||||
jarvis bench run -b throughput -n 50 --json # Throughput, 50 samples, JSON output
|
||||
jarvis bench run -o results.jsonl # Write JSONL results to file
|
||||
jarvis bench run -m qwen3:8b -e ollama # Specific model and engine
|
||||
```
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|----------------------------|--------|---------|------------------------------------------|
|
||||
| `-m`, `--model MODEL` | string | auto | Model to benchmark |
|
||||
| `-e`, `--engine ENGINE` | string | auto | Engine backend |
|
||||
| `-n`, `--samples N` | int | `10` | Number of samples per benchmark |
|
||||
| `-b`, `--benchmark NAME` | string | all | Specific benchmark to run |
|
||||
| `-o`, `--output PATH` | path | none | Write JSONL results to file |
|
||||
| `--json` | flag | off | Output JSON summary to stdout |
|
||||
|
||||
Available benchmarks:
|
||||
|
||||
- **latency** -- Measures per-call inference latency (mean, p50, p95, min, max)
|
||||
- **throughput** -- Measures tokens-per-second throughput
|
||||
|
||||
---
|
||||
|
||||
## `jarvis serve`
|
||||
|
||||
Start an OpenAI-compatible API server.
|
||||
|
||||
```bash
|
||||
jarvis serve # Default host/port from config
|
||||
jarvis serve --port 8000 # Custom port
|
||||
jarvis serve --host 0.0.0.0 --port 9000 # Bind to all interfaces
|
||||
jarvis serve --model qwen3:8b # Specify default model
|
||||
jarvis serve --agent orchestrator # Route requests through an agent
|
||||
```
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|--------------------------|--------|---------|------------------------------------------|
|
||||
| `--host HOST` | string | config | Bind address |
|
||||
| `--port PORT` | int | config | Port number |
|
||||
| `-e`, `--engine ENGINE` | string | auto | Engine backend |
|
||||
| `-m`, `--model MODEL` | string | config | Default model for inference |
|
||||
| `-a`, `--agent AGENT` | string | none | Agent for non-streaming requests |
|
||||
|
||||
!!! note "Server Dependencies"
|
||||
The `serve` command requires the server extra:
|
||||
|
||||
```bash
|
||||
pip install 'openjarvis[server]'
|
||||
```
|
||||
|
||||
This installs FastAPI, uvicorn, and related dependencies.
|
||||
|
||||
### API Endpoints
|
||||
|
||||
The server exposes the following OpenAI-compatible endpoints:
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|--------------------------|--------------------------------|
|
||||
| POST | `/v1/chat/completions` | Chat completions (streaming & non-streaming) |
|
||||
| GET | `/v1/models` | List available models |
|
||||
| GET | `/health` | Health check |
|
||||
|
||||
**Example with curl:**
|
||||
|
||||
```bash
|
||||
curl http://localhost:8000/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "qwen3:8b",
|
||||
"messages": [{"role": "user", "content": "Hello!"}]
|
||||
}'
|
||||
```
|
||||
|
||||
When an agent is configured (e.g., `--agent orchestrator`), non-streaming requests are routed through the agent with access to all registered tools. For tool-capable agents (`orchestrator`, `react`, `openhands`), all registered tools are automatically loaded and made available.
|
||||
@@ -0,0 +1,365 @@
|
||||
# Memory
|
||||
|
||||
The memory system provides persistent, searchable document storage for retrieval-augmented generation (RAG). It supports multiple retrieval backends, a configurable chunking pipeline, document ingestion from files and directories, and automatic context injection into prompts.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Documents --> Chunking Pipeline --> Memory Backend --> Context Injection --> Prompt
|
||||
(files) (split + overlap) (store + index) (retrieve + format) (to LLM)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## MemoryBackend ABC
|
||||
|
||||
All memory backends implement the `MemoryBackend` abstract base class.
|
||||
|
||||
```python
|
||||
class MemoryBackend(ABC):
|
||||
backend_id: str
|
||||
|
||||
def store(self, content: str, *, source: str = "", metadata: dict | None = None) -> str:
|
||||
"""Persist content and return a unique document ID."""
|
||||
|
||||
def retrieve(self, query: str, *, top_k: int = 5, **kwargs) -> list[RetrievalResult]:
|
||||
"""Search for query and return the top-k results."""
|
||||
|
||||
def delete(self, doc_id: str) -> bool:
|
||||
"""Delete a document by ID. Return True if it existed."""
|
||||
|
||||
def clear(self) -> None:
|
||||
"""Remove all stored documents."""
|
||||
```
|
||||
|
||||
### RetrievalResult
|
||||
|
||||
Each retrieval returns a list of `RetrievalResult` objects:
|
||||
|
||||
| Field | Type | Description |
|
||||
|------------|------------------|--------------------------------------------|
|
||||
| `content` | `str` | The retrieved text chunk |
|
||||
| `score` | `float` | Relevance score (higher is better) |
|
||||
| `source` | `str` | Originating file path or identifier |
|
||||
| `metadata` | `dict[str, Any]` | Additional metadata (chunk index, etc.) |
|
||||
|
||||
---
|
||||
|
||||
## Backends
|
||||
|
||||
### SQLite / FTS5 (Default)
|
||||
|
||||
**Registry key:** `sqlite`
|
||||
|
||||
The default backend using SQLite's built-in FTS5 full-text search extension. Zero external dependencies -- uses Python's standard `sqlite3` module.
|
||||
|
||||
- **Scoring:** BM25 ranking via FTS5 MATCH queries
|
||||
- **Persistence:** SQLite database file (default: `~/.openjarvis/memory.db`)
|
||||
- **Dependencies:** None (built into Python)
|
||||
|
||||
```python
|
||||
from openjarvis.core.registry import MemoryRegistry
|
||||
|
||||
backend = MemoryRegistry.create("sqlite", db_path="./memory.db")
|
||||
doc_id = backend.store("Hello world", source="test.txt")
|
||||
results = backend.retrieve("hello")
|
||||
backend.close()
|
||||
```
|
||||
|
||||
!!! tip "When to use SQLite/FTS5"
|
||||
Use this backend when you want zero-configuration setup, keyword-based search is sufficient, and you need persistent storage across restarts. It works well for small to medium document collections.
|
||||
|
||||
### FAISS
|
||||
|
||||
**Registry key:** `faiss`
|
||||
|
||||
Dense neural retrieval using Facebook AI Similarity Search. Embeds documents and queries into dense vectors and retrieves by cosine similarity.
|
||||
|
||||
- **Scoring:** Cosine similarity via inner-product search on L2-normalized vectors
|
||||
- **Persistence:** In-memory only (data is lost on restart)
|
||||
- **Dependencies:** `faiss-cpu` (or `faiss-gpu`), `sentence-transformers`
|
||||
|
||||
```bash
|
||||
pip install faiss-cpu sentence-transformers
|
||||
```
|
||||
|
||||
```python
|
||||
backend = MemoryRegistry.create("faiss")
|
||||
doc_id = backend.store("Neural networks are computational models")
|
||||
results = backend.retrieve("deep learning architectures")
|
||||
```
|
||||
|
||||
!!! tip "When to use FAISS"
|
||||
Use this backend when you need semantic search (finding conceptually similar content even without exact keyword matches). Best for use cases where you can re-index on each run since data is not persisted.
|
||||
|
||||
### ColBERTv2
|
||||
|
||||
**Registry key:** `colbert`
|
||||
|
||||
Late-interaction retrieval using ColBERT's token-level embeddings with MaxSim scoring. Provides the highest retrieval quality among the available backends.
|
||||
|
||||
- **Scoring:** MaxSim -- for each query token, take the maximum cosine similarity across all document tokens, then sum
|
||||
- **Persistence:** In-memory only
|
||||
- **Dependencies:** `colbert-ai`, `torch`
|
||||
|
||||
```bash
|
||||
pip install colbert-ai torch
|
||||
```
|
||||
|
||||
```python
|
||||
backend = MemoryRegistry.create(
|
||||
"colbert",
|
||||
checkpoint="colbert-ir/colbertv2.0",
|
||||
device="cpu",
|
||||
)
|
||||
```
|
||||
|
||||
| Parameter | Default | Description |
|
||||
|--------------|----------------------------|-------------------------------------|
|
||||
| `checkpoint` | `"colbert-ir/colbertv2.0"` | ColBERT model checkpoint |
|
||||
| `device` | `"cpu"` | Computation device (`cpu` or `cuda`) |
|
||||
|
||||
!!! tip "When to use ColBERTv2"
|
||||
Use this backend when retrieval quality is the top priority and you have the compute resources for it. The checkpoint is lazily loaded on first use to avoid slow imports. Best for research and evaluation workloads.
|
||||
|
||||
### BM25
|
||||
|
||||
**Registry key:** `bm25`
|
||||
|
||||
Classic probabilistic ranking using the BM25 Okapi algorithm. In-memory implementation using the `rank_bm25` library.
|
||||
|
||||
- **Scoring:** BM25 Okapi term-frequency scoring
|
||||
- **Persistence:** In-memory only
|
||||
- **Dependencies:** `rank-bm25`
|
||||
|
||||
```bash
|
||||
pip install rank-bm25
|
||||
```
|
||||
|
||||
```python
|
||||
backend = MemoryRegistry.create("bm25")
|
||||
backend.store("Python is a programming language", source="intro.txt")
|
||||
results = backend.retrieve("programming language")
|
||||
```
|
||||
|
||||
!!! tip "When to use BM25"
|
||||
Use this backend when you want classic keyword-based retrieval without database dependencies. Useful as the sparse component in a hybrid retrieval setup.
|
||||
|
||||
### Hybrid (RRF Fusion)
|
||||
|
||||
**Registry key:** `hybrid`
|
||||
|
||||
Combines a sparse retriever and a dense retriever using Reciprocal Rank Fusion (RRF). Documents are stored in both sub-backends, and retrieval results are merged.
|
||||
|
||||
- **Scoring:** `RRF_score(d) = sum(weight_i / (k + rank_i(d)))` across both ranked lists
|
||||
- **Persistence:** Depends on sub-backends
|
||||
- **Dependencies:** Depends on sub-backends
|
||||
|
||||
```python
|
||||
from openjarvis.memory.bm25 import BM25Memory
|
||||
from openjarvis.memory.faiss_backend import FAISSMemory
|
||||
|
||||
sparse = BM25Memory()
|
||||
dense = FAISSMemory()
|
||||
|
||||
backend = MemoryRegistry.create(
|
||||
"hybrid",
|
||||
sparse=sparse,
|
||||
dense=dense,
|
||||
k=60,
|
||||
sparse_weight=1.0,
|
||||
dense_weight=1.0,
|
||||
)
|
||||
```
|
||||
|
||||
| Parameter | Default | Description |
|
||||
|-----------------|---------|------------------------------------------|
|
||||
| `sparse` | -- | Sparse retrieval backend (e.g., BM25) |
|
||||
| `dense` | -- | Dense retrieval backend (e.g., FAISS) |
|
||||
| `k` | `60` | RRF constant |
|
||||
| `sparse_weight` | `1.0` | Weight for sparse retriever results |
|
||||
| `dense_weight` | `1.0` | Weight for dense retriever results |
|
||||
|
||||
The hybrid backend over-fetches (3x `top_k`) from each sub-backend before applying fusion to improve result quality.
|
||||
|
||||
!!! tip "When to use Hybrid"
|
||||
Use this backend when you want the best of both keyword matching and semantic similarity. The RRF fusion approach is robust and does not require tuning score distributions across different retrieval methods.
|
||||
|
||||
---
|
||||
|
||||
## Backend Comparison
|
||||
|
||||
| Backend | Search Type | Persistence | Dependencies | Quality | Speed |
|
||||
|-------------|-------------------|-------------|----------------------|----------|----------|
|
||||
| SQLite/FTS5 | Keyword (BM25) | Yes | None | Good | Fast |
|
||||
| FAISS | Dense (cosine) | No | faiss, transformers | Better | Fast |
|
||||
| ColBERTv2 | Late interaction | No | colbert-ai, torch | Best | Slower |
|
||||
| BM25 | Keyword (Okapi) | No | rank-bm25 | Good | Fast |
|
||||
| Hybrid | Fusion (RRF) | Mixed | Sub-backend deps | Better | Medium |
|
||||
|
||||
---
|
||||
|
||||
## Chunking Pipeline
|
||||
|
||||
Documents are split into chunks before storage using a configurable pipeline. The chunker respects paragraph boundaries when possible.
|
||||
|
||||
### ChunkConfig
|
||||
|
||||
| Field | Type | Default | Description |
|
||||
|-----------------|-------|---------|------------------------------------------|
|
||||
| `chunk_size` | `int` | `512` | Target chunk size in whitespace tokens |
|
||||
| `chunk_overlap` | `int` | `64` | Overlap between consecutive chunks |
|
||||
| `min_chunk_size`| `int` | `50` | Minimum chunk size (smaller chunks are discarded) |
|
||||
|
||||
### How Chunking Works
|
||||
|
||||
1. The document is split into paragraphs (separated by double newlines).
|
||||
2. Paragraphs are accumulated until the token count exceeds `chunk_size`.
|
||||
3. The accumulated content is emitted as a chunk.
|
||||
4. The last `chunk_overlap` tokens are retained as context for the next chunk.
|
||||
5. Paragraphs exceeding `chunk_size` are split into fixed-size windows with overlap.
|
||||
|
||||
### Chunk Output
|
||||
|
||||
Each chunk is a `Chunk` object with:
|
||||
|
||||
| Field | Type | Description |
|
||||
|------------|------------------|------------------------------------------|
|
||||
| `content` | `str` | The chunk text |
|
||||
| `source` | `str` | Originating file path |
|
||||
| `offset` | `int` | Token offset within the document |
|
||||
| `index` | `int` | Sequential chunk index |
|
||||
| `metadata` | `dict[str, Any]` | Additional metadata |
|
||||
|
||||
---
|
||||
|
||||
## Document Ingestion
|
||||
|
||||
The `ingest_path()` function reads files or recursively walks directories, producing chunks ready for storage.
|
||||
|
||||
### Supported File Types
|
||||
|
||||
| Type | Extensions |
|
||||
|----------|-------------------------------------------------------------|
|
||||
| Text | `.txt` and other plain text files |
|
||||
| Markdown | `.md`, `.markdown`, `.mdx` |
|
||||
| Code | `.py`, `.js`, `.ts`, `.rs`, `.go`, `.java`, `.c`, `.cpp`, `.rb`, `.sh`, `.yaml`, `.json`, `.html`, `.css`, and more |
|
||||
| PDF | `.pdf` (requires `pdfplumber`: `pip install openjarvis[memory-pdf]`) |
|
||||
|
||||
### Automatic Skipping
|
||||
|
||||
The ingestion pipeline automatically skips:
|
||||
|
||||
- Hidden files and directories (starting with `.`)
|
||||
- Common non-content directories: `__pycache__`, `node_modules`, `.venv`, `.git`, etc.
|
||||
- Binary files: images, audio, video, archives, compiled files
|
||||
- Files that cannot be read (permission errors, encoding issues)
|
||||
|
||||
### Usage
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
from openjarvis.memory.chunking import ChunkConfig
|
||||
from openjarvis.memory.ingest import ingest_path
|
||||
|
||||
# Default chunking
|
||||
chunks = ingest_path(Path("./docs/"))
|
||||
|
||||
# Custom chunking
|
||||
config = ChunkConfig(chunk_size=256, chunk_overlap=32)
|
||||
chunks = ingest_path(Path("./notes.md"), config=config)
|
||||
|
||||
print(f"Produced {len(chunks)} chunks")
|
||||
for chunk in chunks[:3]:
|
||||
print(f" [{chunk.index}] {chunk.source}: {chunk.content[:60]}...")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Context Injection
|
||||
|
||||
When memory context injection is enabled (the default), queries are automatically augmented with relevant retrieved documents before being sent to the model. Each retrieved passage includes source attribution.
|
||||
|
||||
### ContextConfig
|
||||
|
||||
| Field | Type | Default | Description |
|
||||
|---------------------|---------|---------|--------------------------------------------------|
|
||||
| `enabled` | `bool` | `True` | Whether context injection is active |
|
||||
| `top_k` | `int` | `5` | Number of results to retrieve |
|
||||
| `min_score` | `float` | `0.1` | Minimum relevance score threshold |
|
||||
| `max_context_tokens`| `int` | `2048` | Maximum total tokens in injected context |
|
||||
|
||||
### How It Works
|
||||
|
||||
1. The user's query is searched against the memory backend.
|
||||
2. Results below `min_score` are filtered out.
|
||||
3. Results are truncated to fit within `max_context_tokens`.
|
||||
4. A system message is prepended to the conversation with the formatted context:
|
||||
|
||||
```
|
||||
The following context was retrieved from the knowledge base.
|
||||
Use it to inform your response, citing sources where applicable:
|
||||
|
||||
[Source: docs/intro.md] OpenJarvis is a modular AI framework...
|
||||
|
||||
[Source: docs/config.md] Configuration is stored in TOML format...
|
||||
```
|
||||
|
||||
### Disabling Context Injection
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
jarvis ask --no-context "Tell me about Python"
|
||||
```
|
||||
|
||||
=== "Python SDK"
|
||||
|
||||
```python
|
||||
response = j.ask("Tell me about Python", context=False)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## CLI Usage
|
||||
|
||||
```bash
|
||||
# Index a directory
|
||||
jarvis memory index ./docs/
|
||||
|
||||
# Index with custom chunking
|
||||
jarvis memory index ./notes/ --chunk-size 256 --chunk-overlap 32
|
||||
|
||||
# Search the memory store
|
||||
jarvis memory search "machine learning"
|
||||
|
||||
# Search with more results
|
||||
jarvis memory search -k 10 "neural networks"
|
||||
|
||||
# Show memory statistics
|
||||
jarvis memory stats
|
||||
```
|
||||
|
||||
## SDK Usage
|
||||
|
||||
```python
|
||||
from openjarvis import Jarvis
|
||||
|
||||
j = Jarvis()
|
||||
|
||||
# Index documents
|
||||
result = j.memory.index("./docs/", chunk_size=512, chunk_overlap=64)
|
||||
print(f"Indexed {result['chunks']} chunks")
|
||||
|
||||
# Search
|
||||
results = j.memory.search("configuration", top_k=3)
|
||||
for r in results:
|
||||
print(f" [{r['score']:.4f}] {r['source']}: {r['content'][:80]}...")
|
||||
|
||||
# Statistics
|
||||
stats = j.memory.stats()
|
||||
print(f"Backend: {stats['backend']}, Documents: {stats.get('count', 'N/A')}")
|
||||
|
||||
# Clean up
|
||||
j.close()
|
||||
```
|
||||
@@ -0,0 +1,376 @@
|
||||
# Python SDK
|
||||
|
||||
The OpenJarvis Python SDK provides a high-level interface for interacting with local inference engines, managing memory, and running agent workflows. The primary entry point is the `Jarvis` class.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install openjarvis
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
```python
|
||||
from openjarvis import Jarvis
|
||||
|
||||
j = Jarvis()
|
||||
response = j.ask("What is the capital of France?")
|
||||
print(response)
|
||||
j.close()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Jarvis Class
|
||||
|
||||
### Constructor
|
||||
|
||||
```python
|
||||
Jarvis(
|
||||
*,
|
||||
config: JarvisConfig | None = None,
|
||||
config_path: str | None = None,
|
||||
engine_key: str | None = None,
|
||||
model: str | None = None,
|
||||
)
|
||||
```
|
||||
|
||||
| Parameter | Type | Default | Description |
|
||||
|---------------|------------------|---------|----------------------------------------------------------------|
|
||||
| `config` | `JarvisConfig` | `None` | Provide a pre-built configuration object |
|
||||
| `config_path` | `str` | `None` | Path to a TOML configuration file |
|
||||
| `engine_key` | `str` | `None` | Override the engine backend (`"ollama"`, `"vllm"`, etc.) |
|
||||
| `model` | `str` | `None` | Override the default model (e.g., `"qwen3:8b"`) |
|
||||
|
||||
If no `config` or `config_path` is provided, the SDK loads configuration from the default location (`~/.openjarvis/config.toml`), falling back to built-in defaults.
|
||||
|
||||
**Examples:**
|
||||
|
||||
```python
|
||||
# Default configuration — auto-detects engine
|
||||
j = Jarvis()
|
||||
|
||||
# Override the model
|
||||
j = Jarvis(model="qwen3:8b")
|
||||
|
||||
# Override the engine
|
||||
j = Jarvis(engine_key="ollama")
|
||||
|
||||
# Load from a specific config file
|
||||
j = Jarvis(config_path="/path/to/config.toml")
|
||||
```
|
||||
|
||||
### Properties
|
||||
|
||||
| Property | Type | Description |
|
||||
|-----------|----------------|-----------------------------------|
|
||||
| `config` | `JarvisConfig` | The active configuration object |
|
||||
| `version` | `str` | The OpenJarvis version string |
|
||||
| `memory` | `MemoryHandle` | Proxy for memory operations |
|
||||
|
||||
---
|
||||
|
||||
## `ask()` Method
|
||||
|
||||
Send a query and receive a plain-text response.
|
||||
|
||||
```python
|
||||
ask(
|
||||
query: str,
|
||||
*,
|
||||
model: str | None = None,
|
||||
agent: str | None = None,
|
||||
tools: list[str] | None = None,
|
||||
temperature: float = 0.7,
|
||||
max_tokens: int = 1024,
|
||||
context: bool = True,
|
||||
) -> str
|
||||
```
|
||||
|
||||
| Parameter | Type | Default | Description |
|
||||
|---------------|--------------|---------|------------------------------------------------------|
|
||||
| `query` | `str` | -- | The question or prompt to send |
|
||||
| `model` | `str` | `None` | Override the model for this call |
|
||||
| `agent` | `str` | `None` | Route through an agent (`"simple"`, `"orchestrator"`) |
|
||||
| `tools` | `list[str]` | `None` | Tool names to enable (requires agent mode) |
|
||||
| `temperature` | `float` | `0.7` | Sampling temperature |
|
||||
| `max_tokens` | `int` | `1024` | Maximum tokens to generate |
|
||||
| `context` | `bool` | `True` | Whether to inject memory context |
|
||||
|
||||
**Returns:** A `str` containing the model's response text.
|
||||
|
||||
**Examples:**
|
||||
|
||||
```python
|
||||
# Simple query
|
||||
response = j.ask("What is machine learning?")
|
||||
|
||||
# Override model for this call
|
||||
response = j.ask("Hello", model="llama3.2:3b")
|
||||
|
||||
# Disable memory context injection
|
||||
response = j.ask("Tell me about Python", context=False)
|
||||
|
||||
# Adjust generation parameters
|
||||
response = j.ask("Write a haiku", temperature=0.3, max_tokens=50)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## `ask_full()` Method
|
||||
|
||||
Send a query and receive a detailed result dictionary with metadata.
|
||||
|
||||
```python
|
||||
ask_full(
|
||||
query: str,
|
||||
*,
|
||||
model: str | None = None,
|
||||
agent: str | None = None,
|
||||
tools: list[str] | None = None,
|
||||
temperature: float = 0.7,
|
||||
max_tokens: int = 1024,
|
||||
context: bool = True,
|
||||
) -> dict[str, Any]
|
||||
```
|
||||
|
||||
The parameters are identical to `ask()`.
|
||||
|
||||
**Returns:** A dictionary with the following keys:
|
||||
|
||||
=== "Direct Mode"
|
||||
|
||||
| Key | Type | Description |
|
||||
|-----------|--------|------------------------------------------|
|
||||
| `content` | `str` | The response text |
|
||||
| `usage` | `dict` | Token usage (`prompt_tokens`, `completion_tokens`, `total_tokens`) |
|
||||
| `model` | `str` | The model used |
|
||||
| `engine` | `str` | The engine backend used |
|
||||
|
||||
=== "Agent Mode"
|
||||
|
||||
| Key | Type | Description |
|
||||
|----------------|--------------|------------------------------------------|
|
||||
| `content` | `str` | The response text |
|
||||
| `usage` | `dict` | Token usage (may be empty in agent mode) |
|
||||
| `tool_results` | `list[dict]` | Tool execution results |
|
||||
| `turns` | `int` | Number of agent turns taken |
|
||||
| `model` | `str` | The model used |
|
||||
| `engine` | `str` | The engine backend used |
|
||||
|
||||
**Example:**
|
||||
|
||||
```python
|
||||
result = j.ask_full("What is 2+2?")
|
||||
print(result["content"]) # "4"
|
||||
print(result["model"]) # "qwen3:8b"
|
||||
print(result["engine"]) # "ollama"
|
||||
print(result["usage"]) # {"prompt_tokens": 10, ...}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Agent Mode
|
||||
|
||||
Pass the `agent` parameter to route queries through an agent. Agents can manage multi-turn conversations and use tools.
|
||||
|
||||
```python
|
||||
# Simple agent — single turn, no tools
|
||||
response = j.ask("Hello", agent="simple")
|
||||
|
||||
# Orchestrator agent — multi-turn with tool calling
|
||||
response = j.ask(
|
||||
"What is sqrt(144) + 3^2?",
|
||||
agent="orchestrator",
|
||||
tools=["calculator", "think"],
|
||||
)
|
||||
```
|
||||
|
||||
When using agent mode with `ask_full()`, the result includes `tool_results` showing each tool invocation:
|
||||
|
||||
```python
|
||||
result = j.ask_full(
|
||||
"Calculate 15% of 340",
|
||||
agent="orchestrator",
|
||||
tools=["calculator"],
|
||||
)
|
||||
|
||||
print(result["content"]) # "15% of 340 is 51.0"
|
||||
print(result["turns"]) # 2
|
||||
print(result["tool_results"])
|
||||
# [{"tool_name": "calculator", "content": "51.0", "success": True}]
|
||||
```
|
||||
|
||||
Available agents: `simple`, `orchestrator`, `openclaw`, `custom`
|
||||
|
||||
Available tools: `calculator`, `think`, `retrieval`, `llm`, `file_read`
|
||||
|
||||
---
|
||||
|
||||
## MemoryHandle
|
||||
|
||||
The `Jarvis.memory` attribute provides a `MemoryHandle` for document indexing, search, and statistics. The memory backend is lazily initialized on first use.
|
||||
|
||||
### `index()`
|
||||
|
||||
Index a file or directory into the memory store.
|
||||
|
||||
```python
|
||||
index(
|
||||
path: str,
|
||||
*,
|
||||
chunk_size: int = 512,
|
||||
chunk_overlap: int = 64,
|
||||
) -> dict[str, Any]
|
||||
```
|
||||
|
||||
| Parameter | Type | Default | Description |
|
||||
|-----------------|-------|---------|---------------------------------------|
|
||||
| `path` | `str` | -- | Path to a file or directory to index |
|
||||
| `chunk_size` | `int` | `512` | Chunk size in tokens |
|
||||
| `chunk_overlap` | `int` | `64` | Overlap between chunks in tokens |
|
||||
|
||||
**Returns:** A dictionary with `chunks` (count), `doc_ids` (list), and `path`.
|
||||
|
||||
```python
|
||||
result = j.memory.index("./docs/")
|
||||
print(f"Indexed {result['chunks']} chunks")
|
||||
# Indexed 42 chunks
|
||||
|
||||
# Custom chunking parameters
|
||||
result = j.memory.index("./notes/", chunk_size=256, chunk_overlap=32)
|
||||
```
|
||||
|
||||
### `search()`
|
||||
|
||||
Search the memory store for relevant chunks.
|
||||
|
||||
```python
|
||||
search(
|
||||
query: str,
|
||||
*,
|
||||
top_k: int = 5,
|
||||
) -> list[dict[str, Any]]
|
||||
```
|
||||
|
||||
| Parameter | Type | Default | Description |
|
||||
|-----------|-------|---------|--------------------------------|
|
||||
| `query` | `str` | -- | The search query |
|
||||
| `top_k` | `int` | `5` | Number of results to return |
|
||||
|
||||
**Returns:** A list of dictionaries, each containing `content`, `score`, `source`, and `metadata`.
|
||||
|
||||
```python
|
||||
results = j.memory.search("neural networks")
|
||||
for r in results:
|
||||
print(f"[{r['score']:.4f}] {r['source']}: {r['content'][:80]}...")
|
||||
```
|
||||
|
||||
### `stats()`
|
||||
|
||||
Return memory backend statistics.
|
||||
|
||||
```python
|
||||
stats() -> dict[str, Any]
|
||||
```
|
||||
|
||||
**Returns:** A dictionary with `backend` (name) and `count` (document count, if available).
|
||||
|
||||
```python
|
||||
info = j.memory.stats()
|
||||
print(f"Backend: {info['backend']}, Documents: {info.get('count', 'N/A')}")
|
||||
```
|
||||
|
||||
### `close()`
|
||||
|
||||
Release the memory backend and its resources.
|
||||
|
||||
```python
|
||||
j.memory.close()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Model and Engine Discovery
|
||||
|
||||
### `list_models()`
|
||||
|
||||
Return a list of model identifiers available on the active engine.
|
||||
|
||||
```python
|
||||
models = j.list_models()
|
||||
print(models) # ["qwen3:8b", "llama3.2:3b", ...]
|
||||
```
|
||||
|
||||
### `list_engines()`
|
||||
|
||||
Return a list of registered engine keys.
|
||||
|
||||
```python
|
||||
engines = j.list_engines()
|
||||
print(engines) # ["ollama", "vllm", "llamacpp", ...]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Resource Management
|
||||
|
||||
### `close()`
|
||||
|
||||
Release all resources held by the `Jarvis` instance, including the memory backend, telemetry store, and engine connection.
|
||||
|
||||
```python
|
||||
j.close()
|
||||
```
|
||||
|
||||
!!! tip "Context Manager Pattern"
|
||||
While `Jarvis` does not implement `__enter__`/`__exit__` directly, you should always call `close()` when done to free database connections and other resources:
|
||||
|
||||
```python
|
||||
j = Jarvis()
|
||||
try:
|
||||
response = j.ask("Hello")
|
||||
print(response)
|
||||
finally:
|
||||
j.close()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Complete Example
|
||||
|
||||
```python
|
||||
from openjarvis import Jarvis
|
||||
|
||||
# Initialize with auto-detected engine
|
||||
j = Jarvis(model="qwen3:8b")
|
||||
|
||||
# Index documents for context-augmented responses
|
||||
result = j.memory.index("./docs/")
|
||||
print(f"Indexed {result['chunks']} chunks from {result['path']}")
|
||||
|
||||
# Simple query with memory context
|
||||
response = j.ask("What are the main features?")
|
||||
print(response)
|
||||
|
||||
# Detailed query with agent and tools
|
||||
full_result = j.ask_full(
|
||||
"Calculate the square root of 256 and add 10",
|
||||
agent="orchestrator",
|
||||
tools=["calculator"],
|
||||
)
|
||||
print(f"Answer: {full_result['content']}")
|
||||
print(f"Turns: {full_result['turns']}")
|
||||
print(f"Tools used: {[t['tool_name'] for t in full_result['tool_results']]}")
|
||||
|
||||
# Search memory directly
|
||||
results = j.memory.search("configuration")
|
||||
for r in results:
|
||||
print(f" [{r['score']:.3f}] {r['source']}")
|
||||
|
||||
# List available models
|
||||
print("Models:", j.list_models())
|
||||
|
||||
# Clean up
|
||||
j.close()
|
||||
```
|
||||
@@ -0,0 +1,423 @@
|
||||
# Telemetry & Traces
|
||||
|
||||
OpenJarvis has two complementary observability systems: **telemetry** for per-inference metrics and **traces** for full interaction-level recording. Together, they provide comprehensive insight into system behavior and power the learning system's routing policy updates.
|
||||
|
||||
---
|
||||
|
||||
## Telemetry
|
||||
|
||||
The telemetry system records metrics for every inference call -- latency, token counts, cost, and energy consumption. Data is stored in SQLite and can be queried, exported, and aggregated.
|
||||
|
||||
### TelemetryRecord
|
||||
|
||||
Each inference call produces a `TelemetryRecord` with the following fields:
|
||||
|
||||
| Field | Type | Description |
|
||||
|----------------------|------------------|------------------------------------------|
|
||||
| `timestamp` | `float` | Unix timestamp of the call |
|
||||
| `model_id` | `str` | Model identifier |
|
||||
| `engine` | `str` | Engine backend used |
|
||||
| `agent` | `str` | Agent used (if any) |
|
||||
| `prompt_tokens` | `int` | Input tokens consumed |
|
||||
| `completion_tokens` | `int` | Output tokens generated |
|
||||
| `total_tokens` | `int` | Total tokens (prompt + completion) |
|
||||
| `latency_seconds` | `float` | Wall-clock inference time |
|
||||
| `ttft` | `float` | Time to first token |
|
||||
| `cost_usd` | `float` | Estimated cost in USD |
|
||||
| `energy_joules` | `float` | Estimated energy consumption |
|
||||
| `power_watts` | `float` | Power draw during inference |
|
||||
| `metadata` | `dict[str, Any]` | Additional metadata |
|
||||
|
||||
### TelemetryStore
|
||||
|
||||
The `TelemetryStore` is an append-only SQLite database that persists telemetry records. It integrates with the event bus to capture records automatically.
|
||||
|
||||
```python
|
||||
from openjarvis.telemetry.store import TelemetryStore
|
||||
from openjarvis.core.events import EventBus
|
||||
|
||||
bus = EventBus()
|
||||
store = TelemetryStore(db_path="~/.openjarvis/telemetry.db")
|
||||
store.subscribe_to_bus(bus)
|
||||
|
||||
# Records are now captured automatically when TELEMETRY_RECORD events fire.
|
||||
# No manual recording needed -- instrumented_generate() handles this.
|
||||
|
||||
store.close()
|
||||
```
|
||||
|
||||
The store subscribes to `TELEMETRY_RECORD` events on the event bus. When the `instrumented_generate()` wrapper is used (which happens automatically in both CLI and SDK), telemetry records are published and stored without any manual intervention.
|
||||
|
||||
### `instrumented_generate()`
|
||||
|
||||
This wrapper function calls `engine.generate()` and automatically publishes telemetry events:
|
||||
|
||||
1. Publishes `INFERENCE_START` with model and engine info.
|
||||
2. Calls the engine and measures wall-clock latency.
|
||||
3. Extracts token usage from the engine response.
|
||||
4. Creates a `TelemetryRecord` from the measurements.
|
||||
5. Publishes `INFERENCE_END` and `TELEMETRY_RECORD` events.
|
||||
|
||||
All CLI commands and SDK methods use this wrapper, so telemetry is recorded transparently.
|
||||
|
||||
### TelemetryAggregator
|
||||
|
||||
The `TelemetryAggregator` provides read-only query and aggregation methods over stored telemetry data.
|
||||
|
||||
```python
|
||||
from openjarvis.telemetry.aggregator import TelemetryAggregator
|
||||
|
||||
agg = TelemetryAggregator(db_path="~/.openjarvis/telemetry.db")
|
||||
|
||||
# Overall summary
|
||||
summary = agg.summary()
|
||||
print(f"Total calls: {summary.total_calls}")
|
||||
print(f"Total tokens: {summary.total_tokens}")
|
||||
print(f"Total cost: ${summary.total_cost:.6f}")
|
||||
|
||||
# Per-model breakdown
|
||||
for ms in agg.per_model_stats():
|
||||
print(f" {ms.model_id}: {ms.call_count} calls, {ms.avg_latency:.3f}s avg")
|
||||
|
||||
# Per-engine breakdown
|
||||
for es in agg.per_engine_stats():
|
||||
print(f" {es.engine}: {es.call_count} calls, {es.total_tokens} tokens")
|
||||
|
||||
# Top models by usage
|
||||
top = agg.top_models(n=5)
|
||||
|
||||
# Export raw records
|
||||
records = agg.export_records()
|
||||
|
||||
# Time-range filtering (Unix timestamps)
|
||||
recent = agg.summary(since=1700000000.0)
|
||||
|
||||
# Clear all records
|
||||
count = agg.clear()
|
||||
print(f"Deleted {count} records")
|
||||
|
||||
agg.close()
|
||||
```
|
||||
|
||||
#### Aggregation Methods
|
||||
|
||||
| Method | Returns | Description |
|
||||
|---------------------|--------------------|--------------------------------------------|
|
||||
| `summary()` | `AggregatedStats` | Total calls, tokens, cost, latency + per-model and per-engine breakdowns |
|
||||
| `per_model_stats()` | `list[ModelStats]` | Call count, tokens, latency, cost grouped by model |
|
||||
| `per_engine_stats()`| `list[EngineStats]` | Call count, tokens, latency, cost grouped by engine |
|
||||
| `top_models(n)` | `list[ModelStats]` | Top N models by call count |
|
||||
| `export_records()` | `list[dict]` | All records as plain dictionaries |
|
||||
| `record_count()` | `int` | Total number of stored records |
|
||||
| `clear()` | `int` | Delete all records, return count |
|
||||
|
||||
All query methods accept optional `since` and `until` parameters (Unix timestamps) for time-range filtering.
|
||||
|
||||
#### Data Classes
|
||||
|
||||
**ModelStats:**
|
||||
|
||||
| Field | Type | Description |
|
||||
|--------------------|---------|--------------------------------|
|
||||
| `model_id` | `str` | Model identifier |
|
||||
| `call_count` | `int` | Total inference calls |
|
||||
| `total_tokens` | `int` | Total tokens processed |
|
||||
| `prompt_tokens` | `int` | Total input tokens |
|
||||
| `completion_tokens`| `int` | Total output tokens |
|
||||
| `total_latency` | `float` | Sum of all latencies |
|
||||
| `avg_latency` | `float` | Average latency per call |
|
||||
| `total_cost` | `float` | Total cost in USD |
|
||||
|
||||
**EngineStats:**
|
||||
|
||||
| Field | Type | Description |
|
||||
|-----------------|---------|--------------------------------|
|
||||
| `engine` | `str` | Engine identifier |
|
||||
| `call_count` | `int` | Total inference calls |
|
||||
| `total_tokens` | `int` | Total tokens processed |
|
||||
| `total_latency` | `float` | Sum of all latencies |
|
||||
| `avg_latency` | `float` | Average latency per call |
|
||||
| `total_cost` | `float` | Total cost in USD |
|
||||
|
||||
**AggregatedStats:**
|
||||
|
||||
| Field | Type | Description |
|
||||
|-----------------|--------------------|--------------------------------|
|
||||
| `total_calls` | `int` | Total inference calls |
|
||||
| `total_tokens` | `int` | Total tokens across all models |
|
||||
| `total_cost` | `float` | Total cost in USD |
|
||||
| `total_latency` | `float` | Total latency in seconds |
|
||||
| `per_model` | `list[ModelStats]` | Breakdown by model |
|
||||
| `per_engine` | `list[EngineStats]` | Breakdown by engine |
|
||||
|
||||
### CLI Commands
|
||||
|
||||
```bash
|
||||
# Show aggregated statistics
|
||||
jarvis telemetry stats
|
||||
jarvis telemetry stats -n 5 # Top 5 models only
|
||||
|
||||
# Export records
|
||||
jarvis telemetry export # JSON to stdout
|
||||
jarvis telemetry export -f csv # CSV to stdout
|
||||
jarvis telemetry export -o data.json # JSON to file
|
||||
jarvis telemetry export -f csv -o metrics.csv
|
||||
|
||||
# Clear all records
|
||||
jarvis telemetry clear # With confirmation prompt
|
||||
jarvis telemetry clear --yes # Without confirmation
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Traces
|
||||
|
||||
While telemetry captures per-inference metrics, the trace system records **complete interaction sequences** -- the full chain of steps an agent takes to handle a query. Traces are the primary input to the learning system.
|
||||
|
||||
### What is a Trace?
|
||||
|
||||
A `Trace` captures the entire lifecycle of handling a user query:
|
||||
|
||||
| Field | Type | Description |
|
||||
|--------------------------|--------------------|-----------------------------------------------|
|
||||
| `trace_id` | `str` | Unique identifier (auto-generated) |
|
||||
| `query` | `str` | The original user query |
|
||||
| `agent` | `str` | Agent that handled the query |
|
||||
| `model` | `str` | Model used for inference |
|
||||
| `engine` | `str` | Engine backend used |
|
||||
| `steps` | `list[TraceStep]` | Ordered list of processing steps |
|
||||
| `result` | `str` | Final response content |
|
||||
| `outcome` | `str` or `None` | `"success"`, `"failure"`, or `None` (unknown) |
|
||||
| `feedback` | `float` or `None` | User quality score [0, 1] |
|
||||
| `started_at` | `float` | Unix timestamp when processing began |
|
||||
| `ended_at` | `float` | Unix timestamp when processing ended |
|
||||
| `total_tokens` | `int` | Total tokens across all steps |
|
||||
| `total_latency_seconds` | `float` | Total latency across all steps |
|
||||
| `metadata` | `dict[str, Any]` | Additional metadata |
|
||||
|
||||
### Trace vs Telemetry
|
||||
|
||||
| Aspect | Telemetry | Traces |
|
||||
|----------------|----------------------------------------|----------------------------------------------|
|
||||
| **Scope** | Single inference call | Full interaction (multiple steps) |
|
||||
| **Granularity**| Per-call metrics | Step-by-step sequence |
|
||||
| **Purpose** | Performance monitoring, cost tracking | Learning, routing optimization, debugging |
|
||||
| **Data** | Latency, tokens, cost, energy | Route, retrieve, generate, tool_call, respond |
|
||||
| **Storage** | Flat table of records | Traces table + steps table |
|
||||
|
||||
### TraceStep
|
||||
|
||||
Each step in a trace records a single action the agent took.
|
||||
|
||||
| Field | Type | Description |
|
||||
|--------------------|------------------|------------------------------------------|
|
||||
| `step_type` | `StepType` | Type of step (see below) |
|
||||
| `timestamp` | `float` | When the step occurred |
|
||||
| `duration_seconds` | `float` | How long the step took |
|
||||
| `input` | `dict[str, Any]` | Input data for the step |
|
||||
| `output` | `dict[str, Any]` | Output data from the step |
|
||||
| `metadata` | `dict[str, Any]` | Additional metadata |
|
||||
|
||||
### StepType
|
||||
|
||||
| Type | Description | Example Input | Example Output |
|
||||
|--------------|--------------------------------------------------|------------------------------|------------------------------------|
|
||||
| `route` | Model/agent selection decision | `{"query_type": "math"}` | `{"model": "qwen3:8b"}` |
|
||||
| `retrieve` | Memory search for context | `{"query": "topic"}` | `{"num_results": 3}` |
|
||||
| `generate` | LLM inference call | `{"model": "qwen3:8b"}` | `{"tokens": 128}` |
|
||||
| `tool_call` | Tool execution | `{"tool": "calculator"}` | `{"success": true}` |
|
||||
| `respond` | Final response to the user | `{}` | `{"content": "...", "turns": 2}` |
|
||||
|
||||
### TraceCollector
|
||||
|
||||
The `TraceCollector` wraps any `BaseAgent` to automatically record a `Trace` for every `run()` call. It subscribes to event bus events during execution and converts them into `TraceStep` objects.
|
||||
|
||||
```python
|
||||
from openjarvis.agents.orchestrator import OrchestratorAgent
|
||||
from openjarvis.traces.collector import TraceCollector
|
||||
from openjarvis.traces.store import TraceStore
|
||||
from openjarvis.core.events import EventBus
|
||||
|
||||
bus = EventBus()
|
||||
store = TraceStore(db_path="./traces.db")
|
||||
|
||||
agent = OrchestratorAgent(engine, model, tools=tools, bus=bus)
|
||||
collector = TraceCollector(agent, store=store, bus=bus)
|
||||
|
||||
# The trace is recorded automatically
|
||||
result = collector.run("What is 2+2?")
|
||||
print(result.content)
|
||||
# Trace is now saved to the store and published on the bus
|
||||
```
|
||||
|
||||
**How the collector works:**
|
||||
|
||||
1. Subscribes to `INFERENCE_START`, `INFERENCE_END`, `TOOL_CALL_START`, `TOOL_CALL_END`, and `MEMORY_RETRIEVE` events.
|
||||
2. Executes the wrapped agent's `run()` method.
|
||||
3. Converts captured events into `TraceStep` objects with timing data.
|
||||
4. Appends a final `RESPOND` step with the result.
|
||||
5. Builds a complete `Trace` object and saves it to the `TraceStore`.
|
||||
6. Publishes a `TRACE_COMPLETE` event on the bus.
|
||||
7. Unsubscribes from events after the run completes.
|
||||
|
||||
### TraceStore
|
||||
|
||||
The `TraceStore` is an SQLite-backed database for persisting complete traces with their steps.
|
||||
|
||||
```python
|
||||
from openjarvis.traces.store import TraceStore
|
||||
|
||||
store = TraceStore(db_path="./traces.db")
|
||||
|
||||
# Save a trace
|
||||
store.save(trace)
|
||||
|
||||
# Get a specific trace
|
||||
trace = store.get("abc123def456")
|
||||
|
||||
# List traces with filters
|
||||
traces = store.list_traces(
|
||||
agent="orchestrator",
|
||||
model="qwen3:8b",
|
||||
outcome="success",
|
||||
since=1700000000.0,
|
||||
limit=50,
|
||||
)
|
||||
|
||||
# Count total traces
|
||||
count = store.count()
|
||||
|
||||
# Subscribe to event bus for automatic saving
|
||||
store.subscribe_to_bus(bus)
|
||||
|
||||
store.close()
|
||||
```
|
||||
|
||||
#### Filtering Options
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|---------|------------------------------------------|
|
||||
| `agent` | `str` | Filter by agent ID |
|
||||
| `model` | `str` | Filter by model ID |
|
||||
| `outcome` | `str` | Filter by outcome (`"success"`, `"failure"`) |
|
||||
| `since` | `float` | Start of time range (Unix timestamp) |
|
||||
| `until` | `float` | End of time range (Unix timestamp) |
|
||||
| `limit` | `int` | Maximum number of traces to return (default: 100) |
|
||||
|
||||
### TraceAnalyzer
|
||||
|
||||
The `TraceAnalyzer` provides read-only aggregated statistics over stored traces. These statistics are used by the learning system to update routing policies.
|
||||
|
||||
```python
|
||||
from openjarvis.traces.analyzer import TraceAnalyzer
|
||||
|
||||
analyzer = TraceAnalyzer(store=trace_store)
|
||||
|
||||
# Overall summary
|
||||
summary = analyzer.summary()
|
||||
print(f"Total traces: {summary.total_traces}")
|
||||
print(f"Total steps: {summary.total_steps}")
|
||||
print(f"Avg steps/trace: {summary.avg_steps_per_trace:.1f}")
|
||||
print(f"Avg latency: {summary.avg_latency:.3f}s")
|
||||
print(f"Success rate: {summary.success_rate:.1%}")
|
||||
print(f"Step distribution: {summary.step_type_distribution}")
|
||||
|
||||
# Per-route statistics (model + agent combinations)
|
||||
for rs in analyzer.per_route_stats():
|
||||
print(f" {rs.model}/{rs.agent}: {rs.count} traces, "
|
||||
f"{rs.avg_latency:.3f}s avg, {rs.success_rate:.1%} success")
|
||||
|
||||
# Per-tool statistics
|
||||
for ts in analyzer.per_tool_stats():
|
||||
print(f" {ts.tool_name}: {ts.call_count} calls, "
|
||||
f"{ts.avg_latency:.3f}s avg, {ts.success_rate:.1%} success")
|
||||
|
||||
# Find traces matching query characteristics
|
||||
code_traces = analyzer.traces_for_query_type(has_code=True)
|
||||
short_traces = analyzer.traces_for_query_type(max_length=100)
|
||||
|
||||
# Export traces as plain dicts
|
||||
exported = analyzer.export_traces(limit=500)
|
||||
```
|
||||
|
||||
#### Analysis Methods
|
||||
|
||||
| Method | Returns | Description |
|
||||
|---------------------------|--------------------|------------------------------------------------------|
|
||||
| `summary()` | `TraceSummary` | Overall statistics: counts, averages, distributions |
|
||||
| `per_route_stats()` | `list[RouteStats]` | Stats grouped by (model, agent) combinations |
|
||||
| `per_tool_stats()` | `list[ToolStats]` | Stats grouped by tool name |
|
||||
| `traces_for_query_type()` | `list[Trace]` | Filter traces by query characteristics |
|
||||
| `export_traces()` | `list[dict]` | Export traces as serializable dictionaries |
|
||||
|
||||
All analysis methods accept optional `since` and `until` parameters for time-range filtering.
|
||||
|
||||
#### Data Classes
|
||||
|
||||
**TraceSummary:**
|
||||
|
||||
| Field | Type | Description |
|
||||
|--------------------------|-----------------|------------------------------------------|
|
||||
| `total_traces` | `int` | Total number of traces |
|
||||
| `total_steps` | `int` | Total steps across all traces |
|
||||
| `avg_steps_per_trace` | `float` | Average number of steps per trace |
|
||||
| `avg_latency` | `float` | Average total latency per trace |
|
||||
| `avg_tokens` | `float` | Average tokens per trace |
|
||||
| `success_rate` | `float` | Fraction of evaluated traces that succeeded |
|
||||
| `step_type_distribution` | `dict[str, int]`| Count of each step type |
|
||||
|
||||
**RouteStats:**
|
||||
|
||||
| Field | Type | Description |
|
||||
|----------------|----------------|------------------------------------------|
|
||||
| `model` | `str` | Model identifier |
|
||||
| `agent` | `str` | Agent identifier |
|
||||
| `count` | `int` | Number of traces for this route |
|
||||
| `avg_latency` | `float` | Average latency for this route |
|
||||
| `avg_tokens` | `float` | Average tokens for this route |
|
||||
| `success_rate` | `float` | Success rate for this route |
|
||||
| `avg_feedback` | `float` or `None` | Average user feedback (if available) |
|
||||
|
||||
**ToolStats:**
|
||||
|
||||
| Field | Type | Description |
|
||||
|----------------|---------|------------------------------------------|
|
||||
| `tool_name` | `str` | Tool identifier |
|
||||
| `call_count` | `int` | Number of times the tool was called |
|
||||
| `avg_latency` | `float` | Average execution latency |
|
||||
| `success_rate` | `float` | Fraction of successful executions |
|
||||
|
||||
---
|
||||
|
||||
## Data Flow
|
||||
|
||||
The following diagram shows how telemetry and trace data flows through the system:
|
||||
|
||||
```
|
||||
User Query
|
||||
|
|
||||
v
|
||||
Agent.run() --> EventBus --> TraceCollector (captures steps)
|
||||
| |
|
||||
v v
|
||||
Engine.generate() TelemetryStore (captures per-call metrics)
|
||||
|
|
||||
v
|
||||
instrumented_generate()
|
||||
|
|
||||
+---> INFERENCE_START event
|
||||
+---> INFERENCE_END event
|
||||
+---> TELEMETRY_RECORD event
|
||||
|
|
||||
v
|
||||
TraceCollector
|
||||
|
|
||||
+---> Builds Trace with TraceSteps
|
||||
+---> Saves to TraceStore
|
||||
+---> Publishes TRACE_COMPLETE event
|
||||
|
|
||||
v
|
||||
TraceAnalyzer / TelemetryAggregator --> Learning System
|
||||
```
|
||||
|
||||
Both systems operate transparently -- no manual instrumentation is needed when using the CLI or SDK, as they automatically set up the event bus and telemetry store.
|
||||
@@ -0,0 +1,391 @@
|
||||
# Tools
|
||||
|
||||
The tool system enables agents to perform actions beyond text generation -- calculations, memory lookups, file reading, and sub-model calls. Tools follow a spec-driven design with a central dispatch engine and OpenAI function-calling format support.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Agent --> Engine (with tool defs) --> tool_calls response --> ToolExecutor --> Tool.execute()
|
||||
^ |
|
||||
| v
|
||||
+------------------------------- ToolResult <--------------------------------------------+
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## BaseTool ABC
|
||||
|
||||
All tools implement the `BaseTool` abstract base class.
|
||||
|
||||
```python
|
||||
from abc import ABC, abstractmethod
|
||||
from openjarvis.tools._stubs import ToolSpec
|
||||
from openjarvis.core.types import ToolResult
|
||||
|
||||
class BaseTool(ABC):
|
||||
tool_id: str
|
||||
|
||||
@property
|
||||
@abstractmethod
|
||||
def spec(self) -> ToolSpec:
|
||||
"""Return the tool specification."""
|
||||
|
||||
@abstractmethod
|
||||
def execute(self, **params) -> ToolResult:
|
||||
"""Execute the tool with the given parameters."""
|
||||
|
||||
def to_openai_function(self) -> dict:
|
||||
"""Convert to OpenAI function-calling format."""
|
||||
```
|
||||
|
||||
The `to_openai_function()` method is provided by the base class and converts the tool's spec into the format expected by OpenAI-compatible APIs:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "calculator",
|
||||
"description": "Evaluate a mathematical expression safely.",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"expression": {
|
||||
"type": "string",
|
||||
"description": "Math expression to evaluate"
|
||||
}
|
||||
},
|
||||
"required": ["expression"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ToolSpec
|
||||
|
||||
The `ToolSpec` dataclass describes a tool's interface and characteristics.
|
||||
|
||||
| Field | Type | Default | Description |
|
||||
|------------------------|------------------|---------|----------------------------------------------------|
|
||||
| `name` | `str` | -- | Unique tool identifier |
|
||||
| `description` | `str` | -- | Human-readable description (sent to the model) |
|
||||
| `parameters` | `dict[str, Any]` | `{}` | JSON Schema for the tool's parameters |
|
||||
| `category` | `str` | `""` | Tool category (e.g., `math`, `memory`, `reasoning`) |
|
||||
| `cost_estimate` | `float` | `0.0` | Estimated cost per invocation |
|
||||
| `latency_estimate` | `float` | `0.0` | Estimated latency per invocation |
|
||||
| `requires_confirmation`| `bool` | `False` | Whether the tool requires user confirmation |
|
||||
| `metadata` | `dict[str, Any]` | `{}` | Additional metadata |
|
||||
|
||||
---
|
||||
|
||||
## ToolResult
|
||||
|
||||
The `ToolResult` dataclass holds the result of a tool execution.
|
||||
|
||||
| Field | Type | Default | Description |
|
||||
|-------------------|------------------|---------|------------------------------------------|
|
||||
| `tool_name` | `str` | -- | Name of the tool that was called |
|
||||
| `content` | `str` | -- | The tool's output (text) |
|
||||
| `success` | `bool` | `True` | Whether the execution succeeded |
|
||||
| `usage` | `dict[str, Any]` | `{}` | Token usage (for LLM tool) |
|
||||
| `cost_usd` | `float` | `0.0` | Actual cost of the invocation |
|
||||
| `latency_seconds` | `float` | `0.0` | Measured execution latency |
|
||||
| `metadata` | `dict[str, Any]` | `{}` | Additional metadata |
|
||||
|
||||
---
|
||||
|
||||
## ToolExecutor
|
||||
|
||||
The `ToolExecutor` is the central dispatch engine for tool calls. It manages a set of tool instances, parses JSON arguments, measures execution latency, and publishes events on the event bus.
|
||||
|
||||
```python
|
||||
from openjarvis.tools._stubs import ToolExecutor
|
||||
|
||||
executor = ToolExecutor(tools=[calculator, think_tool], bus=event_bus)
|
||||
|
||||
# Get OpenAI-format tool definitions
|
||||
openai_tools = executor.get_openai_tools()
|
||||
|
||||
# Execute a tool call
|
||||
from openjarvis.core.types import ToolCall
|
||||
tc = ToolCall(id="call_1", name="calculator", arguments='{"expression": "2+2"}')
|
||||
result = executor.execute(tc)
|
||||
print(result.content) # "4"
|
||||
```
|
||||
|
||||
### Execution Flow
|
||||
|
||||
1. **Parse arguments:** The `arguments` JSON string from the `ToolCall` is deserialized.
|
||||
2. **Publish start event:** `TOOL_CALL_START` is emitted on the event bus with tool name and arguments.
|
||||
3. **Execute:** The tool's `execute()` method is called with the parsed parameters.
|
||||
4. **Measure latency:** Execution time is recorded in `result.latency_seconds`.
|
||||
5. **Publish end event:** `TOOL_CALL_END` is emitted with success status and latency.
|
||||
6. **Return result:** The `ToolResult` is returned to the caller.
|
||||
|
||||
If the tool name is unknown, a `ToolResult` with `success=False` is returned. If JSON parsing fails or the tool raises an exception, the error is captured and returned as a failed `ToolResult`.
|
||||
|
||||
### Methods
|
||||
|
||||
| Method | Returns | Description |
|
||||
|---------------------|------------------------|--------------------------------------------|
|
||||
| `execute(tool_call)`| `ToolResult` | Parse args, dispatch, measure, emit events |
|
||||
| `available_tools()` | `list[ToolSpec]` | Return specs for all registered tools |
|
||||
| `get_openai_tools()`| `list[dict]` | Return tools in OpenAI function format |
|
||||
|
||||
---
|
||||
|
||||
## Built-in Tools
|
||||
|
||||
### Calculator
|
||||
|
||||
**Registry key:** `calculator` | **Category:** `math`
|
||||
|
||||
Evaluates mathematical expressions safely using Python's `ast` module. No arbitrary code execution -- only whitelisted operations are allowed.
|
||||
|
||||
**Parameters:**
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
|--------------|--------|----------|------------------------------------------|
|
||||
| `expression` | string | Yes | Math expression (e.g., `"2+3*4"`, `"sqrt(16)"`) |
|
||||
|
||||
**Supported operations:**
|
||||
|
||||
| Category | Operations |
|
||||
|--------------|---------------------------------------------------------------|
|
||||
| Arithmetic | `+`, `-`, `*`, `/`, `//` (floor div), `%` (mod), `**` (power) |
|
||||
| Functions | `abs`, `round`, `min`, `max`, `sqrt`, `log`, `log10`, `log2` |
|
||||
| Trigonometry | `sin`, `cos`, `tan` |
|
||||
| Rounding | `ceil`, `floor` |
|
||||
| Constants | `pi`, `e` |
|
||||
|
||||
**Example:**
|
||||
|
||||
```python
|
||||
from openjarvis.tools.calculator import CalculatorTool
|
||||
|
||||
calc = CalculatorTool()
|
||||
result = calc.execute(expression="sqrt(144) + 3**2")
|
||||
print(result.content) # "21.0"
|
||||
print(result.success) # True
|
||||
```
|
||||
|
||||
### Think
|
||||
|
||||
**Registry key:** `think` | **Category:** `reasoning`
|
||||
|
||||
A zero-cost reasoning scratchpad. The input is echoed back as the output, allowing the model to "think out loud" during a tool-calling loop. This enables chain-of-thought reasoning within the agent workflow.
|
||||
|
||||
**Parameters:**
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
|-----------|--------|----------|------------------------------------------|
|
||||
| `thought` | string | Yes | The reasoning or thought process |
|
||||
|
||||
**Example:**
|
||||
|
||||
```python
|
||||
from openjarvis.tools.think import ThinkTool
|
||||
|
||||
think = ThinkTool()
|
||||
result = think.execute(thought="Let me break this problem into steps...")
|
||||
print(result.content) # "Let me break this problem into steps..."
|
||||
print(result.success) # True
|
||||
```
|
||||
|
||||
!!! info "Cost and Latency"
|
||||
The Think tool has zero cost and near-zero latency, making it ideal for structured reasoning without consuming additional resources.
|
||||
|
||||
### Retrieval
|
||||
|
||||
**Registry key:** `retrieval` | **Category:** `memory`
|
||||
|
||||
Searches the memory backend for relevant context and returns formatted results with source attribution.
|
||||
|
||||
**Parameters:**
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
|-----------|---------|----------|------------------------------------------|
|
||||
| `query` | string | Yes | Search query |
|
||||
| `top_k` | integer | No | Number of results (default: 5) |
|
||||
|
||||
**Constructor parameters:**
|
||||
|
||||
| Parameter | Type | Default | Description |
|
||||
|-----------|-----------------|---------|--------------------------------|
|
||||
| `backend` | `MemoryBackend` | `None` | Memory backend to search |
|
||||
| `top_k` | `int` | `5` | Default number of results |
|
||||
|
||||
**Example:**
|
||||
|
||||
```python
|
||||
from openjarvis.tools.retrieval import RetrievalTool
|
||||
from openjarvis.memory.sqlite import SQLiteMemory
|
||||
|
||||
backend = SQLiteMemory(db_path="./memory.db")
|
||||
retrieval = RetrievalTool(backend=backend)
|
||||
result = retrieval.execute(query="machine learning")
|
||||
print(result.content) # Formatted context with source tags
|
||||
```
|
||||
|
||||
### LLM
|
||||
|
||||
**Registry key:** `llm` | **Category:** `inference`
|
||||
|
||||
Delegates a sub-query to an inference engine. Useful for summarization, sub-questions, or generating structured output within an agent workflow.
|
||||
|
||||
**Parameters:**
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
|-----------|--------|----------|------------------------------------------|
|
||||
| `prompt` | string | Yes | The prompt to send to the model |
|
||||
| `system` | string | No | Optional system message for context |
|
||||
|
||||
**Constructor parameters:**
|
||||
|
||||
| Parameter | Type | Default | Description |
|
||||
|-----------|-------------------|---------|--------------------------------|
|
||||
| `engine` | `InferenceEngine` | `None` | Inference engine to use |
|
||||
| `model` | `str` | `""` | Model identifier |
|
||||
|
||||
**Example:**
|
||||
|
||||
```python
|
||||
from openjarvis.tools.llm_tool import LLMTool
|
||||
|
||||
llm = LLMTool(engine=my_engine, model="qwen3:8b")
|
||||
result = llm.execute(
|
||||
prompt="Summarize: AI is transforming industries...",
|
||||
system="You are a concise summarizer.",
|
||||
)
|
||||
print(result.content)
|
||||
```
|
||||
|
||||
### FileRead
|
||||
|
||||
**Registry key:** `file_read` | **Category:** `filesystem`
|
||||
|
||||
Reads file contents with safety validations. Supports optional directory restrictions, file size limits (1 MB max), and line count limiting.
|
||||
|
||||
**Parameters:**
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
|-------------|---------|----------|------------------------------------------|
|
||||
| `path` | string | Yes | Path to the file to read |
|
||||
| `max_lines` | integer | No | Maximum lines to return (default: all) |
|
||||
|
||||
**Constructor parameters:**
|
||||
|
||||
| Parameter | Type | Default | Description |
|
||||
|----------------|--------------|---------|-----------------------------------------------|
|
||||
| `allowed_dirs` | `list[str]` | `None` | Restrict file access to these directories |
|
||||
|
||||
**Safety features:**
|
||||
|
||||
- Path validation against allowed directories (when configured)
|
||||
- Maximum file size: 1 MB
|
||||
- UTF-8 encoding required (rejects binary files)
|
||||
- Existence and file-type checks
|
||||
|
||||
**Example:**
|
||||
|
||||
```python
|
||||
from openjarvis.tools.file_read import FileReadTool
|
||||
|
||||
reader = FileReadTool(allowed_dirs=["/home/user/projects"])
|
||||
result = reader.execute(path="/home/user/projects/README.md", max_lines=50)
|
||||
print(result.content)
|
||||
print(result.metadata) # {"path": "/home/user/projects/README.md", "size_bytes": 1234}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Tool Registration
|
||||
|
||||
Tools are registered via the `@ToolRegistry.register()` decorator, making them discoverable by name at runtime.
|
||||
|
||||
```python
|
||||
from openjarvis.core.registry import ToolRegistry
|
||||
from openjarvis.tools._stubs import BaseTool, ToolSpec
|
||||
from openjarvis.core.types import ToolResult
|
||||
|
||||
|
||||
@ToolRegistry.register("my_tool")
|
||||
class MyTool(BaseTool):
|
||||
tool_id = "my_tool"
|
||||
|
||||
@property
|
||||
def spec(self) -> ToolSpec:
|
||||
return ToolSpec(
|
||||
name="my_tool",
|
||||
description="A custom tool that does something useful.",
|
||||
parameters={
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"input": {
|
||||
"type": "string",
|
||||
"description": "The input to process.",
|
||||
},
|
||||
},
|
||||
"required": ["input"],
|
||||
},
|
||||
category="custom",
|
||||
)
|
||||
|
||||
def execute(self, **params) -> ToolResult:
|
||||
value = params.get("input", "")
|
||||
return ToolResult(
|
||||
tool_name="my_tool",
|
||||
content=f"Processed: {value}",
|
||||
success=True,
|
||||
)
|
||||
```
|
||||
|
||||
After registration, use the tool with an agent:
|
||||
|
||||
```bash
|
||||
jarvis ask --agent orchestrator --tools my_tool "Process this data"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Using Tools with Agents
|
||||
|
||||
### Via CLI
|
||||
|
||||
Tools are specified as a comma-separated list with the `--tools` flag. An agent (typically `orchestrator`) must be selected:
|
||||
|
||||
```bash
|
||||
# Single tool
|
||||
jarvis ask --agent orchestrator --tools calculator "What is 15% of 340?"
|
||||
|
||||
# Multiple tools
|
||||
jarvis ask --agent orchestrator --tools calculator,think "Solve: 2x + 5 = 13"
|
||||
|
||||
# All available tools (list them)
|
||||
jarvis ask --agent orchestrator --tools calculator,think,retrieval,file_read "..."
|
||||
```
|
||||
|
||||
### Via Python SDK
|
||||
|
||||
Tools are passed as a list of name strings:
|
||||
|
||||
```python
|
||||
from openjarvis import Jarvis
|
||||
|
||||
j = Jarvis()
|
||||
|
||||
# Use calculator and think tools
|
||||
result = j.ask_full(
|
||||
"What is the area of a circle with radius 7?",
|
||||
agent="orchestrator",
|
||||
tools=["calculator", "think"],
|
||||
)
|
||||
|
||||
for tr in result["tool_results"]:
|
||||
print(f" {tr['tool_name']}: {tr['content']} (success={tr['success']})")
|
||||
|
||||
j.close()
|
||||
```
|
||||
|
||||
The SDK automatically instantiates tool objects with appropriate dependencies. For example, the `retrieval` tool receives the configured memory backend, and the `llm` tool receives the active engine and model.
|
||||
Reference in New Issue
Block a user