OpenJarvis/docs/index.md

title, description

title	description
OpenJarvis	Programming abstractions for on-device AI

OpenJarvis

Programming abstractions for on-device AI.

OpenJarvis is a modular framework for building, running, and learning from local AI systems. It provides composable abstractions across four core pillars — Intelligence, Engine, Agentic Logic, and Memory — with a cross-cutting trace-driven learning system that improves routing decisions over time.

Everything runs on your hardware. Cloud APIs are optional.

---

Key Features

• Four Core Pillars

  ---

  Intelligence (model routing), Engine (inference runtime), Agentic Logic (tool-calling agents), and Memory (persistent searchable storage) — each with a clear ABC interface and decorator-based registry.

• 5 Engine Backends

  ---

  Ollama, vLLM, SGLang, llama.cpp, and cloud (OpenAI/Anthropic/Google). All implement the same InferenceEngine ABC with generate(), stream(), list_models(), and health().

• 5 Memory Backends

  ---

  SQLite/FTS5 (default, zero-dependency), FAISS, ColBERTv2, BM25, and Hybrid (reciprocal rank fusion). Document chunking, indexing, and context injection built in.

• Hardware-Aware

  ---

  Auto-detects GPU vendor, model, and VRAM via nvidia-smi, rocm-smi, and system_profiler. Recommends the optimal engine for your hardware automatically.

• Offline-First

  ---

  All core functionality works without a network connection. Cloud API backends are optional extras for when you need them.

• OpenAI-Compatible API

  ---

  jarvis serve starts a FastAPI server with POST /v1/chat/completions, GET /v1/models, and SSE streaming. Drop-in replacement for OpenAI-compatible clients.

• Trace-Driven Learning

  ---

  Every agent interaction is recorded as a trace. The learning system uses accumulated traces to improve model routing decisions. Pluggable router policies: heuristic, trace-driven, and GRPO.

• Python SDK

  ---

  The Jarvis class provides a high-level sync API. Three lines of code to ask a question. Full access to agents, tools, memory, and model routing.

• CLI-First

  ---

  jarvis ask, jarvis serve, jarvis memory, jarvis bench, jarvis telemetry — every capability is accessible from the command line with rich terminal output.

---

Quick Start

Python SDK

from openjarvis import Jarvis

j = Jarvis()
response = j.ask("Explain quicksort in two sentences.")
print(response)
j.close()

For more control, use ask_full() to get usage stats, model info, and tool results:

result = j.ask_full(
    "What is 2 + 2?",
    agent="orchestrator",
    tools=["calculator"],
)
print(result["content"])       # "4"
print(result["tool_results"])  # [{tool_name: "calculator", ...}]

CLI

# Ask a question
jarvis ask "What is the capital of France?"

# Use an agent with tools
jarvis ask --agent orchestrator --tools calculator,think "What is 137 * 42?"

# Start the API server
jarvis serve --port 8000

# Index documents and search memory
jarvis memory index ./docs/
jarvis memory search "configuration options"

# Run inference benchmarks
jarvis bench run --json

---

Project Status

OpenJarvis v1.0 is complete. The framework includes the full four-pillar architecture, Python SDK, CLI, OpenAI-compatible API server, OpenClaw agent infrastructure, benchmarking framework, and Docker deployment. The test suite contains over 1,000 tests. Phase 6 (trace system and trace-driven learning) is in active development.

Component	Status
Intelligence (model routing)	Stable
Engine (5 backends)	Stable
Agentic Logic (agents + tools)	Stable
Memory (5 backends)	Stable
Python SDK	Stable
CLI	Stable
API Server	Stable
Trace System	Active Development
Trace-Driven Learning	Active Development
Docker Deployment	Stable

---

Documentation

• Getting Started

  ---

  Install OpenJarvis, configure your first engine, and run your first query in minutes.

• User Guide

  ---

  Comprehensive guides for the CLI, Python SDK, agents, memory, tools, telemetry, and benchmarks.

• Architecture

  ---

  Deep dive into the four-pillar design, registry pattern, query flow, and cross-cutting learning system.

• API Reference

  ---

  Auto-generated reference for every module: SDK, core, engine, agents, memory, tools, intelligence, learning, traces, telemetry, and server.

• Deployment

  ---

  Deploy OpenJarvis with Docker, systemd, or launchd. Includes GPU-accelerated container images.

• Development

  ---

  Contributing guide, extension patterns, roadmap, and changelog.