OpenJarvis

Programming abstractions for on-device AI.

OpenJarvis is a modular framework for building, running, and learning from local AI systems. It provides composable abstractions across five pillars with a cross-cutting trace-driven learning system:

Intelligence -- The LM itself: Llama, Qwen, Claude, GPT, etc. Model catalog, generation defaults, quantization, and preferred engine configuration.
Agents -- The agentic harness for running it: system prompt (including objective, available tools, available models), context from past turns, retry logic, looping logic, exit logic. Seven agent types from simple single-turn to recursive decomposition.
Tools -- In an MCP interface, the available tools and LMs that can be called: web search, calculator, file read, code interpreter, retrieval systems, SQLite, sub-model calls, and any external MCP server.
Engine -- The inference runtime: Ollama, SGLang, vLLM, llama.cpp, cloud APIs (OpenAI, Anthropic, Google). All implement the same InferenceEngine ABC.
Learning -- Methodologies for improving Intelligence (weight updates via SFT) or Agents (changes to system prompt, tools available, models available, retry/looping/exit logic via agent advisor and ICL updater). Trace-driven feedback loop.

Everything runs on your hardware. Cloud APIs are optional.

Key Features

Five Composable Pillars

Intelligence (the model), Agents (agentic harness), Tools (MCP-based tool system with storage), Engine (inference runtime), and Learning (trace-driven improvement) — each with a clear ABC interface and decorator-based registry.
5 Engine Backends

Ollama, vLLM, SGLang, llama.cpp, and cloud (OpenAI/Anthropic/Google). All implement the same InferenceEngine ABC with generate(), stream(), list_models(), and health().
5 Memory Backends

SQLite/FTS5 (default, zero-dependency), FAISS, ColBERTv2, BM25, and Hybrid (reciprocal rank fusion). Document chunking, indexing, and context injection built in.
Hardware-Aware

Auto-detects GPU vendor, model, and VRAM via nvidia-smi, rocm-smi, and system_profiler. Recommends the optimal engine for your hardware automatically.
Offline-First

All core functionality works without a network connection. Cloud API backends are optional extras for when you need them.
OpenAI-Compatible API

jarvis serve starts a FastAPI server with POST /v1/chat/completions, GET /v1/models, and SSE streaming. Drop-in replacement for OpenAI-compatible clients.
Trace-Driven Learning

Every agent interaction is recorded as a trace. The learning system improves Intelligence (SFT weight updates) and Agents (system prompt, tool selection, retry logic). Pluggable policies: heuristic, trace-driven, SFT, agent advisor, ICL updater, GRPO.
Python SDK

The Jarvis class provides a high-level sync API. Three lines of code to ask a question. Full access to agents, tools, memory, and model routing.
CLI-First

jarvis ask, jarvis serve, jarvis memory, jarvis bench, jarvis telemetry — every capability is accessible from the command line with rich terminal output.

Quick Start

Browser App

Run the full chat UI locally with one script:

git clone https://github.com/HazyResearch/OpenJarvis.git
cd OpenJarvis
./scripts/quickstart.sh

This installs dependencies, starts Ollama + a local model, launches the backend and frontend, and opens http://localhost:5173 in your browser.

Desktop App

Download the native desktop app — it bundles Ollama and the Python backend so everything works out of the box.

Download for macOS (Apple Silicon){ .md-button .md-button--primary }

Also available for macOS (Intel), Windows, Linux (DEB), and Linux (RPM). See the Downloads page for details.

Python SDK

from openjarvis import Jarvis

j = Jarvis()
response = j.ask("Explain quicksort in two sentences.")
print(response)
j.close()

For more control, use ask_full() to get usage stats, model info, and tool results:

result = j.ask_full(
    "What is 2 + 2?",
    agent="orchestrator",
    tools=["calculator"],
)
print(result["content"])       # "4"
print(result["tool_results"])  # [{tool_name: "calculator", ...}]

CLI

# Ask a question
jarvis ask "What is the capital of France?"

# Use an agent with tools
jarvis ask --agent orchestrator --tools calculator,think "What is 137 * 42?"

# Start the API server
jarvis serve --port 8000

# Index documents and search memory
jarvis memory index ./docs/
jarvis memory search "configuration options"

# Run inference benchmarks
jarvis bench run --json

Project Status

OpenJarvis v1.5 (Phase 10) is complete. The framework includes the full five-pillar architecture, seven agent types, Python SDK, CLI, OpenAI-compatible API server, benchmarking framework, and Docker deployment. The test suite contains over 1,800 tests.

Component	Status
Intelligence (model catalog + config)	Stable
Agents (7 types: Simple, Orchestrator, NativeReAct, NativeOpenHands, RLM, OpenHands SDK, OpenClaw)	Stable
Tools (MCP interface + 5 storage backends)	Stable
Engine (5 backends)	Stable
Learning (routing, SFT, agent advisor, ICL updater)	Stable
Python SDK	Stable
CLI	Stable
API Server	Stable
Trace System	Stable
Docker Deployment	Stable

Documentation

Getting Started

Install OpenJarvis, configure your first engine, and run your first query in minutes.
User Guide

Comprehensive guides for the CLI, Python SDK, agents, memory, tools, telemetry, and benchmarks.
Architecture

Deep dive into the five-pillar design, registry pattern, query flow, and cross-cutting learning system.
API Reference

Auto-generated reference for every module: SDK, core, engine, agents, memory, tools, intelligence, learning, traces, telemetry, and server.
Deployment

Deploy OpenJarvis with Docker, systemd, or launchd. Includes GPU-accelerated container images.
Development

Contributing guide, extension patterns, roadmap, and changelog.

7.1 KiB Raw Blame History