Programming abstractions for on-device AI.

---

OpenJarvis is a framework for building AI systems that run entirely on local hardware. Rather than treating intelligence as a cloud service, OpenJarvis provides composable abstractions for local model selection, inference, agentic reasoning, tool use, and learning — all aware of the hardware they run on.

You write Python programs that compose five pillars — Intelligence (which model), Engine (which runtime), Agents (which reasoning strategy), Tools (which capabilities, via MCP), and Learning (which adaptation policy) — and OpenJarvis handles hardware detection, model routing, telemetry, and trace-driven improvement automatically.

from openjarvis import Jarvis

j = Jarvis()                                      # auto-detect hardware + engine
response = j.ask("Explain backpropagation")       # route to best local model

j.ask("Solve x^2 - 5x + 6 = 0",                  # multi-turn agent with tools
      agent="orchestrator",
      tools=["calculator", "think"])

j.memory.index("./papers/")                       # index documents into local storage
results = j.memory.search("attention mechanism")  # semantic retrieval

j.close()

pip install openjarvis
jarvis ask "Hello, what can you do?"
jarvis serve --port 8000                           # OpenAI-compatible API

Installation

pip install openjarvis            # core framework
pip install openjarvis[server]    # + FastAPI server
pip install openjarvis[openclaw]  # + OpenClaw agent (requires Node.js 22+)

You also need a local inference backend: Ollama, vLLM, SGLang, or llama.cpp.

The Five Pillars

Pillar	What it does	Key abstractions
Intelligence	Model management and routing	RouterPolicy, QueryAnalyzer, ModelCatalog
Engine	Inference runtime abstraction	InferenceEngine ABC — Ollama, vLLM, SGLang, llama.cpp, MLX, cloud
Agents	Pluggable reasoning strategies	BaseAgent ABC — Simple, Orchestrator, ReAct, OpenHands, OpenClaw
Tools	Capabilities via MCP	BaseTool ABC — calculator, code interpreter, web search, memory, LLM sub-calls; external MCP servers auto-discovered
Learning	Trace-driven adaptation	LearningPolicy ABC — SFT (model routing), AgentAdvisor (restructuring), ICL (tool usage)

Every interaction produces a Trace — a structured record of the full reasoning chain (routing decisions, tool calls, latencies, outcomes). Learning policies consume traces to improve model selection, agent behavior, and tool usage over time.

Config-Driven Composition

OpenJarvis is fully configurable via ~/.openjarvis/config.toml or programmatically via SystemBuilder:

from openjarvis.system import SystemBuilder

system = (SystemBuilder()
          .engine("ollama")
          .model("qwen3:8b")
          .agent("orchestrator")
          .tools(["calculator", "think", "memory_retrieve"])
          .telemetry(True)
          .build())

result = system.ask("What is 2+2?")
system.close()

Hardware auto-detection selects the best engine: Apple Silicon → Ollama, NVIDIA datacenter GPUs → vLLM, AMD → vLLM, CPU-only → llama.cpp.

MCP Interoperability

All tools are managed via the Model Context Protocol (MCP). The built-in MCP server exposes every OpenJarvis tool — including memory operations — to any MCP-compatible client (Claude, GPT, Gemini, etc.). External MCP servers are auto-discovered and their tools appear as native BaseTool instances inside OpenJarvis agents.

Documentation

Full docs at the OpenJarvis documentation site or in-repo:

• VISION.md — Project vision and design principles
• CLAUDE.md — Developer reference for the codebase
• docs/ — Architecture guides, API reference, tutorials

About

OpenJarvis is part of Intelligence Per Watt, a research initiative studying the efficiency of on-device AI systems. The project is developed at Hazy Research and the Scaling Intelligence Lab at Stanford SAIL.

License

Apache 2.0