manuel.richarz
b7ad2d2c12
Add `stream: bool` parameter to `POST /v1/managed-agents/{id}/messages`.
When `stream=true`, the agent processes the message synchronously and
returns an SSE stream (OpenAI-compatible format) with token-by-token
response, tool result events, and usage metadata.
This enables real-time voice assistants and chat UIs to receive agent
responses as they are generated, rather than polling for completion.
- Extend SendMessageRequest with `stream` field (default: false)
- Add _stream_managed_agent() helper using asyncio.to_thread()
- Build AgentContext from conversation history for multi-turn support
- Emit tool_results as named SSE events
- Persist agent response in DB after streaming completes
- Add 6 new tests covering streaming behavior
- Update agents.md documentation with streaming examples
Personal AI, On Personal Devices.
Why OpenJarvis?
Personal AI agents are exploding in popularity, but nearly all of them still route intelligence through cloud APIs. Your "personal" AI continues to depend on someone else's server. At the same time, our Intelligence Per Watt research showed that local language models already handle 88.7% of single-turn chat and reasoning queries, with intelligence efficiency improving 5.3× from 2023 to 2025. The models and hardware are increasingly ready. What has been missing is the software stack to make local-first personal AI practical.
OpenJarvis is that stack. It is an opinionated framework for local-first personal AI, built around three core ideas: shared primitives for building on-device agents; evaluations that treat energy, FLOPs, latency, and dollar cost as first-class constraints alongside accuracy; and a learning loop that improves models using local trace data. The goal is simple: make it possible to build personal AI agents that run locally by default, calling the cloud only when truly necessary. OpenJarvis aims to be both a research platform and a production foundation for local AI, in the spirit of PyTorch.
Installation
git clone https://github.com/open-jarvis/OpenJarvis.git
cd OpenJarvis
uv sync # core framework
uv sync --extra server # + FastAPI server
You also need a local inference backend: Ollama, vLLM, SGLang, or llama.cpp. Alternatively, use the cloud engine with OpenAI, Anthropic, Google Gemini, OpenRouter, or MiniMax by setting the corresponding API key environment variable.
Quick Start
The fastest path is Ollama on any machine with Python 3.10+:
# 1. Install OpenJarvis
git clone https://github.com/open-jarvis/OpenJarvis.git
cd OpenJarvis
uv sync
# 2. Detect hardware and generate config
uv run jarvis init
# 3. Install and start Ollama (https://ollama.com)
curl -fsSL https://ollama.com/install.sh | sh
ollama serve # start the Ollama server
# 4. Pull a model
ollama pull qwen3:8b
# 5. Ask a question
uv run jarvis ask "What is the capital of France?"
# 6. Verify your setup
uv run jarvis doctor
jarvis init auto-detects your hardware and recommends the best engine. After init, it prints engine-specific next steps. Run uv run jarvis doctor at any time to diagnose configuration or connectivity issues.
Docker
A Docker Compose configuration bundles Jarvis with Ollama:
# CPU-only (default)
docker compose -f deploy/docker/docker-compose.yml up -d
# NVIDIA GPU (requires NVIDIA Container Toolkit)
docker compose -f deploy/docker/docker-compose.yml \
-f deploy/docker/docker-compose.gpu.nvidia.yml up -d
# AMD GPU (requires ROCm)
docker compose -f deploy/docker/docker-compose.yml \
-f deploy/docker/docker-compose.gpu.rocm.yml up -d
# Pull a model into Ollama
docker compose -f deploy/docker/docker-compose.yml exec ollama ollama pull qwen3:8b
Services: Jarvis on :8000, Ollama on :11434.
Development
From source, you need to make sure Rust is installed on System:
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
Then, you need the Rust extension for full functionality (security, tools, agents, etc.):
# 1. Clone and install Python deps
git clone https://github.com/open-jarvis/OpenJarvis.git
cd OpenJarvis
uv sync --extra dev
# 2. Build and install the Rust extension (requires Rust toolchain)
uv run maturin develop -m rust/crates/openjarvis-python/Cargo.toml
# 3. Run tests
uv run pytest tests/ -v
See Contributing for more.
About
OpenJarvis is part of Intelligence Per Watt, a research initiative studying the efficiency of on-device AI systems. The project is developed at Hazy Research and the Scaling Intelligence Lab at Stanford SAIL.
Sponsors
Laude Institute • Stanford Marlowe • Google Cloud Platform • Lambda Labs • Ollama • IBM Research • Stanford HAI
Citation
@misc{saadfalcon2026openjarvis,
title={OpenJarvis: Personal AI, On Personal Devices},
author={Jon Saad-Falcon and Avanika Narayan and Herumb Shandilya and Hakki Orhun Akengin and Robby Manihani and Gabriel Bo and John Hennessy and Christopher R\'{e} and Azalia Mirhoseini},
year={2026},
howpublished={\url{https://scalingintelligence.stanford.edu/blogs/openjarvis/}},
}