79e23719d4
OpenJarvis can run vision-capable local models (gemma3, qwen2.5-vl), but the CLI had no way to send them a picture -- the Ollama engine only serialized text. This adds end-to-end image input. What's new - `jarvis ask -i/--image <file>` attaches one or more images to the query. - `jarvis ask -S/--screen` captures the primary monitor (dependency-free on Windows via .NET; mss/Pillow fallback elsewhere). - Vision auto-routes to direct-to-engine mode; with an explicit --agent it warns rather than silently dropping the image. - Privacy guard: warns before sending an image to a non-local engine, keeping OpenJarvis local-first by default. - Context-window default raised 8k -> 16k (JARVIS_NUM_CTX) so an image plus a conversation fit. Implementation - Message.images carries base64 data; messages_to_dicts() forwards it to Ollama's /api/chat "images" field. Text-only messages are unchanged. - GuardrailsEngine preserves images when it rewrites a flagged message. Tests (tests/test_vision.py, 6/6 pass, ruff-clean) - payload forwarding, text path untouched, num_ctx override, guardrail image preservation. Verified on AMD RX 9070 XT (Ollama/Vulkan, 100% GPU) with gemma3:4b: solid-color image, file image, and live screen capture all described. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Jon Saad-Falcon <jonsaadfalcon@gmail.com>
Personal AI, On Personal Devices.
Why OpenJarvis?
Personal AI agents are exploding in popularity, but nearly all of them still route intelligence through cloud APIs. Your "personal" AI continues to depend on someone else's server. At the same time, our Intelligence Per Watt research showed that local language models already handle 88.7% of single-turn chat and reasoning queries, with intelligence efficiency improving 5.3× from 2023 to 2025. The models and hardware are increasingly ready. What has been missing is the software stack to make local-first personal AI practical.
OpenJarvis is that stack. It is a framework for local-first personal AI, built around three core ideas: shared primitives for building on-device agents; evaluations that treat energy, FLOPs, latency, and dollar cost as first-class constraints alongside accuracy; and a learning loop that improves models using local trace data. The goal is simple: make it possible to build personal AI agents that run locally by default, calling the cloud only when truly necessary. OpenJarvis aims to be both a research platform and a production foundation for local AI, in the spirit of PyTorch.
Installation
Pick your platform and run one command. Each installer handles uv, the Python venv, Ollama, and a starter model — about 3 minutes on broadband.
| Platform | One-liner |
|---|---|
| macOS · Linux · WSL2 | curl -fsSL https://open-jarvis.github.io/OpenJarvis/install.sh | bash |
| Native Windows | irm https://open-jarvis.github.io/OpenJarvis/install.ps1 | iex |
| Desktop GUI | Download .exe / .dmg / .deb / .rpm / .AppImage from the latest release |
Then jarvis to start. The Rust extension and larger models continue downloading in the background; jarvis doctor shows status.
Platform-specific notes (WSL2 setup, native-Windows scheduled-task service, desktop prerequisites, manual / contributor install): see the installation docs.
Quick Start
jarvis # start chatting (default: chat-simple)
jarvis init --preset <name> # switch to a starter config
Prefix
jarvis ...withuv run, orsource .venv/bin/activatefirst.
| Preset | What it does |
|---|---|
morning-digest-mac / morning-digest-linux / morning-digest-minimal |
Spoken daily briefing from email, calendar, health, news |
deep-research |
Multi-hop research across indexed docs with citations |
code-assistant |
Agent with code execution, file I/O, and shell access |
scheduled-monitor |
Stateful agent on a schedule with memory |
chat-simple |
Lightweight conversation, no tools |
Example:
jarvis init --preset morning-digest-mac
jarvis connect gdrive # one OAuth covers Gmail / Calendar / Tasks
jarvis digest --fresh # generate and play your first briefing
Per-preset deep dives: morning digest · deep research · code assistant · scheduled monitor · chat simple · or the full quickstart guide.
Skills
Skills teach agents how to better use tools and improve their reasoning. Every skill is a tool — agents discover them from a catalog and invoke them on demand.
# Install skills from public sources
jarvis skill install hermes:arxiv
jarvis skill sync hermes --category research
# Use skills with any agent
jarvis ask "Use the code-explainer skill to explain this Python code: for i in range(5): print(i*2)"
# Optimize skills from your trace history
jarvis optimize skills --policy dspy
# Benchmark the impact
jarvis bench skills --max-samples 5 --seeds 42
Import from Hermes Agent (~150 skills), OpenClaw (~13,700 community skills), or any GitHub repo. Skills follow the agentskills.io open standard.
See the Skills User Guide and Skills Tutorial for details.
Built-in Agents
OpenJarvis ships with eight built-in agents across three execution modes (on-demand, scheduled, continuous):
| Agent | Type | What it does |
|---|---|---|
morning_digest |
Scheduled | Daily briefing from email, calendar, health, news — with TTS audio |
deep_research |
On-demand | Multi-hop research with citations across web and local docs |
monitor_operative |
Continuous | Long-horizon monitoring with memory, compression, and retrieval |
orchestrator |
On-demand | Multi-turn reasoning with automatic tool selection |
native_react |
On-demand | ReAct (Thought-Action-Observation) loop agent |
operative |
Continuous | Persistent autonomous agent with state management |
native_openhands |
On-demand | CodeAct — generates and executes Python code |
simple |
On-demand | Single-turn chat, no tools |
See the User Guide and Tutorials for detailed setup instructions.
Full documentation — including Docker deployment, cloud engines, development setup, and tutorials — at open-jarvis.github.io/OpenJarvis.
Community
- GitHub: github.com/open-jarvis/OpenJarvis
- Discord: discord.gg/CMVBmDQ5Fj
- X / Twitter: @OpenJarvisAI
- Docs: open-jarvis.github.io/OpenJarvis
Contributing
We welcome contributions! See the Contributing Guide for incentives, contribution types, and the PR process.
Quick start for contributors:
git clone https://github.com/open-jarvis/OpenJarvis.git
cd OpenJarvis
uv sync --extra dev
uv run pre-commit install
uv run pytest tests/ -v
Browse the Roadmap for areas where help is needed. Comment "take" on any issue to get auto-assigned.
About
OpenJarvis is part of Intelligence Per Watt, a research initiative studying the intelligence efficiency of AI systems. The project is developed at Hazy Research and the Scaling Intelligence Lab at Stanford SAIL.
Sponsors
Laude Institute • Stanford Marlowe • Google Cloud Platform • Lambda Labs • Ollama • IBM Research • Stanford HAI
Citation
@misc{saadfalcon2026openjarvispersonalaipersonal,
title={OpenJarvis: Personal AI, On Personal Devices},
author={Jon Saad-Falcon and Avanika Narayan and Robby Manihani and Tanvir Bhathal and Herumb Shandilya and Hakki Orhun Akengin and Gabriel Bo and Andrew Park and Matthew Hart and Caia Costello and Chuan Li and Christopher Ré and Azalia Mirhoseini},
year={2026},
eprint={2605.17172},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2605.17172},
}