release: v1.0.0 (#349)

This commit is contained in:
Jon Saad-Falcon
2026-05-16 13:47:23 -07:00
committed by GitHub
parent 50b887ba25
commit e97088f199
8 changed files with 259 additions and 30 deletions
+16
View File
@@ -0,0 +1,16 @@
# CODEOWNERS — gates which approvals satisfy the "Require review from
# Code Owners" branch ruleset on `main`.
#
# Anyone listed here may approve PRs against the patterns they own.
# Combined with the matching branch ruleset toggle, only their approvals
# count toward the merge requirement. Non-owners can still leave reviews
# and comments; their approvals simply do not unblock merge.
#
# See: https://docs.github.com/repositories/managing-your-repositories-settings-and-features/customizing-your-repository/about-code-owners
#
# To add more owners, append GitHub handles (`@username`) or team slugs
# (`@open-jarvis/<team>`) to the line below. To gate specific paths
# differently, add a more-specific pattern beneath it (later, more
# specific rules win).
* @jonsaadfalcon @ANarayan @robbym-dev
+140 -28
View File
@@ -8,30 +8,110 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
## [Unreleased]
### Added
## [1.0.0] - 2026-05-15
- AI stack support for evaluating other agentic frameworks via subprocess.
New `evals/backends/external/` subpackage wraps Hermes Agent and OpenClaw
as one-shot subprocess backends behind the existing `InferenceBackend`
ABC; new `evals/comparison/` toolkit provides path + commit-pin
enforcement (`third_party.py`), config templating (`make_configs.py`),
and LaTeX table generation (`table_gen.py`).
- New optional extra `framework-comparison` (depends on `polars`).
- New pytest marker `live_external` for integration tests requiring real
foreign-framework installations.
The five-primitive architecture (Intelligence, Engine, Agents,
Tools & Memory, Learning) is now stable, with efficiency and
on-device learning as first-class capabilities alongside accuracy.
Companion blog post:
[From Minions to OpenJarvis: A Retrospective on Two Years in Local AI](https://hazyresearch.stanford.edu/blog/2026-05-19-minions-to-openjarvis-retrospective).
### Changed
### Highlights
- `JarvisAgentBackend.generate_full` and `JarvisDirectBackend.generate_full` now return
the spec §6.2 extended fields (`energy_joules`, `peak_power_w`, `tool_calls`,
`turn_count`, `framework`, `framework_commit`, `error`) for cross-framework
comparison parity. Existing callers that didn't read these fields are unaffected.
- `_third_party.toml` no longer ships user-specific default paths. Set
`HERMES_AGENT_PATH` and `OPENCLAW_PATH` env vars to point at your local
checkouts before running the framework-comparison harness; missing or
empty paths now raise `ThirdPartyNotFoundError` with an actionable hint.
**Five composable primitives.** Intelligence, Engine, Agents, Tools & Memory,
and Learning each sit behind a single typed interface — any slot is
substitutable without touching the rest. The composition layer is
`JarvisSystem` in `src/openjarvis/system.py`, driven by a TOML config.
#### Skills System (Plans 1, 2A, 2B)
**Built-in agents across three execution modes.** Eight agents spanning a
single-turn chat baseline, a deep-research agent with inline citations,
a CodeAct-style coder, and a continuous monitor with memory compression
for long-horizon workflows. Execution modes cover on-demand, scheduled,
and continuous.
**Starter presets.** Eight preset configs installable via
`jarvis init --preset <name>` bundle an agent with a hardware-appropriate
engine, connectors, and tools. Variants cover Apple Silicon, Linux GPU
servers, and CPU-only laptops, plus a quickstart for LLM-guided spec search.
**Inference engines.** Four first-class local engines (Ollama, vLLM, SGLang,
llama.cpp) and five cloud providers (OpenAI, Anthropic, Google Gemini,
OpenRouter, MiniMax) sit behind a single `Engine` interface. Discovery
in `engine/_discovery.py` picks a sensible default per host.
### Added — hybrid local-cloud capabilities
**Per-query routing via a query-complexity analyzer**
(`src/openjarvis/learning/routing/complexity.py`). Produces a 0.01.0
complexity score with code/math/reasoning signals and a suggested token
budget, populating `RoutingContext` so easy queries stay local and only
queries that need frontier capability escalate.
**LLM-guided spec search** (`src/openjarvis/learning/spec_search/`).
`SpecSearchOrchestrator` wires diagnose → plan → execute → gate into a
single learning session: a frontier model reads traces, proposes
coordinated edits across all five primitives, and a held-out benchmark
gate (`gate/benchmark_gate.py`, `gate/regression.py`, `gate/cold_start.py`)
accepts only non-regressing edits. Ships with the `spec-search-quickstart`
preset and a runnable tutorial at `examples/openjarvis/spec_search_quickstart.py`.
**Six hybrid coordination paradigms** in `src/openjarvis/agents/hybrid/`.
Each paradigm pairs a local student with a frontier cloud teacher under
a different orchestration shape, as `LocalCloudAgent` subclasses:
- `minions` — reactive single-local + single-cloud loop
- `conductor` — static DAG planner
- `advisors` — executor ↔ advisor loop
- `archon` — generate → rank → fuse
- `skillorchestra` — per-query router across local skills
- `toolorchestra` — RL'd local model with a tool pool
A runner CLI (`python -m openjarvis.agents.hybrid.runner --cell <name>`)
and a 35-cell experiment registry (one TOML per method × benchmark ×
model triple) let researchers run, score, and compare these on equal
footing. Includes a Modal-backed SWE-bench-Verified harness scorer
(`evals/scorers/swebench_harness.py`).
### Added — efficiency as a first-class constraint
**Hardware-agnostic energy telemetry at 50ms resolution** across NVIDIA
(`telemetry/energy_nvidia.py`), AMD (`telemetry/energy_amd.py`), Apple
Silicon (`telemetry/energy_apple.py`), and Intel RAPL
(`telemetry/energy_rapl.py`). Energy, dollar cost, FLOPs, and latency
are treated as evaluation targets alongside accuracy.
**Instrumentation for FLOPs, batch, steady-state, ITL, phase energy, and
vLLM-specific metrics.** Joined per-query by the aggregator
(`telemetry/aggregator.py`) so traces carry accuracy + efficiency together.
### Added — local learning loop
**Closed-loop optimization across the stack** — model weights via SFT
(`learning/intelligence/sft_trainer.py`) and GRPO
(`learning/intelligence/grpo_trainer.py` plus an orchestrator-specific
variant under `learning/intelligence/orchestrator/`), prompts via DSPy
(`learning/agents/dspy_optimizer.py`), agent logic via GEPA
(`learning/agents/gepa_optimizer.py`), and engine + stack configuration
via LLM-guided spec search. `LearningOrchestrator` coordinates triggers
and applies optimizer overlays at discovery time so improvements compound
across primitives.
### Added — cross-framework evaluation
**External agentic-framework evaluation via subprocess.** The
`evals/backends/external/` subpackage wraps Hermes Agent and OpenClaw as
one-shot subprocess backends behind the existing `InferenceBackend` ABC.
The `evals/comparison/` toolkit provides path + commit-pin enforcement
(`third_party.py`), config templating (`make_configs.py`), and LaTeX
table generation (`table_gen.py`).
Ships with a new optional extra `framework-comparison` (depends on
`polars`), a `live_external` pytest marker for integration tests
requiring real foreign-framework installations, and a `ToolOrchestra`
evaluation dataset (`evals/datasets/toolorchestra.py`) alongside the
existing 30+ benchmark suite.
### Added — Skills System (Plans 1, 2A, 2B)
- **Skills core** — every skill is a tool. Skills appear in a system prompt catalog, agents invoke them on demand, content (pipeline results, markdown instructions, or both) gets injected into context.
- `SkillManifest` + `SkillStep` types with tags, depends, invocation flags, markdown content
@@ -87,13 +167,45 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
- `docs/getting-started/configuration.md` — expanded with skills config sections
- `CLAUDE.md` — updated architecture section
### Examples & Tutorials
- `examples/openjarvis/spec_search_quickstart.py` — runnable end-to-end
LLM-guided spec search session.
- `docs/user-guide/llm-guided-spec-search.md` — paper-aligned user guide.
- `docs/architecture/learning.md` — Learning primitive deep-dive covering
routing, spec search, optimizers, and the orchestrator.
- `docs/tutorials/` — code-companion, deep-research, messaging-hub,
scheduled-ops, and skills-workflow walkthroughs.
- `src/openjarvis/agents/hybrid/registry/*.toml` — 35-cell registry of
paradigm × benchmark × model experiments.
### Migration from 0.x
- **`learning/distillation/` is now `learning/spec_search/`.** The
subsystem was renamed to match the LLM-guided spec search semantics
documented in the companion paper. Update any imports
(`from openjarvis.learning.distillation.*`
`from openjarvis.learning.spec_search.*`). The `jarvis distillation`
CLI command is removed; use `spec_search`-prefixed config keys instead.
- **`_third_party.toml` no longer ships default paths.** Set
`HERMES_AGENT_PATH` and `OPENCLAW_PATH` env vars to point at your
local checkouts before running the framework-comparison harness;
missing or empty paths now raise `ThirdPartyNotFoundError` with an
actionable hint.
- **Engine `generate_full` return shape extended.**
`JarvisAgentBackend.generate_full` and `JarvisDirectBackend.generate_full`
now return the spec §6.2 extended fields (`energy_joules`,
`peak_power_w`, `tool_calls`, `turn_count`, `framework`,
`framework_commit`, `error`). Existing callers that didn't read these
fields are unaffected; new callers can rely on cross-framework parity.
### Fixed
- **Trace metadata flow** — `ToolResult.metadata` now propagates through `TOOL_CALL_END` event to `TraceStep.metadata` (was silently dropped at the event-bus boundary)
- **TaintSet JSON serialization** — `ToolExecutor._json_safe_metadata()` filters non-JSON-serializable values (like `TaintSet`) from event payloads before they reach `TraceStore`
- **Non-dict YAML frontmatter** — source resolvers handle `yaml.safe_load()` returning a string instead of a dict (discovered on real OpenClaw imports)
- **OpenClaw category/name queries** — `jarvis skill install openclaw:owner/slug` now correctly splits into category + name match
- **SkillDiscovery trace compatibility** — `_extract_tool_sequence` reads from `step.input["tool"]` (the actual `TraceStep` format), not the nonexistent `step.tool_name` attribute
- **LearningOrchestrator skill trigger** — `_maybe_optimize_skills` runs BEFORE the SFT-data short-circuit (skills are tagged via trace metadata, not mined as SFT pairs)
- **PinchBenchScorer constructor** — `SkillBenchmarkRunner` constructs `PinchBenchScorer(judge_backend, model)` instead of no-args
- **EvalRunner results access** — reads per-task data from `eval_runner.results` property, not nonexistent `summary.results`
- **Trace metadata flow** — `ToolResult.metadata` now propagates through `TOOL_CALL_END` event to `TraceStep.metadata` (was silently dropped at the event-bus boundary).
- **TaintSet JSON serialization** — `ToolExecutor._json_safe_metadata()` filters non-JSON-serializable values (like `TaintSet`) from event payloads before they reach `TraceStore`.
- **Non-dict YAML frontmatter** — source resolvers handle `yaml.safe_load()` returning a string instead of a dict (discovered on real OpenClaw imports).
- **OpenClaw category/name queries** — `jarvis skill install openclaw:owner/slug` now correctly splits into category + name match.
- **SkillDiscovery trace compatibility** — `_extract_tool_sequence` reads from `step.input["tool"]` (the actual `TraceStep` format), not the nonexistent `step.tool_name` attribute.
- **LearningOrchestrator skill trigger** — `_maybe_optimize_skills` runs BEFORE the SFT-data short-circuit (skills are tagged via trace metadata, not mined as SFT pairs).
- **PinchBenchScorer constructor** — `SkillBenchmarkRunner` constructs `PinchBenchScorer(judge_backend, model)` instead of no-args.
- **EvalRunner results access** — reads per-task data from `eval_runner.results` property, not nonexistent `summary.results`.
+19
View File
@@ -11,4 +11,23 @@
};
</script>
{% endif %}
{% if config.extra.version and config.extra.version.default %}
<script>
document.addEventListener("DOMContentLoaded", function () {
var header = document.querySelector(".md-header__inner");
if (!header || header.querySelector(".md-version-badge")) return;
var badge = document.createElement("a");
badge.className = "md-version-badge";
badge.textContent = "{{ config.extra.version.default }}";
badge.href = "https://github.com/open-jarvis/OpenJarvis/releases";
badge.rel = "noopener";
var source = header.querySelector(".md-header__source");
if (source) {
header.insertBefore(badge, source);
} else {
header.appendChild(badge);
}
});
</script>
{% endif %}
{% endblock %}
+30
View File
@@ -1,3 +1,33 @@
/* ── Version badge (top-right of header) ─────────────────────────── */
.md-version-badge {
display: inline-flex;
align-items: center;
height: 1.6rem;
padding: 0 0.55rem;
margin: 0 0.4rem 0 0.2rem;
font-family: var(--md-code-font);
font-size: 0.72rem;
font-weight: 500;
line-height: 1;
letter-spacing: 0.02em;
color: var(--md-primary-bg-color, #ffffff);
background-color: rgba(255, 255, 255, 0.12);
border: 1px solid rgba(255, 255, 255, 0.25);
border-radius: 0.6rem;
text-decoration: none;
white-space: nowrap;
transition: background-color 120ms ease, border-color 120ms ease;
}
.md-version-badge:hover,
.md-version-badge:focus {
background-color: rgba(255, 255, 255, 0.22);
border-color: rgba(255, 255, 255, 0.45);
text-decoration: none;
}
@media screen and (max-width: 76.1875em) {
.md-version-badge { display: none; }
}
/* ── Fonts ─────────────────────────────────────────────────────────── */
:root {
--md-text-font: Georgia, "Times New Roman", serif;
+2
View File
@@ -131,6 +131,8 @@ extra_javascript:
- javascripts/docsearch-init.js
extra:
version:
default: v1.0.0
algolia:
app_id: "" # Fill after DocSearch approval
api_key: "" # Search-only key
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "OpenJarvis"
version = "0.1.1"
version = "1.0.0"
description = "OpenJarvis — modular AI assistant backend with composable intelligence primitives"
readme = "README.md"
requires-python = ">=3.10"
+50
View File
@@ -0,0 +1,50 @@
"""Smoke test: every shipped preset config must load cleanly.
Presets are installed via `jarvis init --preset <name>`, which copies
`configs/openjarvis/examples/<name>.toml` to `~/.openjarvis/config.toml`.
A preset that fails to parse via `load_config()` would break first-time
setup, so we validate the whole set on every commit.
"""
from __future__ import annotations
from pathlib import Path
import pytest
from openjarvis.core.config import JarvisConfig, load_config
PRESETS_DIR = (
Path(__file__).resolve().parents[2]
/ "configs"
/ "openjarvis"
/ "examples"
)
def _preset_paths() -> list[Path]:
return sorted(PRESETS_DIR.glob("*.toml"))
def test_presets_directory_exists() -> None:
assert PRESETS_DIR.is_dir(), f"presets dir missing: {PRESETS_DIR}"
def test_at_least_one_preset_ships() -> None:
assert _preset_paths(), f"no preset .toml files in {PRESETS_DIR}"
@pytest.mark.parametrize(
"preset_path",
_preset_paths(),
ids=lambda p: p.stem,
)
def test_preset_loads(preset_path: Path) -> None:
cfg = load_config(path=preset_path)
assert isinstance(cfg, JarvisConfig)
# A preset must at least name an engine and an agent — those are the two
# slots `jarvis init` expects to be populated for a working first run.
assert cfg.engine.default, f"{preset_path.stem}: engine.default is empty"
assert cfg.agent.default_agent, (
f"{preset_path.stem}: agent.default_agent is empty"
)
Generated
+1 -1
View File
@@ -4973,7 +4973,7 @@ wheels = [
[[package]]
name = "openjarvis"
version = "0.1.1"
version = "1.0.0"
source = { editable = "." }
dependencies = [
{ name = "click" },