Final piece — make the ported paradigms runnable end-to-end through the
same flow as the hybrid harness:
- `runner.py`: `python -m openjarvis.agents.hybrid.runner --cell <name>`.
Loads cells from packaged registry TOMLs, constructs the registered
agent via AgentRegistry, iterates tasks from OpenJarvis's existing
GAIADataset / SWEBenchDataset, scores via gaia exact-match or the new
Modal-backed SWE-bench harness scorer, writes
`results.jsonl`+`summary.json` in the same shape as the hybrid harness
so existing rescore / dashboard scripts work unmodified. Supports
resume (drops errored rows), per-cell flock, ThreadPoolExecutor
concurrency, and the same per-row heartbeat format.
- `registry/{advisors,archon,conductor,minions,skillorchestra,toolorchestra}.toml`:
35 cells ported verbatim from the hybrid harness — same models, same N,
same `method_cfg` — so OpenJarvis runs should reproduce the numbers in
`hybrid-local-cloud-compute/docs/results.md`. ToolOrchestra cells
rewritten from the original NotImplementedError stubs to point at the
prompted port (cloud-as-orchestrator, no Nemotron-Orchestrator-8B).
- `scripts/new_experiment.sh`: mirror of hybrid's scaffolder. Maps
shortnames (`qwen3.5-27b`, `claude-opus-4-7`, …) to full HF ids +
endpoints, picks method-appropriate `method_cfg` defaults, appends a
`[cells.<name>]` block to the right registry TOML.
- `README.md`: quickstart, paradigm taxonomy, reproducibility table
pointing at the hybrid results for headline numbers.
End-to-end smoke: `python -m openjarvis.agents.hybrid.runner --help` works,
all 6 agents register, all 35 cells load.
Adds `SWEBenchHarnessScorer` alongside the existing structural-only
scorer. Hands one prediction at a time to `python -m swebench.harness.
run_evaluation`, parses the report JSON it writes, returns the resolved
bool. Backend selectable via `SWEBENCH_BACKEND` env var
(modal default, docker fallback).
Carries forward the two upstream-swebench patches from the hybrid
harness, both idempotent and lazy:
1. **Modal cgroup-v2 fix** — `swebench/harness/modal_eval/run_evaluation_modal.py`
writes to `/sys/fs/cgroup/cpu/cpu.shares` (cgroup v1). Modal v2 sandboxes
are cgroup v2 — the path doesn't exist and every sandbox died on the
write. This was the root cause of "Modal harness failing 100%"; wrapping
`set_cpu_quota` in try/except fixes it.
2. **Rescore `*_ids` fix** — older swebench wrote `resolved_instances` as
a list; current swebench puts the count there and the actual ids in
`resolved_ids`. Reading `*_ids` first; falling back to the list-typed
legacy field if the user is on an older install.
Patch extraction supports the same three formats as the hybrid harness:
fenced `\`\`\`diff`, fenced raw (`\`\`\`\ndiff --git`), and unfenced
`diff --git`.
Inference-only port of NVlabs ToolOrchestra (arXiv:2511.21689). The
paper's RL-trained Nemotron-Orchestrator-8B is NOT in the loop — the
hybrid harness adapter is a documented stub for exactly that reason
(needs a separate vLLM srun for the 8B checkpoint, a FAISS wiki
retriever, a Tavily key, and an upstream refactor).
This port keeps the same scope discipline: a cloud model plays the
orchestrator role and dispatches to a mixed pool of workers (local
Qwen for cheap extraction, Anthropic web_search for unknowns, Opus
4.7 / gpt-5-mini for hard reasoning or final synthesis). Reactive
multi-turn loop emitting JSON action objects per turn; forces a
final answer at the budget cap; hard fallback to the strongest worker
on persistent parse failure.
Treat results as preliminary until a real Orchestrator-8B deployment
exists — included here so OpenJarvis has registry entries for all six
paradigms and so the distillation pipeline can slot ToolOrchestra in
alongside the rest.
Registered as `toolorchestra`. The hybrid harness `toolorchestra_adapter.py`
remains a NotImplementedError stub; this is the first runnable version.
Inference-time skill-aware router from arXiv:2602.19672. The paper's
4-phase explore/learn/select pipeline is out of scope (needs multi-model
serving + FRAMES wiki retriever + a multi-hour learning loop) — the
hybrid harness reproduced only the deployment-time `select_agent()`
path with a hand-curated skill taxonomy + per-agent competence priors
calibrated to Qwen3.5-27B-FP8 vs Opus 4.7. This port keeps that scope.
Per task: Opus reads the question, picks skill weights from a 7-skill
catalog (factual_recall, multi_step_reasoning, arithmetic, web_grounding,
long_text_extraction, format_compliance, code_or_logic), scores each
agent under `score = weighted_competence - 0.5 * avg_cost`, routes to
the winner. JSON schema enforced via Anthropic `output_config`; balanced-
brace fallback if the SDK ignores the schema. Soft-fail row on
unparseable JSON to match hybrid's `err=1` on n=30.
Hybrid harness result: `skillorchestra-gaia-qwen27b-opus-30` = 0.500 acc,
$0.02/task — 30× cheaper than baseline-cloud (0.567 / $0.66) at ~7pp
lower accuracy. Best cost-efficient GAIA paradigm.
Registered as `skillorchestra`. Ported from
`hybrid-local-cloud-compute/adapters/skillorchestra_adapter.py`.
Layered (generator → ranker → fuser) sampling: K local generators propose
candidates, cloud ranker scores them, cloud fuser synthesizes the final
answer. Paper: arXiv:2409.15254. Two presets: `ensemble_rank_fuse` (the
full layered pipeline) and `single_local` (debug shim).
Carries the same patch story as the hybrid adapter:
- Eager-import stubs for groq/google/litellm so Archon's `utils.py` loads
in the OpenJarvis venv without those deps.
- Custom `vllm_local` model_type wired into Archon's GENERATE_MAP via a
per-run closure (endpoint can vary per cell).
- OpenAI / Anthropic generators replaced with token-tallying wrappers so
we can charge cost — Archon ignores `usage` by default.
- Opus 4.7 temperature stripping at the SDK level.
Archon library is lazy-imported. The hybrid clone at
`hybrid-local-cloud-compute/external/Archon/src` is the default
location; override with `ARCHON_SRC` env var.
Hybrid harness results: `archon-gaia-qwen27b-opus-ensemble-K5-30` =
0.333 acc, $0.20/task; `archon-gaia-qwen27b-singlelocal-30` = 0.033 acc.
Underperforms baseline-cloud on GAIA — included for completeness, not as
a recommended paradigm.
Registered as `archon`. Ported from
`hybrid-local-cloud-compute/adapters/archon_adapter.py`.
Cloud supervisor decomposes + reads back; local worker(s) do bulk
reading. Two modes: `minion` (single worker, default) / `minions`
(parallel workers + aggregator).
Carries forward every compatibility patch from the hybrid adapter so the
n=500 numbers transfer:
- Strip `temperature` for Opus 4.7+ (rejected with 400).
- Inject server-side `output_config` JSON schema per supervisor turn
(first-turn `{reasoning, message}` vs conversation-turn anyOf
`{decision, message|answer}`), picked by sniffing the prompt for
`"decision": "provide_final_answer"`.
- Replace `minions._extract_json` with a JSON-first wrapper.
- Inject `timeout=600`/`max_retries=5` into `anthropic.Anthropic()` so
Minions's bare-client construction doesn't 60s-timeout under SWE-bench
concurrency=8.
- GAIA-only `web_search` prefetch so the worker has a real doc to read.
Soft-fail row on JSONDecodeError/BadRequestError/prompt-too-long — same
deterministic-failure handling that produced the n=165 GAIA `err=6` rows
in the hybrid harness without crashing the cell.
`minions` library import is lazy; install via
`uv pip install -e /matx/u/aspark/hybrid-local-cloud-compute/external/minions`.
Hybrid harness results being reproduced:
- `minions-swebenchverified-qwen27b-opus-500` = 0.274 acc, $0.09/task
(+3.8pp vs baseline-cloud at 10× cheaper — cleanest paradigm-validation
result in the harness).
- `minions-gaia-qwen27b-opus-165` = 0.576 acc, $0.67/task
(~tied with baseline-cloud at 1.6× cheaper).
Registered as `minions`. Ported from
`hybrid-local-cloud-compute/adapters/minions_adapter.py`.
Inference-only Stage-1 repro: zero-shot cloud planner substitutes for the
paper's RL-trained Qwen2.5-7B conductor. Emits 3 JSON lists
(model_id/subtasks/access_list), executor runs ≤5 steps over a worker pool.
Two-tier parse fallback: JSON → Python-literal regex → single call to the
strongest worker. Worker pool defaults to {local Qwen if vLLM up, Opus 4.7,
gpt-5-mini}; override via `cfg["workers"]`.
Hybrid harness result: `conductor-swebenchverified-opusplan-30` = 0.367 acc
($0.22/task) vs baseline-cloud 0.267 ($3.36/task) — +10pp at ~15× cheaper.
Registered as `conductor`. Ported from
`hybrid-local-cloud-compute/adapters/conductor_adapter.py`.
Three-step executor (cloud) → advisor (local) → executor (cloud) loop from
arXiv:2510.02453. Inference-only: the paper's RL-trained advisor would
land higher; this matches the hybrid harness's `advisors-gaia-qwen9b-opus-30`
cell at 0.533 acc / $0.02 per task.
Registered as `advisors`. Local model id auto-resolves against vLLM's
`/v1/models` so cell configs naming Qwen3.5-9B don't 404 when the server
is actually serving 27B-FP8.
Ported from `hybrid-local-cloud-compute/adapters/advisors_adapter.py`.
Add `openjarvis.agents.hybrid` — the home for the local+cloud paradigms
being ported from `/matx/u/aspark/hybrid-local-cloud-compute`. This commit
ships only the shared base (no paradigm yet):
- `_base.LocalCloudAgent`: ABC over `BaseAgent` with `_call_anthropic`,
`_call_openai`, `_call_vllm` SDK helpers; standardized AgentResult
metadata shape (tokens_local/tokens_cloud/cost_usd/latency_s/traces);
Opus 4.7 temperature stripping; GPT-5 family `max_completion_tokens`;
soft-fail row helper for deterministic adapter failures.
- `_prices.py`: ported verbatim from hybrid's `prices.py` so the n=500
cost numbers stay aligned with `hybrid-local-cloud-compute/docs/results.md`.
- `_prompts.py`: GAIA + SWE-bench-Verified answer-format instructions,
same `format_prompt(task)` dispatch as `benches/__init__.py`.
- `agents/__init__.py`: wire the new subpackage so its sub-modules
register on package import.
Hybrid harness stays untouched — this is the OpenJarvis-native port.
- calculator: add `ln` alias for `math.log`, replace `^` with `**` before
AST parsing so caret-as-power works, convert SyntaxError → ValueError for
consistent error handling, and return `math.inf` on division by zero
(matching the documented meval behaviour) instead of raising an exception
- file_read / file_write: replace the hardcoded `str(path).startswith(str(d) + "/")
guard with `Path.is_relative_to()` so allowed-directory checks work on
Windows (and any OS whose separator is not `/`)
- git_tool: catch `NotADirectoryError` raised by `subprocess.run` on Windows
when `cwd` points to a non-existent path, returning a proper error result
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Robby Manihani <manihani@stanford.edu>
* fix: close KnowledgeStore connections in connectors_router endpoints
KnowledgeStore opens a persistent SQLite connection in __init__ but was
never closed at the three instantiation sites in connectors_router.py,
leaking one file descriptor (plus its WAL handle) per call. The worst
offender is _connector_summary(), which is invoked per-connector on
every GET /connectors poll from the frontend — after ~100 polls the
backend hits the default macOS per-process FD limit (256) and asyncio's
accept() starts failing with OSError: [Errno 24] Too many open files.
Add __enter__/__exit__ to KnowledgeStore (close() already existed) and
wrap all three instantiation sites in `with` blocks so the connection
is released deterministically when the scope exits.
- connectors/store.py: add context manager protocol
- server/connectors_router.py: use `with KnowledgeStore() as store:`
in _connector_summary, _ingest, and _run_sync
* test: cover KnowledgeStore context manager + connectors_router close leak
- test_store.py: add two tests for the new __enter__/__exit__ protocol
verifying the connection closes on normal exit and on exceptions.
- test_connectors_router.py: add a regression test that monkeypatches
KnowledgeStore.close to count invocations and asserts every store
opened by GET /v1/connectors is paired with a close.
Also fix a pre-existing bug in the test_connectors_router fixture: the
router is created with prefix="/v1/connectors" internally (line 92 of
connectors_router.py), and the fixture was wrapping it again with
prefix="/v1", producing "/v1/v1/connectors". All 6 existing router
tests were failing with 404 before this fix. 5 of them now pass; the
remaining failure (test_trigger_sync) is an unrelated KeyError on a
missing response field and is out of scope for this PR.
* test: drop connectors_router regression test — skipped in CI anyway
The regression test added in the previous commit relies on FastAPI
being importable, but CI's dev-extra install does not include fastapi
(it lives in the `server` optional group). All tests in
test_connectors_router.py silently skip on CI with "fastapi not
installed", so the regression test would never actually execute there.
Revert test_connectors_router.py to the upstream main version. The
fixture bug I fixed (double prefix) and the connection-leak regression
test both deserve a separate PR scoped to test infrastructure — that
PR should either add fastapi to dev deps, split server tests into
their own CI job, or both.
The KnowledgeStore context manager tests in test_store.py are kept
because they have no fastapi dependency and will run in CI.
* feat(mining): add Pearl model conversion workflow
* test(mining): tolerate missing Docker device request type
* docs(mining): record Gemma and Qwen conversion evidence
* docs(mining): record Qwen local validation evidence
---------
Co-authored-by: krypticmouse <herumbshandilya123@gmail.com>
routing.types appears to have been an in-progress module that was
consolidated into routing.router before it was created. The lazy
import inside the router policy block was silently caught by the
surrounding try/except, causing the policy to never activate and
the executor to always fall back to the configured model regardless
of any router_policy setting.
Corrects the import to openjarvis.learning.routing.router where
build_routing_context is defined and exported.
The desktop app's chat-completions stream relies on a raw browser
fetch from the Tauri webview to the FastAPI backend, so it is
subject to CORS. The default allowlist only contained
`tauri://localhost`, which is the production webview origin on
macOS, Linux, and iOS. On Windows and Android, Tauri 2 serves the
app from `http://tauri.localhost` (or `https://tauri.localhost`
when `windows.useHttpsScheme` is enabled), so the preflight for
`POST /v1/chat/completions` was rejected and the streaming fetch
threw `TypeError: Failed to fetch` before any byte was read.
Symptom users reported: in the Logs tab the request appears to
succeed up to "Request sent", followed immediately by
"Stream error: Failed to fetch" and "Response: 22 chars" -- which
is exactly the length of the synthesized fallback string
`Error: Failed to fetch` written by InputArea.tsx when streamChat
throws.
Add `http://tauri.localhost` and `https://tauri.localhost` to both
default origin lists (`ServerConfig.cors_origins` and the
`create_app` fallback), and add a regression test that drives a
real preflight against `/v1/chat/completions` for each of the
three Tauri origin schemes.
Browser model loading kept working because it is routed through
the Rust `tauriInvoke('fetch_models')` command, which is a
server-to-server HTTP call not subject to browser CORS -- only
streaming chat goes through the webview's fetch.
Removes the pull_request trigger so the workflow only runs on explicit
@claude mentions or manual workflow_dispatch.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
server/app.py defaulted to allow_origins=["*"] combined with
allow_credentials=True when cors_origins was not passed. That
combination is invalid per the CORS spec — browsers reject it — and
when used loosely it signals that any cross-origin request with
credentials is allowed, exposing session cookies and auth headers.
The reference config at configs/openjarvis/config.toml binds to
0.0.0.0 without setting cors_origins, so this fallback fired in every
default deployment.
Default to a closed list (Vite dev origin + Tauri webview) instead of
the wildcard. Users who actually need a custom origin already pass it
via cors_origins.
Closes#222
The dashboard at /dashboard renders blank once the Rust security module
is compiled (uv run maturin develop ...). The middleware sets
"Content-Security-Policy: default-src 'self'", which blocks the inline
scripts and styles the dashboard HTML relies on.
Loosen the policy to allow 'unsafe-inline' and 'unsafe-eval' so the
dashboard works once the security middleware is active. The same string
is exported via SECURITY_HEADERS for tests, so update both call sites.
Closes#261
configs/openjarvis/config.toml is the file users copy as their starting
configuration. It explicitly set [security] enabled = false (with a
comment about eval performance) — meaning users who follow the
quickstart unknowingly run with GuardrailsEngine, InjectionScanner,
CapabilityPolicy, and rate limiting all disabled.
The SecurityConfig.enabled default in code is True, so removing the
override here lets every install ship with security on. Eval workflows
that genuinely need it disabled can opt out via their own config.
Closes#224
SQLite FTS5 BM25 scores collapse toward zero on small corpora — when
every indexed document contains the query term, IDF approaches zero so
scores return ~1e-6. The default context_min_score = 0.1 silently
filters out every result before injection, so memory is stored but
never reaches the model on a fresh install.
Lowering both defaults to 0.0 lets retrieved context flow through;
users who want strict filtering can still set [memory]
context_min_score in config.toml.
Closes#262
uv.lock pinned mlx-lm==0.29.1, which predates the qwen3_5 architecture.
Running mlx_lm.server with mlx-community/Qwen3.5-* models therefore
fails with 'Model type qwen3_5 not supported.' on every uv sync, even
when pyproject.toml's loose constraint (>=0.19) would otherwise allow
a newer release.
Tighten the inference-mlx extra to mlx-lm>=0.31.1 (the first release
with Qwen3.5 support) and regenerate uv.lock — resolves to mlx-lm
0.31.3.
The lockfile diff is large because uv produces a single cross-platform
lock and adds 'split' entries for a handful of transitive deps
(transformers, vllm, huggingface-hub, mistral-common, etc.). Every
split entry is marker-gated to:
python_full_version >= '3.14' and sys_platform != 'linux'
and sys_platform != 'win32'
i.e. macOS on Python 3.14+ — the very subset that actually pulls in
mlx-lm 0.31.x. Linux, Windows, and Python <= 3.13 paths keep the
existing pins (transformers 4.57.6, vllm 0.17.1, …).
Closes#191
Andrew Park