mirror of https://github.com/open-jarvis/OpenJarvis.git synced 2026-07-30 19:02:16 +00:00

Files

T

Jon Saad-FalconandClaude Opus 4.6 880e3f67f2 feat: feature gap closure — streaming SDK, parallel tools, structured output, Docker sandbox, session checkpoints, 5 hero demos

Closes the remaining feature gaps vs Qwen-Agent, CoPaw, Google ADK, Apple FM SDK, and LFM2:

SDK:
- Add ask_stream() and ask_full_stream() async generator methods to Jarvis class

Orchestrator:
- Parallel tool execution via ThreadPoolExecutor (parallel_tools=True by default)

Engine:
- ResponseFormat dataclass for structured output / JSON mode
- OpenAI, Anthropic, Google, and Ollama structured output support

Tools:
- DockerCodeInterpreterTool (code_interpreter_docker) with sandboxed execution
- sandbox-docker optional dependency

Rust:
- Session checkpointing with checkpoint(), rewind(), list_checkpoints()

Examples (5 hero demo apps):
- browser_assistant — web browsing agent
- security_scanner — project security auditor
- daily_digest — morning news briefing
- doc_qa — document Q&A with memory indexing
- multi_model_router — cost-optimized model routing

Docs:
- 10 copy-paste code snippets page
- "What You Can Build" quickstart section

Tests: 275 Python tests pass, 392 Rust tests pass, lint clean.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

2026-03-08 04:47:24 +00:00

multi_model_router.py

feat: feature gap closure — streaming SDK, parallel tools, structured output, Docker sandbox, session checkpoints, 5 hero demos

2026-03-08 04:47:24 +00:00

README.md

feat: feature gap closure — streaming SDK, parallel tools, structured output, Docker sandbox, session checkpoints, 5 hero demos

2026-03-08 04:47:24 +00:00

README.md

Multi-Model Router

Route queries to the cheapest capable model using OpenJarvis's learning/routing system. Simple queries go to small fast models; complex code or math queries go to larger models.

Requirements

OpenJarvis installed (pip install openjarvis or uv sync --extra dev)
An inference engine running with multiple models available

Usage

python examples/multi_model_router/multi_model_router.py --help

# Simple query -> routes to smallest model
python examples/multi_model_router/multi_model_router.py --query "What is 2+2?"

# Complex reasoning -> routes to largest model
python examples/multi_model_router/multi_model_router.py \
    --query "Explain quantum entanglement step by step" --verbose

# Code query -> routes to code-specialized model
python examples/multi_model_router/multi_model_router.py \
    --query "def fibonacci(n):" --verbose

# Specify available models explicitly
python examples/multi_model_router/multi_model_router.py \
    --query "Summarize this paper" \
    --models "qwen3:0.6b,qwen3:8b,qwen3:32b"

# Use bandit (Thompson Sampling) strategy
python examples/multi_model_router/multi_model_router.py \
    --query "Solve the integral of x^2" --strategy bandit

How It Works

The script uses OpenJarvis's routing infrastructure from the learning pillar:

HeuristicRouter (default) -- rule-based routing that analyzes the query for code patterns, math keywords, length, and complexity to pick the right model tier. Short simple queries go to the smallest model; code and math queries go to larger or specialized models.
BanditRouterPolicy -- Thompson Sampling multi-armed bandit that learns which model performs best for each query class over time.

Both routers use build_routing_context() to extract query features (length, has_code, has_math) and then select from the available model pool. Use --verbose to see the routing decision details.