diff --git a/.claude/rules/evals.md b/.claude/rules/evals.md new file mode 100644 index 00000000..d88e3a73 --- /dev/null +++ b/.claude/rules/evals.md @@ -0,0 +1,27 @@ +# Evals Framework + +## Overview +The eval framework lives in `src/openjarvis/evals/` and is run via: +```bash +uv run python -m openjarvis.evals --config src/openjarvis/evals/configs/.toml +``` + +## Structure +- **CLI**: `evals/cli.py` — command-line entry point +- **Core**: `evals/core/` — config loading (`config.py`), types (`types.py`), runner +- **Configs**: `evals/configs/` — TOML config files defining eval runs +- **Datasets**: `evals/datasets/` — dataset loaders (one file per benchmark) +- **Scorers**: `evals/scorers/` — scoring logic (one file per benchmark) + +## Available Datasets +SuperGPQA, GPQA, MMLU-Pro, MATH-500, GAIA, SWE-bench, FRAMES, SimpleQA, TerminalBench, PaperArena, LifelongAgent, and more. + +## Config Format +Eval configs are TOML files that specify: +- Model / engine to evaluate +- Dataset(s) to run +- Scoring method +- Number of samples, seeds, and other parameters + +## Lint +E501 (line length) is relaxed for `evals/datasets/*.py` and `evals/scorers/*.py`. diff --git a/.claude/rules/frontend-dev.md b/.claude/rules/frontend-dev.md new file mode 100644 index 00000000..d1f54db7 --- /dev/null +++ b/.claude/rules/frontend-dev.md @@ -0,0 +1,21 @@ +# Frontend / Desktop Development + +## Stack + +- **Frontend**: Vite-based web app in `frontend/` +- **Desktop**: Tauri app (Rust + web frontend) in `desktop/` + +## Commands + +```bash +# Frontend +cd frontend && npm install && npm run dev + +# Desktop (Tauri) +cd desktop && npm install && npm run tauri dev +``` + +## Notes + +- The desktop app wraps the frontend via Tauri, which uses a Rust backend with a webview. +- Frontend changes are reflected in the desktop app automatically during development. diff --git a/CLAUDE.md b/.claude/rules/python-dev.md similarity index 56% rename from CLAUDE.md rename to .claude/rules/python-dev.md index 6d15d4b5..ab486201 100644 --- a/CLAUDE.md +++ b/.claude/rules/python-dev.md @@ -1,55 +1,4 @@ -# CLAUDE.md - -This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. - -## Project Overview - -OpenJarvis is a modular AI assistant backend / research framework for on-device AI systems. It is organized around **five composable "pillars"** that are wired together via a config-driven `JarvisSystem` composition layer. - -## Build & Development Commands - -**Package manager:** [uv](https://github.com/astral-sh/uv) (lock file `uv.lock` is tracked for reproducibility) - -```bash -# Install core + dev dependencies -uv sync --extra dev - -# Lint (ruff, rules: E/F/I/W, target Python 3.10) -uv run ruff check src/ tests/ - -# Run full test suite -uv run pytest tests/ -v --tb=short - -# Run a single test file / single test -uv run pytest tests/agents/test_native_react.py -v -uv run pytest tests/agents/test_native_react.py::test_function_name -v - -# Run tests by marker (live, cloud, nvidia, amd, apple, slow) -uv run pytest -m "not live and not cloud" tests/ - -# CLI entry point -uv run jarvis --help - -# Run evals -uv run python -m openjarvis.evals --config src/openjarvis/evals/configs/.toml - -# Docs (MkDocs Material) -uv sync --extra docs -uv run mkdocs serve -``` - -**Rust workspace** (in `rust/`): -```bash -cd rust -cargo clippy --workspace --all-targets -- -D warnings -cargo test --workspace -``` - -**Frontend / Desktop** (Tauri + Vite, in `frontend/` and `desktop/`): -```bash -cd frontend && npm install && npm run dev -cd desktop && npm install && npm run tauri dev -``` +# Python Development ## Architecture: The Five Pillars @@ -72,7 +21,6 @@ Improvement methodologies: router policies (heuristic, bandit, trace-based), SFT ### Supporting Systems - **Core** (`core/`): `RegistryBase` pattern (decorator-based registration), types (`Message`, `Conversation`, `ToolCall`), config loader with hardware detection, `EventBus`. -- **Evals** (`evals/`): Benchmark framework with TOML configs. Datasets: SuperGPQA, GPQA, MMLU-Pro, MATH-500, GAIA, SWE-bench, FRAMES, SimpleQA, TerminalBench, PaperArena, etc. Run via `python -m openjarvis.evals`. - **Channels** (`channels/`): Chat platform integrations (Telegram, Discord, Slack, WhatsApp, Signal, IRC, Matrix, etc.). - **Telemetry** (`telemetry/`): GPU monitoring, energy measurement (NVIDIA/AMD/Apple/RAPL), latency instrumentation, vLLM metrics. - **Traces** (`traces/`): Execution trace recording for analysis. @@ -80,7 +28,6 @@ Improvement methodologies: router policies (heuristic, bandit, trace-based), SFT - **Security** (`security/`): PII scanning, capability policies. - **Server** (`server/`): FastAPI REST API. - **SDK** (`sdk.py`): High-level `Jarvis` and `JarvisSystem` classes, `MemoryHandle` for memory operations. -- **Rust** (`rust/`): Parallel Rust implementation with PyO3 bindings. Workspace crates mirror Python pillars (core, engine, agents, tools, learning, telemetry, traces, security, mcp, python). ## Key Patterns @@ -88,10 +35,3 @@ Improvement methodologies: router policies (heuristic, bandit, trace-based), SFT - **Optional dependencies**: Heavy deps are extras in `pyproject.toml` (e.g., `inference-cloud`, `memory-faiss`, `channel-telegram`). Import failures are caught with try/except so the core stays lightweight. - **OpenAI-compatible**: All engines expose an OpenAI-format chat completions interface. `messages_to_dicts()` in `engine/_base.py` handles conversion. - **Config-driven**: TOML configs control everything. `load_config()` detects hardware, fills defaults, then overlays user overrides. - -## Testing Conventions - -- Tests mirror `src/` structure under `tests/`. -- Markers: `live` (needs running engine), `cloud` (needs API keys), `nvidia`/`amd`/`apple` (GPU-specific), `slow`. -- `conftest.py` provides hardware fixtures (`hardware_nvidia`, `hardware_apple`, etc.) and `mock_engine` factory. -- E501 line length is relaxed for `evals/datasets/*.py` and `evals/scorers/*.py`. diff --git a/.claude/rules/rust-dev.md b/.claude/rules/rust-dev.md new file mode 100644 index 00000000..b1770d90 --- /dev/null +++ b/.claude/rules/rust-dev.md @@ -0,0 +1,40 @@ +# Rust Development + +## Workspace Layout + +The Rust implementation lives in `rust/` and mirrors the Python pillars: + +``` +rust/ + Cargo.toml # workspace root + crates/ + core/ # core types, registry + engine/ # inference engine abstraction + agents/ # agent implementations + tools/ # tool implementations + learning/ # learning orchestrator + telemetry/ # GPU monitoring, energy measurement + traces/ # execution trace recording + security/ # PII scanning, capability policies + mcp/ # Model Context Protocol + python/ # PyO3 bindings exposing Rust to Python +``` + +## Commands + +```bash +cd rust + +# Lint — treat warnings as errors +cargo clippy --workspace --all-targets -- -D warnings + +# Run tests +cargo test --workspace + +# Build +cargo build --workspace +``` + +## PyO3 Bindings + +The `python` crate (`rust/crates/python/`) exposes Rust functionality to the Python package via PyO3. When modifying Rust APIs that are bound to Python, update both the Rust implementation and the PyO3 binding layer. diff --git a/.claude/rules/testing.md b/.claude/rules/testing.md new file mode 100644 index 00000000..af1b36d1 --- /dev/null +++ b/.claude/rules/testing.md @@ -0,0 +1,22 @@ +# Testing Conventions + +## Structure +- Tests mirror `src/` structure under `tests/`. +- Run tests: `uv run pytest tests/ -v --tb=short` +- Run a single file: `uv run pytest tests/agents/test_native_react.py -v` +- Run a single test: `uv run pytest tests/agents/test_native_react.py::test_function_name -v` + +## Markers +- `live` — needs a running inference engine +- `cloud` — needs API keys (OpenAI, Anthropic, etc.) +- `nvidia` / `amd` / `apple` — GPU-specific tests +- `slow` — long-running tests + +Skip markers for local testing: `uv run pytest -m "not live and not cloud" tests/` + +## Fixtures +- `conftest.py` provides hardware detection fixtures (`hardware_nvidia`, `hardware_apple`, etc.) and a `mock_engine` factory. +- All registries are auto-cleared between tests via a `conftest.py` fixture — no manual cleanup needed. + +## Lint Exceptions +- E501 (line length) is relaxed for `evals/datasets/*.py` and `evals/scorers/*.py`. diff --git a/.claude/rules/workflow.md b/.claude/rules/workflow.md new file mode 100644 index 00000000..c45cd280 --- /dev/null +++ b/.claude/rules/workflow.md @@ -0,0 +1,23 @@ +# Workflow Rules + +## Task Completion Contract +- A task is not done until tests pass and lint is clean (`uv run ruff check src/ tests/`). +- Never leave stubs, TODOs, or placeholder implementations unless explicitly told to. +- Run the relevant tests before declaring a task complete. + +## Context Recovery After Compaction +- After context compaction, re-read your task plan and the relevant source files before continuing. +- Do not rely on memory of file contents from before compaction — re-read them. + +## Research vs. Implementation +- If unsure about the right approach, research first: read the relevant code, check tests, understand the pattern. +- Once you've decided on an approach, implement it without second-guessing. Don't explore alternatives mid-implementation. + +## Precision +- Make only the changes requested. Don't refactor surrounding code, add comments to unchanged code, or "improve" things beyond scope. +- When fixing a bug, understand the root cause before writing a fix. + +## Neutral Investigation +- When asked to find bugs or review code, report findings neutrally. +- Don't bias toward finding problems — if the code is correct, say so. +- Don't inflate minor style issues into bugs. diff --git a/.gitignore b/.gitignore index 82ab9ef4..ee6ba835 100644 --- a/.gitignore +++ b/.gitignore @@ -73,6 +73,9 @@ target/ # Claude plan artifacts docs/plans/ +# Claude Code project instructions (per-developer) +CLAUDE.md + # Tauri auto-generated schemas **/src-tauri/gen/schemas/ diff --git a/README.md b/README.md index 032ebeb9..00d2c502 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,7 @@

Project - Docs + Docs Python License

@@ -13,7 +13,7 @@ --- -> **[Documentation](https://hazyresearch.stanford.edu/OpenJarvis/)** +> **[Documentation](https://open-jarvis.github.io/OpenJarvis/)** > > **[Project Site](https://www.intelligence-per-watt.ai/)** @@ -76,7 +76,7 @@ From source, you need the Rust extension for full functionality (security, tools ```bash # 1. Clone and install Python deps -git clone https://github.com/HazyResearch/OpenJarvis.git +git clone https://github.com/open-jarvis/OpenJarvis.git cd OpenJarvis uv sync --extra dev