mirror of
https://github.com/open-jarvis/OpenJarvis.git
synced 2026-07-31 03:12:16 +00:00
13 incremental tasks covering structure cleanup, data migration, shim removal, engine consolidation, and init.py refactoring. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
783 lines
23 KiB
Markdown
783 lines
23 KiB
Markdown
# Codebase Simplification Implementation Plan
|
|
|
|
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
|
|
|
**Goal:** Simplify the OpenJarvis repository structure by reducing root sprawl, moving data files into the package, removing backward-compat shims, and consolidating repetitive code patterns.
|
|
|
|
**Architecture:** Incremental commits, each independently testable. Data TOML files move into `src/openjarvis/*/data/` as package data. The `evals/` framework becomes `src/openjarvis/evals/` with all internal imports rewritten from `evals.` to `openjarvis.evals.`. The `memory/` shim package is removed. Engine wrappers are consolidated.
|
|
|
|
**Tech Stack:** Python, TOML, hatch/hatchling build system, pytest, ruff
|
|
|
|
---
|
|
|
|
### Task 1: Remove get-pip.py from git tracking
|
|
|
|
**Files:**
|
|
- Delete: `get-pip.py`
|
|
- Modify: `.gitignore`
|
|
|
|
**Step 1: Add get-pip.py to .gitignore**
|
|
|
|
In `.gitignore`, add after the `# Project` section (after line 44):
|
|
|
|
```
|
|
get-pip.py
|
|
```
|
|
|
|
**Step 2: Remove get-pip.py from git tracking and delete**
|
|
|
|
Run:
|
|
```bash
|
|
git rm get-pip.py
|
|
```
|
|
|
|
**Step 3: Commit**
|
|
|
|
```bash
|
|
git add .gitignore
|
|
git commit -m "chore: remove get-pip.py from repository (2.2MB regenerable artifact)"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 2: Move Docker files to deploy/docker/
|
|
|
|
**Files:**
|
|
- Move: `Dockerfile` -> `deploy/docker/Dockerfile`
|
|
- Move: `Dockerfile.gpu` -> `deploy/docker/Dockerfile.gpu`
|
|
- Move: `Dockerfile.gpu.rocm` -> `deploy/docker/Dockerfile.gpu.rocm`
|
|
- Move: `Dockerfile.sandbox` -> `deploy/docker/Dockerfile.sandbox`
|
|
- Move: `docker-compose.yml` -> `deploy/docker/docker-compose.yml`
|
|
- Move: `docker-compose.gpu.rocm.yml` -> `deploy/docker/docker-compose.gpu.rocm.yml`
|
|
- Modify: `deploy/docker/docker-compose.yml` (update dockerfile paths)
|
|
- Modify: `deploy/docker/docker-compose.gpu.rocm.yml` (update dockerfile path)
|
|
|
|
**Step 1: Move all Docker files**
|
|
|
|
```bash
|
|
mv Dockerfile deploy/docker/Dockerfile
|
|
mv Dockerfile.gpu deploy/docker/Dockerfile.gpu
|
|
mv Dockerfile.gpu.rocm deploy/docker/Dockerfile.gpu.rocm
|
|
mv Dockerfile.sandbox deploy/docker/Dockerfile.sandbox
|
|
mv docker-compose.yml deploy/docker/docker-compose.yml
|
|
mv docker-compose.gpu.rocm.yml deploy/docker/docker-compose.gpu.rocm.yml
|
|
```
|
|
|
|
**Step 2: Update docker-compose.yml build contexts**
|
|
|
|
In `deploy/docker/docker-compose.yml`, change the `build` section from:
|
|
```yaml
|
|
build:
|
|
context: .
|
|
dockerfile: Dockerfile
|
|
```
|
|
to:
|
|
```yaml
|
|
build:
|
|
context: ../..
|
|
dockerfile: deploy/docker/Dockerfile
|
|
```
|
|
|
|
**Step 3: Update docker-compose.gpu.rocm.yml**
|
|
|
|
In `deploy/docker/docker-compose.gpu.rocm.yml`, change:
|
|
```yaml
|
|
dockerfile: Dockerfile.gpu.rocm
|
|
```
|
|
to:
|
|
```yaml
|
|
context: ../..
|
|
dockerfile: deploy/docker/Dockerfile.gpu.rocm
|
|
```
|
|
|
|
**Step 4: Commit**
|
|
|
|
```bash
|
|
git add -A deploy/docker/
|
|
git add -A Dockerfile* docker-compose*
|
|
git commit -m "refactor: move Docker files to deploy/docker/"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 3: Move recipes/ TOML data into package
|
|
|
|
**Files:**
|
|
- Move: `recipes/*.toml` -> `src/openjarvis/recipes/data/*.toml`
|
|
- Move: `recipes/operators/*.toml` -> `src/openjarvis/recipes/data/operators/*.toml`
|
|
- Move: `recipes/operators/*.md` -> `src/openjarvis/recipes/data/operators/*.md`
|
|
- Modify: `src/openjarvis/recipes/loader.py` (lines 15-16, update `_PROJECT_RECIPES_DIR`)
|
|
- Modify: `pyproject.toml` (add package data include)
|
|
- Test: `tests/recipes/` (run existing tests)
|
|
|
|
**Step 1: Create data directory and move files**
|
|
|
|
```bash
|
|
mkdir -p src/openjarvis/recipes/data/operators
|
|
mv recipes/*.toml src/openjarvis/recipes/data/
|
|
mv recipes/operators/*.toml src/openjarvis/recipes/data/operators/
|
|
mv recipes/operators/*.md src/openjarvis/recipes/data/operators/
|
|
rmdir recipes/operators
|
|
rmdir recipes
|
|
```
|
|
|
|
**Step 2: Update loader.py**
|
|
|
|
In `src/openjarvis/recipes/loader.py`, change line 16 from:
|
|
```python
|
|
_PROJECT_RECIPES_DIR = Path(__file__).resolve().parents[3] / "recipes"
|
|
```
|
|
to:
|
|
```python
|
|
_PROJECT_RECIPES_DIR = Path(__file__).resolve().parent / "data"
|
|
```
|
|
|
|
**Step 3: Run tests**
|
|
|
|
```bash
|
|
uv run pytest tests/recipes/ -v
|
|
```
|
|
Expected: All recipe tests pass.
|
|
|
|
**Step 4: Commit**
|
|
|
|
```bash
|
|
git add src/openjarvis/recipes/data/ src/openjarvis/recipes/loader.py
|
|
git add recipes/
|
|
git commit -m "refactor: move recipes/ TOML data into src/openjarvis/recipes/data/"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 4: Move templates/ agent TOML data into package
|
|
|
|
**Files:**
|
|
- Move: `templates/agents/*.toml` -> `src/openjarvis/templates/data/*.toml`
|
|
- Modify: `src/openjarvis/templates/agent_templates.py` (lines 68-71, update `_builtin_templates_dir`)
|
|
- Test: `tests/templates/` (run existing tests)
|
|
|
|
**Step 1: Create data directory and move files**
|
|
|
|
```bash
|
|
mkdir -p src/openjarvis/templates/data
|
|
mv templates/agents/*.toml src/openjarvis/templates/data/
|
|
rmdir templates/agents
|
|
rmdir templates
|
|
```
|
|
|
|
**Step 2: Update agent_templates.py**
|
|
|
|
In `src/openjarvis/templates/agent_templates.py`, change lines 68-71 from:
|
|
```python
|
|
def _builtin_templates_dir() -> Path:
|
|
"""Return the path to the built-in templates shipped with the package."""
|
|
# templates/agents/ at project root, 3 levels above this file
|
|
return Path(__file__).resolve().parents[3] / "templates" / "agents"
|
|
```
|
|
to:
|
|
```python
|
|
def _builtin_templates_dir() -> Path:
|
|
"""Return the path to the built-in templates shipped with the package."""
|
|
return Path(__file__).resolve().parent / "data"
|
|
```
|
|
|
|
**Step 3: Run tests**
|
|
|
|
```bash
|
|
uv run pytest tests/templates/ -v
|
|
```
|
|
Expected: All template tests pass.
|
|
|
|
**Step 4: Commit**
|
|
|
|
```bash
|
|
git add src/openjarvis/templates/data/ src/openjarvis/templates/agent_templates.py
|
|
git add templates/
|
|
git commit -m "refactor: move templates/agents/ TOML data into src/openjarvis/templates/data/"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 5: Move skills/builtin/ TOML data into package
|
|
|
|
**Files:**
|
|
- Move: `skills/builtin/*.toml` -> `src/openjarvis/skills/data/*.toml`
|
|
- Modify: `tests/skills/test_bundled_skills.py` (line 12, update `BUILTIN_DIR` path)
|
|
- Test: `tests/skills/` (run existing tests)
|
|
|
|
**Step 1: Create data directory and move files**
|
|
|
|
```bash
|
|
mkdir -p src/openjarvis/skills/data
|
|
mv skills/builtin/*.toml src/openjarvis/skills/data/
|
|
rmdir skills/builtin
|
|
rmdir skills
|
|
```
|
|
|
|
**Step 2: Update test_bundled_skills.py**
|
|
|
|
In `tests/skills/test_bundled_skills.py`, change line 12 from:
|
|
```python
|
|
BUILTIN_DIR = Path(__file__).resolve().parents[2] / "skills" / "builtin"
|
|
```
|
|
to:
|
|
```python
|
|
BUILTIN_DIR = Path(__file__).resolve().parents[2] / "src" / "openjarvis" / "skills" / "data"
|
|
```
|
|
|
|
**Step 3: Run tests**
|
|
|
|
```bash
|
|
uv run pytest tests/skills/ -v
|
|
```
|
|
Expected: All skill tests pass (20 bundled skills found).
|
|
|
|
**Step 4: Commit**
|
|
|
|
```bash
|
|
git add src/openjarvis/skills/data/ tests/skills/test_bundled_skills.py
|
|
git add skills/
|
|
git commit -m "refactor: move skills/builtin/ TOML data into src/openjarvis/skills/data/"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 6: Move operators/ TOML data into package
|
|
|
|
**Files:**
|
|
- Move: `operators/*.toml` -> `src/openjarvis/operators/data/*.toml`
|
|
- Modify: `src/openjarvis/cli/operators_cmd.py` (lines 35, 247, 274 — replace `Path("operators")` with package data path)
|
|
- Test: `tests/operators/` (run existing tests)
|
|
|
|
**Step 1: Create data directory and move files**
|
|
|
|
```bash
|
|
mkdir -p src/openjarvis/operators/data
|
|
mv operators/*.toml src/openjarvis/operators/data/
|
|
rmdir operators
|
|
```
|
|
|
|
**Step 2: Update operators_cmd.py**
|
|
|
|
The CLI currently checks `Path("operators")` (cwd-relative). We need to replace this with the package data directory. In `src/openjarvis/cli/operators_cmd.py`:
|
|
|
|
Add a helper at the top of the file (after imports):
|
|
```python
|
|
def _builtin_operators_dir() -> Path:
|
|
"""Return the path to built-in operator manifests shipped with the package."""
|
|
return Path(__file__).resolve().parents[1] / "operators" / "data"
|
|
```
|
|
|
|
Then update the three places that reference `Path("operators")`:
|
|
|
|
Line 35: Change `local_ops = Path("operators")` to `local_ops = _builtin_operators_dir()`
|
|
|
|
Line 247: Change `dirs = [DEFAULT_CONFIG_DIR / "operators", Path("operators")]` to `dirs = [DEFAULT_CONFIG_DIR / "operators", _builtin_operators_dir()]`
|
|
|
|
Line 274: Change `for d in [DEFAULT_CONFIG_DIR / "operators", Path("operators")]:` to `for d in [DEFAULT_CONFIG_DIR / "operators", _builtin_operators_dir()]:`
|
|
|
|
**Step 3: Run tests**
|
|
|
|
```bash
|
|
uv run pytest tests/operators/ -v
|
|
```
|
|
Expected: All operator tests pass.
|
|
|
|
**Step 4: Commit**
|
|
|
|
```bash
|
|
git add src/openjarvis/operators/data/ src/openjarvis/cli/operators_cmd.py
|
|
git add operators/
|
|
git commit -m "refactor: move operators/ TOML data into src/openjarvis/operators/data/"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 7: Move evals/ into src/openjarvis/evals/
|
|
|
|
This is the most complex migration. The `evals/` directory is a standalone Python package with internal imports like `from evals.core.config import ...`. All these must be rewritten to `from openjarvis.evals.core.config import ...`.
|
|
|
|
**Files:**
|
|
- Move: `evals/` -> `src/openjarvis/evals/` (entire directory tree)
|
|
- Modify: All `*.py` files in `src/openjarvis/evals/` (rewrite `from evals.` to `from openjarvis.evals.`)
|
|
- Modify: `src/openjarvis/cli/eval_cmd.py` (rewrite `from evals.` imports)
|
|
- Modify: `tests/evals/*.py` (rewrite `from evals.` imports)
|
|
- Modify: `pyproject.toml` (update ruff per-file-ignores paths)
|
|
|
|
**Step 1: Move the directory**
|
|
|
|
```bash
|
|
# Move evals/ into src/openjarvis/evals/
|
|
# First remove any existing __pycache__
|
|
find evals/ -type d -name __pycache__ -exec rm -rf {} + 2>/dev/null
|
|
mv evals/ src/openjarvis/evals/
|
|
```
|
|
|
|
**Step 2: Rewrite all internal imports in evals/**
|
|
|
|
Run a find-and-replace across all Python files in `src/openjarvis/evals/`:
|
|
```bash
|
|
find src/openjarvis/evals/ -name "*.py" -exec sed -i 's/from evals\./from openjarvis.evals./g; s/import evals\./import openjarvis.evals./g' {} +
|
|
```
|
|
|
|
**Step 3: Rewrite imports in eval_cmd.py**
|
|
|
|
In `src/openjarvis/cli/eval_cmd.py`, replace all occurrences of:
|
|
- `from evals.core.config import` -> `from openjarvis.evals.core.config import`
|
|
- `from evals.cli import` -> `from openjarvis.evals.cli import`
|
|
- `from evals.core.types import` -> `from openjarvis.evals.core.types import`
|
|
|
|
**Step 4: Rewrite imports in tests/evals/**
|
|
|
|
Run find-and-replace across test files:
|
|
```bash
|
|
find tests/evals/ -name "*.py" -exec sed -i 's/from evals\./from openjarvis.evals./g; s/import evals\./import openjarvis.evals./g' {} +
|
|
```
|
|
|
|
**Step 5: Update pyproject.toml**
|
|
|
|
Change lines 134-136 from:
|
|
```toml
|
|
[tool.ruff.lint.per-file-ignores]
|
|
"evals/datasets/*.py" = ["E501"]
|
|
"evals/scorers/*.py" = ["E501"]
|
|
```
|
|
to:
|
|
```toml
|
|
[tool.ruff.lint.per-file-ignores]
|
|
"src/openjarvis/evals/datasets/*.py" = ["E501"]
|
|
"src/openjarvis/evals/scorers/*.py" = ["E501"]
|
|
```
|
|
|
|
**Step 6: Update __main__.py**
|
|
|
|
In `src/openjarvis/evals/__main__.py`, the import was rewritten in step 2 but verify it now reads:
|
|
```python
|
|
"""Allow running as ``python -m openjarvis.evals``."""
|
|
|
|
from openjarvis.evals.cli import main
|
|
|
|
if __name__ == "__main__":
|
|
main()
|
|
```
|
|
|
|
**Step 7: Run tests**
|
|
|
|
```bash
|
|
uv run pytest tests/evals/ -v
|
|
```
|
|
Expected: All eval tests pass.
|
|
|
|
**Step 8: Lint check**
|
|
|
|
```bash
|
|
uv run ruff check src/openjarvis/evals/ tests/evals/ src/openjarvis/cli/eval_cmd.py
|
|
```
|
|
Expected: No import errors.
|
|
|
|
**Step 9: Commit**
|
|
|
|
```bash
|
|
git add src/openjarvis/evals/ tests/evals/ src/openjarvis/cli/eval_cmd.py pyproject.toml
|
|
git commit -m "refactor: move evals/ into src/openjarvis/evals/ as proper subpackage"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 8: Remove memory/ backward-compat shim package
|
|
|
|
**Files:**
|
|
- Delete: `src/openjarvis/memory/` (entire directory, 11 files)
|
|
- Modify: `src/openjarvis/cli/memory_cmd.py` (lines 15-16)
|
|
- Modify: `src/openjarvis/cli/ask.py` (lines 130, 304)
|
|
- Modify: `src/openjarvis/sdk.py` (lines 36, 59-60, 391, 431)
|
|
- Modify: `tests/tools/test_storage_stubs.py` (lines 68-69, 80)
|
|
- Modify: `tests/tools/test_retrieval.py` (line 7)
|
|
- Modify: `tests/cli/test_memory_cmd.py` (line 12)
|
|
- Modify: `tests/cli/test_ask_context.py` (lines 34, 55)
|
|
- Modify: `tests/test_integration_extended.py` (lines 402, 419)
|
|
- Delete: `tests/memory/` (entire directory — these test the shim, not the real backends)
|
|
|
|
**Step 1: Update src/ imports**
|
|
|
|
In `src/openjarvis/cli/memory_cmd.py`, change:
|
|
```python
|
|
from openjarvis.memory.chunking import ChunkConfig
|
|
from openjarvis.memory.ingest import ingest_path
|
|
```
|
|
to:
|
|
```python
|
|
from openjarvis.tools.storage.chunking import ChunkConfig
|
|
from openjarvis.tools.storage.ingest import ingest_path
|
|
```
|
|
|
|
In `src/openjarvis/cli/ask.py`, change all occurrences of:
|
|
- `from openjarvis.memory.context import` -> `from openjarvis.tools.storage.context import`
|
|
- `from openjarvis.memory.sqlite import` -> `from openjarvis.tools.storage.sqlite import`
|
|
|
|
In `src/openjarvis/sdk.py`, change:
|
|
- `from openjarvis.memory.sqlite import SqliteMemory` -> `from openjarvis.tools.storage.sqlite import SQLiteMemory as SqliteMemory`
|
|
- `from openjarvis.memory.chunking import ChunkConfig` -> `from openjarvis.tools.storage.chunking import ChunkConfig`
|
|
- `from openjarvis.memory.ingest import ingest_path` -> `from openjarvis.tools.storage.ingest import ingest_path`
|
|
- `from openjarvis.memory.context import ContextConfig, inject_context` -> `from openjarvis.tools.storage.context import ContextConfig, inject_context`
|
|
|
|
**Step 2: Update test imports**
|
|
|
|
In `tests/tools/test_storage_stubs.py`, change:
|
|
- `from openjarvis.memory._stubs import MemoryBackend as MB` -> `from openjarvis.tools.storage._stubs import MemoryBackend as MB`
|
|
- `from openjarvis.memory._stubs import RetrievalResult as RR` -> `from openjarvis.tools.storage._stubs import RetrievalResult as RR`
|
|
- `from openjarvis.memory.sqlite import SQLiteMemory as S2` -> `from openjarvis.tools.storage.sqlite import SQLiteMemory as S2`
|
|
|
|
In `tests/tools/test_retrieval.py`, change:
|
|
- `from openjarvis.memory._stubs import MemoryBackend, RetrievalResult` -> `from openjarvis.tools.storage._stubs import MemoryBackend, RetrievalResult`
|
|
|
|
In `tests/cli/test_memory_cmd.py`, change:
|
|
- `from openjarvis.memory.sqlite import SQLiteMemory` -> `from openjarvis.tools.storage.sqlite import SQLiteMemory`
|
|
|
|
In `tests/cli/test_ask_context.py`, change:
|
|
- `from openjarvis.memory.sqlite import SQLiteMemory` -> `from openjarvis.tools.storage.sqlite import SQLiteMemory`
|
|
|
|
In `tests/test_integration_extended.py`, change:
|
|
- `from openjarvis.memory.sqlite import SQLiteMemory` -> `from openjarvis.tools.storage.sqlite import SQLiteMemory`
|
|
- `from openjarvis.memory.bm25 import BM25Memory` -> `from openjarvis.tools.storage.bm25 import BM25Memory`
|
|
|
|
**Step 3: Update tests/memory/ imports and move to tests/tools/**
|
|
|
|
The `tests/memory/` files test the same backends as `tests/tools/test_storage_*.py`. Since they import from the shim, we need to rewrite their imports and merge them. However, since `tests/tools/` already has storage tests, the simplest approach is:
|
|
|
|
Rewrite all `tests/memory/*.py` imports from `openjarvis.memory.*` to `openjarvis.tools.storage.*`:
|
|
```bash
|
|
find tests/memory/ -name "*.py" -exec sed -i 's/from openjarvis\.memory\./from openjarvis.tools.storage./g' {} +
|
|
```
|
|
|
|
Then move the directory to be a subdirectory of tests/tools:
|
|
```bash
|
|
mv tests/memory tests/tools/memory_tests
|
|
```
|
|
|
|
Actually, the cleanest approach: just rewrite imports in-place and keep `tests/memory/` as-is. The tests are valuable and test different aspects than `tests/tools/`.
|
|
|
|
**Step 4: Delete the shim package**
|
|
|
|
```bash
|
|
rm -rf src/openjarvis/memory/
|
|
```
|
|
|
|
**Step 5: Run full test suite**
|
|
|
|
```bash
|
|
uv run pytest tests/memory/ tests/tools/ tests/cli/ tests/sdk/ -v
|
|
```
|
|
Expected: All tests pass.
|
|
|
|
**Step 6: Commit**
|
|
|
|
```bash
|
|
git add -A src/openjarvis/memory/ src/openjarvis/cli/memory_cmd.py src/openjarvis/cli/ask.py src/openjarvis/sdk.py
|
|
git add tests/memory/ tests/tools/ tests/cli/ tests/test_integration_extended.py
|
|
git commit -m "refactor: remove memory/ backward-compat shims, use tools.storage directly"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 9: Consolidate engine wrappers into single file
|
|
|
|
**Files:**
|
|
- Delete: `src/openjarvis/engine/vllm.py`
|
|
- Delete: `src/openjarvis/engine/sglang.py`
|
|
- Delete: `src/openjarvis/engine/llamacpp.py`
|
|
- Delete: `src/openjarvis/engine/mlx.py`
|
|
- Delete: `src/openjarvis/engine/lmstudio.py`
|
|
- Create: `src/openjarvis/engine/openai_compat_engines.py`
|
|
- Modify: `src/openjarvis/engine/__init__.py` (update imports)
|
|
- Test: Run `tests/engine/` tests
|
|
|
|
**Step 1: Write the consolidated engine file**
|
|
|
|
Create `src/openjarvis/engine/openai_compat_engines.py`:
|
|
|
|
```python
|
|
"""Data-driven registration of OpenAI-compatible inference engines."""
|
|
|
|
from openjarvis.core.registry import EngineRegistry
|
|
from openjarvis.engine._openai_compat import _OpenAICompatibleEngine
|
|
|
|
_ENGINES = {
|
|
"vllm": ("VLLMEngine", "http://localhost:8000"),
|
|
"sglang": ("SGLangEngine", "http://localhost:30000"),
|
|
"llamacpp": ("LlamaCppEngine", "http://localhost:8080"),
|
|
"mlx": ("MLXEngine", "http://localhost:8080"),
|
|
"lmstudio": ("LMStudioEngine", "http://localhost:1234"),
|
|
}
|
|
|
|
for _key, (_cls_name, _default_host) in _ENGINES.items():
|
|
_cls = type(
|
|
_cls_name,
|
|
(_OpenAICompatibleEngine,),
|
|
{"engine_id": _key, "_default_host": _default_host},
|
|
)
|
|
EngineRegistry.register(_key)(_cls)
|
|
globals()[_cls_name] = _cls
|
|
|
|
__all__ = [name for name, _ in _ENGINES.values()]
|
|
```
|
|
|
|
**Step 2: Update engine/__init__.py**
|
|
|
|
Replace lines 5-11 (the 5 individual imports) with a single import:
|
|
|
|
Change from:
|
|
```python
|
|
import openjarvis.engine.llamacpp # noqa: F401
|
|
import openjarvis.engine.mlx # noqa: F401
|
|
|
|
# Import engine modules to trigger @EngineRegistry.register() decorators
|
|
import openjarvis.engine.ollama # noqa: F401
|
|
import openjarvis.engine.sglang # noqa: F401
|
|
import openjarvis.engine.vllm # noqa: F401
|
|
```
|
|
|
|
To:
|
|
```python
|
|
# Import engine modules to trigger @EngineRegistry.register() decorators
|
|
import openjarvis.engine.ollama # noqa: F401
|
|
import openjarvis.engine.openai_compat_engines # noqa: F401
|
|
```
|
|
|
|
**Step 3: Delete the 5 individual wrapper files**
|
|
|
|
```bash
|
|
rm src/openjarvis/engine/vllm.py
|
|
rm src/openjarvis/engine/sglang.py
|
|
rm src/openjarvis/engine/llamacpp.py
|
|
rm src/openjarvis/engine/mlx.py
|
|
rm src/openjarvis/engine/lmstudio.py
|
|
```
|
|
|
|
**Step 4: Run tests**
|
|
|
|
```bash
|
|
uv run pytest tests/engine/ -v
|
|
```
|
|
Expected: All engine tests pass.
|
|
|
|
**Step 5: Commit**
|
|
|
|
```bash
|
|
git add src/openjarvis/engine/
|
|
git commit -m "refactor: consolidate 5 OpenAI-compat engine wrappers into single data-driven file"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 10: Refactor repetitive __init__.py imports
|
|
|
|
**Files:**
|
|
- Modify: `src/openjarvis/channels/__init__.py`
|
|
- Modify: `src/openjarvis/engine/__init__.py` (optional engines section)
|
|
- Test: Run `tests/channels/` and `tests/engine/` tests
|
|
|
|
**Step 1: Refactor channels/__init__.py**
|
|
|
|
Replace the entire file content with:
|
|
|
|
```python
|
|
"""Channel abstraction for multi-platform messaging."""
|
|
|
|
import importlib
|
|
|
|
from openjarvis.channels._stubs import (
|
|
BaseChannel,
|
|
ChannelHandler,
|
|
ChannelMessage,
|
|
ChannelStatus,
|
|
)
|
|
|
|
# Trigger registration of built-in channels.
|
|
# Each module uses @ChannelRegistry.register() — importing is sufficient.
|
|
_CHANNEL_MODULES = [
|
|
"telegram",
|
|
"discord_channel",
|
|
"slack",
|
|
"webhook",
|
|
"email_channel",
|
|
"whatsapp",
|
|
"signal_channel",
|
|
"google_chat",
|
|
"irc_channel",
|
|
"webchat",
|
|
"teams",
|
|
"matrix_channel",
|
|
"mattermost",
|
|
"feishu",
|
|
"bluebubbles",
|
|
"whatsapp_baileys",
|
|
"line_channel",
|
|
"viber_channel",
|
|
"messenger_channel",
|
|
"reddit_channel",
|
|
"mastodon_channel",
|
|
"xmpp_channel",
|
|
"rocketchat_channel",
|
|
"zulip_channel",
|
|
"twitch_channel",
|
|
"nostr_channel",
|
|
]
|
|
|
|
for _mod in _CHANNEL_MODULES:
|
|
try:
|
|
importlib.import_module(f".{_mod}", __name__)
|
|
except ImportError:
|
|
pass
|
|
|
|
__all__ = [
|
|
"BaseChannel",
|
|
"ChannelHandler",
|
|
"ChannelMessage",
|
|
"ChannelStatus",
|
|
]
|
|
```
|
|
|
|
**Step 2: Refactor optional engine imports in engine/__init__.py**
|
|
|
|
Replace lines 19-29 with:
|
|
|
|
```python
|
|
# Optional engines — only register if their SDK deps are present
|
|
for _optional in ("cloud", "litellm"):
|
|
try:
|
|
importlib.import_module(f".{_optional}", __name__)
|
|
except ImportError:
|
|
pass
|
|
```
|
|
|
|
And add `import importlib` at the top.
|
|
|
|
**Step 3: Run tests**
|
|
|
|
```bash
|
|
uv run pytest tests/channels/ tests/engine/ -v
|
|
```
|
|
Expected: All tests pass.
|
|
|
|
**Step 4: Commit**
|
|
|
|
```bash
|
|
git add src/openjarvis/channels/__init__.py src/openjarvis/engine/__init__.py
|
|
git commit -m "refactor: replace repetitive try/except imports with loop-based auto-registration"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 11: Update pyproject.toml package data includes
|
|
|
|
**Files:**
|
|
- Modify: `pyproject.toml`
|
|
|
|
**Step 1: Add package data includes**
|
|
|
|
The TOML data files that moved into `src/openjarvis/` need to be included in wheel builds. In `pyproject.toml`, after the existing `[tool.hatch.build.targets.wheel.force-include]` section, add the data directories.
|
|
|
|
Actually, since all the data directories are under `src/openjarvis/` and `packages = ["src/openjarvis"]` is already set, hatch will include them automatically as long as they are inside the package tree. But TOML files are non-Python files, so we need to tell hatch to include them.
|
|
|
|
Add to pyproject.toml after line 114:
|
|
|
|
```toml
|
|
[tool.hatch.build.targets.wheel.shared-data]
|
|
|
|
[tool.hatch.build]
|
|
include = [
|
|
"src/openjarvis/**/*.py",
|
|
"src/openjarvis/**/*.toml",
|
|
"src/openjarvis/**/*.md",
|
|
"src/openjarvis/agents/claude_code_runner/**",
|
|
"src/openjarvis/channels/whatsapp_baileys_bridge/**",
|
|
]
|
|
```
|
|
|
|
Wait — hatchling by default includes all files under `packages`. Non-Python files inside the package are included by default. No change needed here. Verify with:
|
|
|
|
```bash
|
|
uv run python -c "from pathlib import Path; d = Path('src/openjarvis/recipes/data'); print(list(d.glob('*.toml')))"
|
|
```
|
|
|
|
**Step 2: Commit if any changes were needed**
|
|
|
|
If no pyproject.toml changes needed, skip this task.
|
|
|
|
---
|
|
|
|
### Task 12: Update CLAUDE.md to reflect new structure
|
|
|
|
**Files:**
|
|
- Modify: `CLAUDE.md`
|
|
|
|
**Step 1: Update all references to moved paths**
|
|
|
|
In CLAUDE.md, make the following updates:
|
|
|
|
1. Update the `python -m evals` command references to `python -m openjarvis.evals`
|
|
2. Remove references to top-level `recipes/`, `templates/`, `skills/`, `operators/` directories
|
|
3. Update Docker command references to use `deploy/docker/` paths
|
|
4. Update the architecture section to reflect that data files are now package data
|
|
5. Remove mentions of `memory/` backward-compat shims
|
|
6. Remove the 5 individual engine wrapper files from architecture description
|
|
|
|
**Step 2: Run linting to verify no broken references**
|
|
|
|
```bash
|
|
uv run ruff check src/ tests/
|
|
```
|
|
|
|
**Step 3: Run full test suite**
|
|
|
|
```bash
|
|
uv run pytest tests/ -v --tb=short 2>&1 | tail -20
|
|
```
|
|
Expected: ~3240 tests pass.
|
|
|
|
**Step 4: Commit**
|
|
|
|
```bash
|
|
git add CLAUDE.md
|
|
git commit -m "docs: update CLAUDE.md to reflect simplified repository structure"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 13: Final verification
|
|
|
|
**Step 1: Run full test suite**
|
|
|
|
```bash
|
|
uv run pytest tests/ -v --tb=short 2>&1 | tail -30
|
|
```
|
|
Expected: ~3240 tests pass, ~44 skipped.
|
|
|
|
**Step 2: Run linter**
|
|
|
|
```bash
|
|
uv run ruff check src/ tests/
|
|
```
|
|
Expected: Clean.
|
|
|
|
**Step 3: Verify root structure**
|
|
|
|
```bash
|
|
ls -la | grep -v __pycache__ | grep -v '^\.'
|
|
```
|
|
Expected: 21 or fewer items at root (down from 32).
|
|
|
|
**Step 4: Verify package data is accessible**
|
|
|
|
```bash
|
|
uv run python -c "
|
|
from openjarvis.recipes.loader import discover_recipes
|
|
from openjarvis.templates.agent_templates import discover_templates
|
|
print(f'Recipes: {len(discover_recipes())}')
|
|
print(f'Templates: {len(discover_templates())}')
|
|
print('All data accessible.')
|
|
"
|
|
```
|
|
Expected: Recipes: 3, Templates: 15, All data accessible.
|