diff --git a/CLAUDE.md b/CLAUDE.md index 99f55d99..f0e3d6a3 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -4,13 +4,13 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co ## Project Status -OpenJarvis is a research framework for studying on-device AI systems. Phase 23 complete. Five composable pillars: Intelligence, Engine, Agents, Tools (with storage + MCP), and Learning — with trace-driven learning as a cross-cutting concern. ~3240 tests pass (~44 skipped for optional deps). Python SDK (`Jarvis` class), composition layer (`SystemBuilder`/`JarvisSystem`), eval framework (15 real benchmarks), composable recipes, agent templates, bundled skills, operator recipes, trace-driven learning pipeline, Docker deployment, Tauri desktop app, 40+ tools, 20+ CLI commands, 40+ API endpoints all ready. +OpenJarvis is a research framework for studying on-device AI systems. Phase 24 complete. Five composable pillars: Intelligence, Engine, Agents, Tools (with storage + MCP), and Learning — with trace-driven learning as a cross-cutting concern. Speech subsystem (STT) with pluggable backends. ~3295 tests pass (~44 skipped for optional deps). Python SDK (`Jarvis` class), composition layer (`SystemBuilder`/`JarvisSystem`), eval framework (15 real benchmarks), composable recipes, agent templates, bundled skills, operator recipes, trace-driven learning pipeline, Docker deployment, Tauri desktop app, 40+ tools, 20+ CLI commands, 40+ API endpoints all ready. ## Build & Development Commands ```bash uv sync --extra dev # Install deps + dev tools -uv run pytest tests/ -v # Run ~3241 tests (~44 skipped if optional deps missing) +uv run pytest tests/ -v # Run ~3295 tests (~44 skipped if optional deps missing) uv run ruff check src/ tests/ # Lint uv run jarvis --version # 1.0.0 uv run jarvis ask "Hello" # Query via discovered engine (direct mode) @@ -103,7 +103,7 @@ j.close() # Release resources ``` - **Package manager:** `uv` with `hatchling` build backend -- **Config:** `pyproject.toml` with extras for optional backends (e.g., `openjarvis[inference-vllm]`, `openjarvis[inference-mlx]`, `openjarvis[memory-colbert]`, `openjarvis[server]`, `openjarvis[openclaw]`, `openjarvis[energy-amd]`, `openjarvis[energy-apple]`, `openjarvis[energy-all]`, `openjarvis[security-signing]`, `openjarvis[sandbox-wasm]`, `openjarvis[dashboard]`, `openjarvis[browser]`, `openjarvis[media]`, `openjarvis[pdf]`, `openjarvis[channel-line]`, `openjarvis[channel-viber]`, `openjarvis[channel-reddit]`, `openjarvis[channel-mastodon]`, `openjarvis[channel-xmpp]`, `openjarvis[channel-rocketchat]`, `openjarvis[channel-zulip]`, `openjarvis[channel-twitch]`, `openjarvis[channel-nostr]`) +- **Config:** `pyproject.toml` with extras for optional backends (e.g., `openjarvis[inference-vllm]`, `openjarvis[inference-mlx]`, `openjarvis[memory-colbert]`, `openjarvis[server]`, `openjarvis[openclaw]`, `openjarvis[energy-amd]`, `openjarvis[energy-apple]`, `openjarvis[energy-all]`, `openjarvis[security-signing]`, `openjarvis[sandbox-wasm]`, `openjarvis[dashboard]`, `openjarvis[browser]`, `openjarvis[media]`, `openjarvis[pdf]`, `openjarvis[channel-line]`, `openjarvis[channel-viber]`, `openjarvis[channel-reddit]`, `openjarvis[channel-mastodon]`, `openjarvis[channel-xmpp]`, `openjarvis[channel-rocketchat]`, `openjarvis[channel-zulip]`, `openjarvis[channel-twitch]`, `openjarvis[channel-nostr]`, `openjarvis[speech]`, `openjarvis[speech-deepgram]`) - **CLI entry point:** `jarvis` (Click-based) — subcommands: `init`, `ask`, `serve`, `start`, `stop`, `restart`, `status`, `chat`, `model`, `memory`, `telemetry`, `bench`, `eval`, `channel`, `scheduler`, `doctor`, `agent`, `workflow`, `skill`, `vault`, `add` - **Python:** 3.10+ required - **Node.js:** 22+ required only for OpenClaw agent @@ -133,6 +133,10 @@ OpenJarvis is a research framework for on-device AI organized around **five comp - All registered via `@ToolRegistry.register("name")` decorator 5. **Learning** (`src/openjarvis/learning/`) — Structured learning with nested per-pillar sub-policies. `LearningConfig` sections: `routing` (heuristic/learned/grpo/bandit), `intelligence` (none/sft), `agent` (none/agent_advisor/icl_updater), `metrics` (accuracy/latency/cost/efficiency weights). Policies: `SFTRouterPolicy` (query→model from traces), `AgentAdvisorPolicy` (LM-guided), `ICLUpdaterPolicy` (in-context with example DB, versioning, rollback, quality gates), `GRPORouterPolicy` (softmax sampling, group relative advantage, per-query-class weights), `BanditRouterPolicy` (Thompson Sampling / UCB1, per-arm stats). `SkillDiscovery` mines tool subsequences from traces to auto-generate skill manifests. Router policies: `HeuristicRouter`, `TraceDrivenPolicy`. Orchestrator training subpackage provides SFT and GRPO pipelines. **Trace-driven learning pipeline**: `TrainingDataMiner` (extracts SFT pairs from traces with quality filters), `LoRATrainer` (LoRA fine-tuning with configurable rank/alpha, requires torch), `AgentConfigEvolver` (LM-guided agent config recommendations from trace patterns), `LearningOrchestrator` (wired into `SystemBuilder`, orchestrates mine→train→evolve cycle on schedule). +### Speech Subsystem + +- **Speech** (`src/openjarvis/speech/`) — Speech-to-text with pluggable backends. `SpeechBackend` ABC (`transcribe()`, `health()`, `supported_formats()`). `TranscriptionResult` + `Segment` dataclasses. Backends: `FasterWhisperBackend` (local, CTranslate2, key `"faster-whisper"`), `OpenAIWhisperBackend` (cloud, `whisper-1`, key `"openai"`), `DeepgramSpeechBackend` (cloud, `nova-2`, key `"deepgram"`). Auto-discovery with local-first priority. `SpeechConfig` in `JarvisConfig`. `SpeechRegistry` for backend registration. Wired into `SystemBuilder.speech()`, `create_app()`, `jarvis serve`. API: `POST /v1/speech/transcribe` (multipart), `GET /v1/speech/health`. Frontend: `useSpeech` hook + `MicButton` component. Tauri: `transcribe_audio` + `speech_health` commands. Optional deps: `openjarvis[speech]` (faster-whisper), `openjarvis[speech-deepgram]` (deepgram-sdk). + ### Cross-cutting Systems - **Traces** (`src/openjarvis/traces/`) — Full interaction recording. `Trace` captures `TraceStep`s (route, retrieve, generate, tool_call, respond) with timing. `TraceStore` (SQLite), `TraceCollector` (auto-wraps agents), `TraceAnalyzer` (stats for learning). @@ -194,6 +198,7 @@ OpenAI-compatible server via `jarvis serve`: - **Telemetry**: `GET /v1/telemetry/stats`, `GET /v1/telemetry/energy` - **Learning**: `GET /v1/learning/stats`, `GET /v1/learning/policy` - **Skills**: `GET /v1/skills`, `POST /v1/skills`, `DELETE /v1/skills/{name}` +- **Speech**: `POST /v1/speech/transcribe`, `GET /v1/speech/health` - **Sessions**: `GET /v1/sessions`, `GET /v1/sessions/{id}` - **Budget**: `GET /v1/budget`, `PUT /v1/budget/limits` - **Metrics**: `GET /metrics` (Prometheus-compatible) @@ -238,3 +243,4 @@ OpenAI-compatible server via `jarvis serve`: | v2.6 | 21 | 10 new channels: LINE, Viber, Messenger, Reddit, Mastodon, XMPP, Rocket.Chat, Zulip, Twitch, Nostr | | v2.7 | 22 | Operators: persistent, scheduled autonomous agents with recipe + schedule + channel output | | v2.8 | 23 | Differentiated functionalities: trace-driven learning pipeline (TrainingDataMiner, LoRATrainer, AgentConfigEvolver, LearningOrchestrator), 15 real IPW benchmarks, composable recipes, 15 agent templates, 20 bundled skills, 3 operator recipes | +| v2.9 | 24 | Speech subsystem: SpeechBackend ABC, SpeechRegistry, 3 backends (FasterWhisper local, OpenAI cloud, Deepgram cloud), auto-discovery, API endpoints, frontend MicButton + useSpeech hook, Tauri commands, SystemBuilder wiring | diff --git a/desktop/src-tauri/Cargo.lock b/desktop/src-tauri/Cargo.lock index 08231e32..ba5f0965 100644 --- a/desktop/src-tauri/Cargo.lock +++ b/desktop/src-tauri/Cargo.lock @@ -2161,6 +2161,16 @@ version = "0.3.17" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "6877bb514081ee2a7ff5ef9de3281f14a4dd4bceac4c09388074a6b5df8a139a" +[[package]] +name = "mime_guess" +version = "2.0.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f7c44f8e672c00fe5308fa235f821cb4198414e1c77935c1ab6948d3fd78550e" +dependencies = [ + "mime", + "unicase", +] + [[package]] name = "minisign-verify" version = "0.2.4" @@ -3289,6 +3299,7 @@ dependencies = [ "bytes", "encoding_rs", "futures-core", + "futures-util", "h2", "http", "http-body", @@ -3300,6 +3311,7 @@ dependencies = [ "js-sys", "log", "mime", + "mime_guess", "native-tls", "percent-encoding", "pin-project-lite", @@ -4921,6 +4933,12 @@ dependencies = [ "unic-common", ] +[[package]] +name = "unicase" +version = "2.9.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dbc4bc3a9f746d862c45cb89d705aa10f187bb96c76001afab07a0d35ce60142" + [[package]] name = "unicode-ident" version = "1.0.24" diff --git a/desktop/src-tauri/Cargo.toml b/desktop/src-tauri/Cargo.toml index 1d4639fe..157c718e 100644 --- a/desktop/src-tauri/Cargo.toml +++ b/desktop/src-tauri/Cargo.toml @@ -19,7 +19,7 @@ tauri-plugin-single-instance = "2" tauri-plugin-process = "2" serde = { version = "1", features = ["derive"] } serde_json = "1" -reqwest = { version = "0.12", features = ["json"] } +reqwest = { version = "0.12", features = ["json", "multipart"] } tokio = { version = "1", features = ["full"] } [features] diff --git a/desktop/src-tauri/src/lib.rs b/desktop/src-tauri/src/lib.rs index 494c87bb..73bf7301 100644 --- a/desktop/src-tauri/src/lib.rs +++ b/desktop/src-tauri/src/lib.rs @@ -405,6 +405,50 @@ async fn run_jarvis_command(args: Vec) -> Result { } } +/// Transcribe audio via the speech API endpoint. +#[tauri::command] +async fn transcribe_audio( + api_url: String, + audio_data: Vec, + filename: String, +) -> Result { + let url = format!("{}/v1/speech/transcribe", api_url); + let client = reqwest::Client::new(); + + let part = reqwest::multipart::Part::bytes(audio_data) + .file_name(filename) + .mime_str("audio/webm") + .map_err(|e| format!("Failed to create multipart: {}", e))?; + + let form = reqwest::multipart::Form::new().part("file", part); + + let resp = client + .post(&url) + .multipart(form) + .send() + .await + .map_err(|e| format!("Connection failed: {}", e))?; + let body: serde_json::Value = resp + .json() + .await + .map_err(|e| format!("Invalid response: {}", e))?; + Ok(body) +} + +/// Check speech backend health. +#[tauri::command] +async fn speech_health(api_url: String) -> Result { + let url = format!("{}/v1/speech/health", api_url); + let resp = reqwest::get(&url) + .await + .map_err(|e| format!("Connection failed: {}", e))?; + let body: serde_json::Value = resp + .json() + .await + .map_err(|e| format!("Invalid response: {}", e))?; + Ok(body) +} + // --------------------------------------------------------------------------- // App entry point // --------------------------------------------------------------------------- @@ -497,6 +541,8 @@ pub fn run() { fetch_agents, fetch_models, run_jarvis_command, + transcribe_audio, + speech_health, ]) .build(tauri::generate_context!()) .expect("error while building OpenJarvis Desktop") diff --git a/docs/plans/2026-03-03-speech-to-text-design.md b/docs/plans/2026-03-03-speech-to-text-design.md new file mode 100644 index 00000000..93c07113 --- /dev/null +++ b/docs/plans/2026-03-03-speech-to-text-design.md @@ -0,0 +1,280 @@ +# Speech-to-Text Design + +**Date:** 2026-03-03 +**Status:** Approved +**Scope:** Add voice input (speech-to-text) to OpenJarvis — desktop app and browser + +## Overview + +Add a new Speech subsystem to OpenJarvis that lets users speak commands instead of typing them. The system transcribes audio to text using local open-source models (Faster-Whisper) by default, with cloud backends (OpenAI, Deepgram) as alternatives. Transcribed text is inserted into the input box for user review before sending. + +## Design Decisions + +| Decision | Choice | Rationale | +|----------|--------|-----------| +| Scope | STT only (no TTS) | Ship voice input first; TTS follows later | +| Runtime | Separate process | Avoid VRAM conflicts with the LLM engine | +| Surfaces | Desktop app + browser | Both use the same React frontend | +| UX mode | Record-then-transcribe | Simpler to implement, works with all backends | +| Architecture | New Speech subsystem | Fits OpenJarvis patterns (ABC + registry + decorator) | + +## Architecture + +### New Module: `src/openjarvis/speech/` + +``` +src/openjarvis/speech/ +├── __init__.py # Imports, ensure_registered() +├── _stubs.py # SpeechBackend ABC, TranscriptionResult dataclass +├── faster_whisper.py # FasterWhisperBackend (local, default) +├── whisper_cpp.py # WhisperCppBackend (local, llama.cpp ecosystem) +├── openai_whisper.py # OpenAIWhisperBackend (cloud) +├── deepgram.py # DeepgramBackend (cloud) +└── _discovery.py # Auto-discover available backend (local preferred) +``` + +### Core Types (`_stubs.py`) + +```python +@dataclass +class Segment: + text: str + start: float # Start time in seconds + end: float # End time in seconds + confidence: float | None + +@dataclass +class TranscriptionResult: + text: str # The transcribed text + language: str | None # Detected language code (e.g., "en") + confidence: float | None # Overall confidence [0, 1] + duration_seconds: float # Audio duration + segments: list[Segment] # Word/phrase-level timing (optional) + +class SpeechBackend(ABC): + backend_id: str + + @abstractmethod + def transcribe(self, audio: bytes, *, format: str = "wav", + language: str | None = None) -> TranscriptionResult: ... + + @abstractmethod + def health(self) -> bool: ... + + @abstractmethod + def supported_formats(self) -> list[str]: ... +``` + +### Registry + +New `SpeechRegistry` added to `core/registry.py` using `RegistryBase[T]`. Backends register via `@SpeechRegistry.register("faster-whisper")`. + +### Discovery (`_discovery.py`) + +Priority order (local-first): +1. Faster-Whisper (if `faster-whisper` package installed) +2. WhisperCpp (if `whisper-cpp-python` package installed) +3. OpenAI Whisper API (if `OPENAI_API_KEY` set) +4. Deepgram (if `DEEPGRAM_API_KEY` set) + +Function: `get_speech_backend(config) -> SpeechBackend | None` + +### Config + +New `[speech]` section in `JarvisConfig`: + +```toml +[speech] +backend = "auto" # "auto", "faster-whisper", "whisper-cpp", "openai", "deepgram" +model = "base" # Whisper model size: tiny, base, small, medium, large-v3 +language = "" # Empty = auto-detect +device = "auto" # "auto", "cpu", "cuda" +compute_type = "float16" # "float16", "int8", "float32" +``` + +New `SpeechConfig` dataclass in `core/config.py`. + +## API Layer + +### New Endpoints + +``` +POST /v1/speech/transcribe + Content-Type: multipart/form-data + Body: audio file (field name: "file") + Optional form fields: language, model + + Response 200: { + "text": "Hello, what's the weather like?", + "language": "en", + "confidence": 0.94, + "duration_seconds": 2.3 + } + +GET /v1/speech/backends + Response 200: { + "backends": ["faster-whisper"], + "active": "faster-whisper", + "model": "base" + } + +GET /v1/speech/health + Response 200: {"available": true, "backend": "faster-whisper", "model": "base"} + Response 200: {"available": false, "reason": "No speech backend installed"} +``` + +### Server Wiring + +- `SystemBuilder.with_speech(backend=..., model=...)` — configure speech +- `JarvisSystem.speech` — active `SpeechBackend` instance or `None` +- Speech backend initializes lazily on first `transcribe()` call +- Routes added to `server/api_routes.py` + +## Frontend Integration + +### Web Frontend (`frontend/src/`) + +**New component: `MicButton.tsx`** +- Sits next to the Send button in `InputArea.tsx` +- States: idle (mic icon), recording (pulsing red), transcribing (spinner) +- Uses `MediaRecorder` API to capture audio +- Sends audio blob as multipart to `/v1/speech/transcribe` +- Appends transcribed text to input textarea +- Hidden if `/v1/speech/health` returns `available: false` + +**New hook: `useSpeech.ts`** +- Manages microphone permissions, `MediaRecorder` lifecycle +- `startRecording()`, `stopRecording()`, `isRecording`, `isTranscribing`, `error` +- Checks `navigator.mediaDevices` support + +**Changes to existing components:** +- `InputArea.tsx` — add `MicButton` next to send button +- `App.tsx` — check speech availability on load + +### Desktop App (`desktop/`) + +Same React UI works in the Tauri WebView. + +**New Tauri command: `transcribe_audio`** in `lib.rs` — proxies multipart POST to backend for cases where WebView fetch has CORS issues. + +### Audio Format + +Record as WebM/Opus (native browser format). Faster-Whisper handles WebM directly. Server-side conversion to WAV as fallback if a backend requires it. + +### User Flow + +``` +User clicks mic button + → Browser requests microphone permission (first time) + → Recording starts (button pulses red) +User clicks mic button again (or releases) + → Recording stops + → Button shows spinner + → Audio blob sent to POST /v1/speech/transcribe + → Response text inserted into input textarea + → User reviews/edits text + → User clicks Send (normal flow) +``` + +## Backend Implementations + +### Faster-Whisper (Default Local) + +```python +@SpeechRegistry.register("faster-whisper") +class FasterWhisperBackend(SpeechBackend): + backend_id = "faster-whisper" + # Uses CTranslate2-based Faster-Whisper (4x faster than original Whisper) + # Lazy model loading on first transcribe() call + # Model sizes: tiny (39M), base (74M), small (244M), medium (769M), large-v3 (1.5B) + # GPU: device="cuda", compute_type="float16" or "int8" + # CPU: device="cpu", compute_type="int8" +``` + +Optional dep: `openjarvis[speech]` → `faster-whisper>=1.0` + +### OpenAI Whisper API (Cloud) + +```python +@SpeechRegistry.register("openai") +class OpenAIWhisperBackend(SpeechBackend): + backend_id = "openai" + # Uses openai.audio.transcriptions.create() + # Requires OPENAI_API_KEY + # Uses existing openai dependency +``` + +### Deepgram (Cloud) + +```python +@SpeechRegistry.register("deepgram") +class DeepgramBackend(SpeechBackend): + backend_id = "deepgram" + # Uses deepgram-sdk + # Requires DEEPGRAM_API_KEY +``` + +Optional dep: `openjarvis[speech-deepgram]` → `deepgram-sdk>=3.0` + +## Error Handling + +| Scenario | Behavior | +|----------|----------| +| Mic permission denied | Toast error in frontend, no crash | +| No speech backend available | Mic button hidden, health endpoint returns `available: false` | +| Audio too short / silence | Return empty text with low confidence | +| Model loading failure | Health returns false, error logged | +| Network failure (cloud) | Error response, frontend shows retry option | +| Unsupported audio format | Server converts to WAV, or returns 400 with message | + +## Testing + +### Backend Tests +- `tests/speech/test_faster_whisper.py` — mock `faster_whisper` import, test transcribe with synthetic WAV +- `tests/speech/test_openai_whisper.py` — mock `openai` client, test transcribe +- `tests/speech/test_deepgram.py` — mock `deepgram` client, test transcribe +- `tests/speech/test_discovery.py` — test auto-discovery priority order + +### API Tests +- `tests/server/test_speech_routes.py` — test endpoints with mocked backend + +### Config Tests +- Test `SpeechConfig` defaults, TOML parsing, `[speech]` section + +All optional-dep backends behind `pytest.importorskip()`. + +## Optional Dependencies + +```toml +[project.optional-dependencies] +speech = ["faster-whisper>=1.0"] +speech-deepgram = ["deepgram-sdk>=3.0"] +``` + +## Files Changed (Estimated) + +### New Files (~12) +- `src/openjarvis/speech/__init__.py` +- `src/openjarvis/speech/_stubs.py` +- `src/openjarvis/speech/faster_whisper.py` +- `src/openjarvis/speech/openai_whisper.py` +- `src/openjarvis/speech/deepgram.py` +- `src/openjarvis/speech/_discovery.py` +- `frontend/src/components/MicButton.tsx` +- `frontend/src/hooks/useSpeech.ts` +- `tests/speech/test_faster_whisper.py` +- `tests/speech/test_openai_whisper.py` +- `tests/speech/test_deepgram.py` +- `tests/speech/test_discovery.py` +- `tests/server/test_speech_routes.py` + +### Modified Files (~8) +- `src/openjarvis/core/registry.py` — add `SpeechRegistry` +- `src/openjarvis/core/config.py` — add `SpeechConfig`, `[speech]` section +- `src/openjarvis/system.py` — add `with_speech()`, wire speech backend +- `src/openjarvis/server/api_routes.py` — add speech endpoints +- `frontend/src/components/InputArea.tsx` — add MicButton +- `frontend/src/App.tsx` — check speech availability +- `desktop/src-tauri/src/lib.rs` — add `transcribe_audio` command +- `pyproject.toml` — add `[speech]` and `[speech-deepgram]` extras +- `tests/conftest.py` — add SpeechRegistry to `_clean_registries` diff --git a/docs/plans/2026-03-03-speech-to-text.md b/docs/plans/2026-03-03-speech-to-text.md new file mode 100644 index 00000000..1282c467 --- /dev/null +++ b/docs/plans/2026-03-03-speech-to-text.md @@ -0,0 +1,1803 @@ +# Speech-to-Text Implementation Plan + +> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. + +**Goal:** Add a Speech subsystem to OpenJarvis that lets users speak commands instead of typing them in the desktop app and browser. + +**Architecture:** New `src/openjarvis/speech/` module following the ABC + registry pattern. `SpeechBackend` ABC with `transcribe()` method. Three backends: Faster-Whisper (local, default), OpenAI Whisper API (cloud), Deepgram (cloud). New `POST /v1/speech/transcribe` endpoint. Frontend `MicButton` component records audio and sends to backend. + +**Tech Stack:** faster-whisper (CTranslate2), openai SDK (existing dep), deepgram-sdk, Web MediaRecorder API, FastAPI multipart uploads, Tauri commands (Rust/reqwest) + +**Design doc:** `docs/plans/2026-03-03-speech-to-text-design.md` + +--- + +### Task 1: Add SpeechRegistry to core + +**Files:** +- Modify: `src/openjarvis/core/registry.py:136-152` +- Modify: `tests/conftest.py:12-35` +- Test: `tests/core/test_registry.py` (existing — verify it still passes) + +**Step 1: Add SpeechRegistry class** + +In `src/openjarvis/core/registry.py`, after line 137 (`class SkillRegistry`), add: + +```python +class SpeechRegistry(RegistryBase[Any]): + """Registry for speech backend implementations.""" +``` + +Add `"SpeechRegistry"` to `__all__` list (alphabetically, after `"SkillRegistry"`). + +**Step 2: Update conftest.py to clear SpeechRegistry** + +In `tests/conftest.py`, add `SpeechRegistry` to the import block (line 12-21): + +```python +from openjarvis.core.registry import ( + AgentRegistry, + BenchmarkRegistry, + ChannelRegistry, + EngineRegistry, + MemoryRegistry, + ModelRegistry, + RouterPolicyRegistry, + SpeechRegistry, + ToolRegistry, +) +``` + +Add `SpeechRegistry.clear()` after line 34 (`ChannelRegistry.clear()`). + +**Step 3: Run tests to verify nothing broke** + +Run: `uv run pytest tests/core/test_registry.py -v` +Expected: All existing registry tests PASS + +**Step 4: Commit** + +```bash +git add src/openjarvis/core/registry.py tests/conftest.py +git commit -m "feat(speech): add SpeechRegistry to core registry module" +``` + +--- + +### Task 2: Add SpeechConfig to config + +**Files:** +- Modify: `src/openjarvis/core/config.py:824-853, 943-948` +- Test: `tests/core/test_config.py` (existing) + +**Step 1: Write the failing test** + +Create `tests/speech/__init__.py` (empty) and `tests/speech/test_config.py`: + +```python +"""Tests for speech configuration.""" + +from openjarvis.core.config import JarvisConfig, SpeechConfig + + +def test_speech_config_defaults(): + cfg = SpeechConfig() + assert cfg.backend == "auto" + assert cfg.model == "base" + assert cfg.language == "" + assert cfg.device == "auto" + assert cfg.compute_type == "float16" + + +def test_jarvis_config_has_speech(): + cfg = JarvisConfig() + assert hasattr(cfg, "speech") + assert isinstance(cfg.speech, SpeechConfig) + assert cfg.speech.backend == "auto" +``` + +**Step 2: Run test to verify it fails** + +Run: `uv run pytest tests/speech/test_config.py -v` +Expected: FAIL with `ImportError: cannot import name 'SpeechConfig'` + +**Step 3: Add SpeechConfig dataclass** + +In `src/openjarvis/core/config.py`, after line 831 (after `OperatorsConfig`), add: + +```python +@dataclass(slots=True) +class SpeechConfig: + """Speech-to-text settings.""" + + backend: str = "auto" # "auto", "faster-whisper", "whisper-cpp", "openai", "deepgram" + model: str = "base" # Whisper model size: tiny, base, small, medium, large-v3 + language: str = "" # Empty = auto-detect + device: str = "auto" # "auto", "cpu", "cuda" + compute_type: str = "float16" # "float16", "int8", "float32" +``` + +Add to `JarvisConfig` (after line 853, `operators: OperatorsConfig`): + +```python + speech: SpeechConfig = field(default_factory=SpeechConfig) +``` + +Add `"speech"` to the `top_sections` tuple in `load_config()` (line 943-948): + +```python + top_sections = ( + "engine", "intelligence", "learning", "agent", + "server", "telemetry", "traces", "security", + "channel", "tools", "sandbox", "scheduler", + "workflow", "sessions", "a2a", "operators", + "speech", + ) +``` + +**Step 4: Run test to verify it passes** + +Run: `uv run pytest tests/speech/test_config.py -v` +Expected: PASS + +**Step 5: Commit** + +```bash +git add src/openjarvis/core/config.py tests/speech/ +git commit -m "feat(speech): add SpeechConfig to configuration system" +``` + +--- + +### Task 3: Create SpeechBackend ABC and TranscriptionResult + +**Files:** +- Create: `src/openjarvis/speech/__init__.py` +- Create: `src/openjarvis/speech/_stubs.py` +- Test: `tests/speech/test_stubs.py` + +**Step 1: Write the failing test** + +Create `tests/speech/test_stubs.py`: + +```python +"""Tests for speech ABC and data types.""" + +from openjarvis.speech._stubs import Segment, SpeechBackend, TranscriptionResult + + +def test_transcription_result(): + result = TranscriptionResult( + text="Hello world", + language="en", + confidence=0.95, + duration_seconds=1.5, + segments=[], + ) + assert result.text == "Hello world" + assert result.language == "en" + assert result.confidence == 0.95 + assert result.duration_seconds == 1.5 + assert result.segments == [] + + +def test_segment(): + seg = Segment(text="Hello", start=0.0, end=0.5, confidence=0.98) + assert seg.text == "Hello" + assert seg.start == 0.0 + assert seg.end == 0.5 + + +def test_speech_backend_is_abstract(): + import pytest + + with pytest.raises(TypeError): + SpeechBackend() +``` + +**Step 2: Run test to verify it fails** + +Run: `uv run pytest tests/speech/test_stubs.py -v` +Expected: FAIL with `ModuleNotFoundError: No module named 'openjarvis.speech'` + +**Step 3: Create the module** + +Create `src/openjarvis/speech/__init__.py`: + +```python +"""Speech subsystem — speech-to-text backends.""" +``` + +Create `src/openjarvis/speech/_stubs.py`: + +```python +"""Abstract base classes and data types for the speech subsystem.""" + +from __future__ import annotations + +from abc import ABC, abstractmethod +from dataclasses import dataclass, field +from typing import List, Optional + + +@dataclass +class Segment: + """A timed segment of transcribed text.""" + + text: str + start: float # Start time in seconds + end: float # End time in seconds + confidence: Optional[float] = None + + +@dataclass +class TranscriptionResult: + """Result of a speech-to-text transcription.""" + + text: str + language: Optional[str] = None + confidence: Optional[float] = None + duration_seconds: float = 0.0 + segments: List[Segment] = field(default_factory=list) + + +class SpeechBackend(ABC): + """Abstract base class for speech-to-text backends.""" + + backend_id: str = "" + + @abstractmethod + def transcribe( + self, + audio: bytes, + *, + format: str = "wav", + language: Optional[str] = None, + ) -> TranscriptionResult: + """Transcribe audio bytes to text.""" + + @abstractmethod + def health(self) -> bool: + """Check if the backend is ready.""" + + @abstractmethod + def supported_formats(self) -> List[str]: + """Return list of supported audio formats.""" +``` + +**Step 4: Run test to verify it passes** + +Run: `uv run pytest tests/speech/test_stubs.py -v` +Expected: PASS + +**Step 5: Commit** + +```bash +git add src/openjarvis/speech/ +git commit -m "feat(speech): add SpeechBackend ABC, TranscriptionResult, Segment" +``` + +--- + +### Task 4: Implement FasterWhisperBackend + +**Files:** +- Create: `src/openjarvis/speech/faster_whisper.py` +- Test: `tests/speech/test_faster_whisper.py` + +**Step 1: Write the failing test** + +Create `tests/speech/test_faster_whisper.py`: + +```python +"""Tests for Faster-Whisper speech backend.""" + +from unittest.mock import MagicMock, patch + +import pytest + + +def test_faster_whisper_backend_registers(): + """Backend registers itself in SpeechRegistry.""" + from openjarvis.core.registry import SpeechRegistry + + # Import triggers registration + import openjarvis.speech.faster_whisper # noqa: F401 + + assert SpeechRegistry.contains("faster-whisper") + + +def test_faster_whisper_transcribe(): + """Transcribe returns a TranscriptionResult.""" + from openjarvis.speech._stubs import TranscriptionResult + + mock_model = MagicMock() + mock_segment = MagicMock() + mock_segment.text = " Hello world" + mock_segment.start = 0.0 + mock_segment.end = 1.2 + mock_segment.avg_logprob = -0.3 + + mock_info = MagicMock() + mock_info.language = "en" + mock_info.language_probability = 0.95 + mock_info.duration = 1.5 + + mock_model.transcribe.return_value = ([mock_segment], mock_info) + + with patch( + "openjarvis.speech.faster_whisper.WhisperModel", + return_value=mock_model, + ): + from openjarvis.speech.faster_whisper import FasterWhisperBackend + + backend = FasterWhisperBackend(model_size="base", device="cpu") + result = backend.transcribe(b"fake audio bytes") + + assert isinstance(result, TranscriptionResult) + assert result.text == "Hello world" + assert result.language == "en" + assert result.duration_seconds == 1.5 + + +def test_faster_whisper_health_no_model(): + """Health returns False before model is loaded.""" + with patch( + "openjarvis.speech.faster_whisper.WhisperModel", + side_effect=ImportError("no module"), + ): + from openjarvis.speech.faster_whisper import FasterWhisperBackend + + backend = FasterWhisperBackend.__new__(FasterWhisperBackend) + backend._model = None + assert backend.health() is False + + +def test_faster_whisper_supported_formats(): + """Backend supports standard audio formats.""" + with patch("openjarvis.speech.faster_whisper.WhisperModel"): + from openjarvis.speech.faster_whisper import FasterWhisperBackend + + backend = FasterWhisperBackend.__new__(FasterWhisperBackend) + formats = backend.supported_formats() + assert "wav" in formats + assert "mp3" in formats + assert "webm" in formats +``` + +**Step 2: Run test to verify it fails** + +Run: `uv run pytest tests/speech/test_faster_whisper.py -v` +Expected: FAIL with `ModuleNotFoundError: No module named 'openjarvis.speech.faster_whisper'` + +**Step 3: Implement FasterWhisperBackend** + +Create `src/openjarvis/speech/faster_whisper.py`: + +```python +"""Faster-Whisper speech-to-text backend (local, CTranslate2-based).""" + +from __future__ import annotations + +import io +import tempfile +from typing import List, Optional + +from openjarvis.core.registry import SpeechRegistry +from openjarvis.speech._stubs import Segment, SpeechBackend, TranscriptionResult + +try: + from faster_whisper import WhisperModel +except ImportError: + WhisperModel = None # type: ignore[assignment, misc] + + +@SpeechRegistry.register("faster-whisper") +class FasterWhisperBackend(SpeechBackend): + """Local speech-to-text using Faster-Whisper (CTranslate2).""" + + backend_id = "faster-whisper" + + def __init__( + self, + model_size: str = "base", + device: str = "auto", + compute_type: str = "float16", + ) -> None: + self._model_size = model_size + self._device = device + self._compute_type = compute_type + self._model: Optional[WhisperModel] = None + + def _ensure_model(self) -> WhisperModel: + """Lazy-load the Whisper model on first use.""" + if self._model is None: + if WhisperModel is None: + raise ImportError( + "faster-whisper is not installed. " + "Install with: pip install 'openjarvis[speech]'" + ) + self._model = WhisperModel( + self._model_size, + device=self._device, + compute_type=self._compute_type, + ) + return self._model + + def transcribe( + self, + audio: bytes, + *, + format: str = "wav", + language: Optional[str] = None, + ) -> TranscriptionResult: + """Transcribe audio bytes using Faster-Whisper.""" + model = self._ensure_model() + + # Write audio to a temp file (faster-whisper needs a file path) + suffix = f".{format}" if not format.startswith(".") else format + with tempfile.NamedTemporaryFile(suffix=suffix, delete=True) as tmp: + tmp.write(audio) + tmp.flush() + + kwargs = {} + if language: + kwargs["language"] = language + + segments_iter, info = model.transcribe(tmp.name, **kwargs) + segments_list = list(segments_iter) + + # Build result + text = "".join(seg.text for seg in segments_list).strip() + segments = [ + Segment( + text=seg.text.strip(), + start=seg.start, + end=seg.end, + confidence=None, + ) + for seg in segments_list + ] + + return TranscriptionResult( + text=text, + language=getattr(info, "language", None), + confidence=getattr(info, "language_probability", None), + duration_seconds=getattr(info, "duration", 0.0), + segments=segments, + ) + + def health(self) -> bool: + """Check if model is loaded or loadable.""" + if self._model is not None: + return True + return WhisperModel is not None + + def supported_formats(self) -> List[str]: + """Supported audio formats (same as ffmpeg/Whisper).""" + return ["wav", "mp3", "m4a", "ogg", "flac", "webm"] +``` + +**Step 4: Run test to verify it passes** + +Run: `uv run pytest tests/speech/test_faster_whisper.py -v` +Expected: PASS + +**Step 5: Commit** + +```bash +git add src/openjarvis/speech/faster_whisper.py tests/speech/test_faster_whisper.py +git commit -m "feat(speech): implement FasterWhisperBackend (local STT)" +``` + +--- + +### Task 5: Implement OpenAIWhisperBackend + +**Files:** +- Create: `src/openjarvis/speech/openai_whisper.py` +- Test: `tests/speech/test_openai_whisper.py` + +**Step 1: Write the failing test** + +Create `tests/speech/test_openai_whisper.py`: + +```python +"""Tests for OpenAI Whisper API speech backend.""" + +from unittest.mock import MagicMock, patch + +from openjarvis.speech._stubs import TranscriptionResult + + +def test_openai_whisper_registers(): + from openjarvis.core.registry import SpeechRegistry + import openjarvis.speech.openai_whisper # noqa: F401 + + assert SpeechRegistry.contains("openai") + + +def test_openai_whisper_transcribe(): + mock_client = MagicMock() + mock_response = MagicMock() + mock_response.text = "Hello from OpenAI" + mock_response.language = "en" + mock_response.duration = 2.0 + mock_client.audio.transcriptions.create.return_value = mock_response + + with patch("openjarvis.speech.openai_whisper.OpenAI", return_value=mock_client): + from openjarvis.speech.openai_whisper import OpenAIWhisperBackend + + backend = OpenAIWhisperBackend(api_key="test-key") + result = backend.transcribe(b"fake audio", format="wav") + + assert isinstance(result, TranscriptionResult) + assert result.text == "Hello from OpenAI" + assert result.language == "en" + + +def test_openai_whisper_health(): + with patch("openjarvis.speech.openai_whisper.OpenAI"): + from openjarvis.speech.openai_whisper import OpenAIWhisperBackend + + backend = OpenAIWhisperBackend(api_key="test-key") + assert backend.health() is True + + +def test_openai_whisper_health_no_key(): + with patch("openjarvis.speech.openai_whisper.OpenAI"): + from openjarvis.speech.openai_whisper import OpenAIWhisperBackend + + backend = OpenAIWhisperBackend.__new__(OpenAIWhisperBackend) + backend._client = None + backend._api_key = "" + assert backend.health() is False +``` + +**Step 2: Run test to verify it fails** + +Run: `uv run pytest tests/speech/test_openai_whisper.py -v` +Expected: FAIL + +**Step 3: Implement OpenAIWhisperBackend** + +Create `src/openjarvis/speech/openai_whisper.py`: + +```python +"""OpenAI Whisper API speech-to-text backend (cloud).""" + +from __future__ import annotations + +import io +import os +from typing import List, Optional + +from openjarvis.core.registry import SpeechRegistry +from openjarvis.speech._stubs import SpeechBackend, TranscriptionResult + +try: + from openai import OpenAI +except ImportError: + OpenAI = None # type: ignore[assignment, misc] + + +@SpeechRegistry.register("openai") +class OpenAIWhisperBackend(SpeechBackend): + """Cloud speech-to-text using OpenAI Whisper API.""" + + backend_id = "openai" + + def __init__(self, api_key: Optional[str] = None) -> None: + self._api_key = api_key or os.environ.get("OPENAI_API_KEY", "") + self._client: Optional[OpenAI] = None + if self._api_key and OpenAI is not None: + self._client = OpenAI(api_key=self._api_key) + + def transcribe( + self, + audio: bytes, + *, + format: str = "wav", + language: Optional[str] = None, + ) -> TranscriptionResult: + """Transcribe audio using OpenAI's Whisper API.""" + if self._client is None: + raise RuntimeError("OpenAI client not initialized (missing API key?)") + + ext = format if not format.startswith(".") else format[1:] + audio_file = io.BytesIO(audio) + audio_file.name = f"audio.{ext}" + + kwargs: dict = {"model": "whisper-1", "file": audio_file} + if language: + kwargs["language"] = language + kwargs["response_format"] = "verbose_json" + + response = self._client.audio.transcriptions.create(**kwargs) + + return TranscriptionResult( + text=getattr(response, "text", str(response)), + language=getattr(response, "language", None), + confidence=None, + duration_seconds=getattr(response, "duration", 0.0), + segments=[], + ) + + def health(self) -> bool: + return self._client is not None and bool(self._api_key) + + def supported_formats(self) -> List[str]: + return ["mp3", "mp4", "mpeg", "mpga", "m4a", "wav", "webm"] +``` + +**Step 4: Run test to verify it passes** + +Run: `uv run pytest tests/speech/test_openai_whisper.py -v` +Expected: PASS + +**Step 5: Commit** + +```bash +git add src/openjarvis/speech/openai_whisper.py tests/speech/test_openai_whisper.py +git commit -m "feat(speech): implement OpenAIWhisperBackend (cloud STT)" +``` + +--- + +### Task 6: Implement DeepgramBackend + +**Files:** +- Create: `src/openjarvis/speech/deepgram.py` +- Test: `tests/speech/test_deepgram.py` + +**Step 1: Write the failing test** + +Create `tests/speech/test_deepgram.py`: + +```python +"""Tests for Deepgram speech backend.""" + +from unittest.mock import MagicMock, patch + +from openjarvis.speech._stubs import TranscriptionResult + + +def test_deepgram_registers(): + from openjarvis.core.registry import SpeechRegistry + import openjarvis.speech.deepgram # noqa: F401 + + assert SpeechRegistry.contains("deepgram") + + +def test_deepgram_transcribe(): + mock_client = MagicMock() + mock_result = MagicMock() + mock_channel = MagicMock() + mock_alternative = MagicMock() + mock_alternative.transcript = "Hello from Deepgram" + mock_alternative.confidence = 0.92 + mock_channel.alternatives = [mock_alternative] + mock_result.results.channels = [mock_channel] + mock_result.metadata.duration = 1.8 + mock_result.results.channels[0].detected_language = "en" + mock_client.listen.rest.v.return_value.transcribe_file.return_value = mock_result + + with patch("openjarvis.speech.deepgram.DeepgramClient", return_value=mock_client): + from openjarvis.speech.deepgram import DeepgramSpeechBackend + + backend = DeepgramSpeechBackend(api_key="test-key") + result = backend.transcribe(b"fake audio", format="wav") + + assert isinstance(result, TranscriptionResult) + assert result.text == "Hello from Deepgram" + + +def test_deepgram_health(): + with patch("openjarvis.speech.deepgram.DeepgramClient"): + from openjarvis.speech.deepgram import DeepgramSpeechBackend + + backend = DeepgramSpeechBackend(api_key="test-key") + assert backend.health() is True + + +def test_deepgram_health_no_key(): + with patch("openjarvis.speech.deepgram.DeepgramClient"): + from openjarvis.speech.deepgram import DeepgramSpeechBackend + + backend = DeepgramSpeechBackend.__new__(DeepgramSpeechBackend) + backend._client = None + backend._api_key = "" + assert backend.health() is False +``` + +**Step 2: Run test to verify it fails** + +Run: `uv run pytest tests/speech/test_deepgram.py -v` +Expected: FAIL + +**Step 3: Implement DeepgramBackend** + +Create `src/openjarvis/speech/deepgram.py`: + +```python +"""Deepgram speech-to-text backend (cloud).""" + +from __future__ import annotations + +import os +from typing import List, Optional + +from openjarvis.core.registry import SpeechRegistry +from openjarvis.speech._stubs import SpeechBackend, TranscriptionResult + +try: + from deepgram import DeepgramClient, PrerecordedOptions +except ImportError: + DeepgramClient = None # type: ignore[assignment, misc] + PrerecordedOptions = None # type: ignore[assignment, misc] + + +@SpeechRegistry.register("deepgram") +class DeepgramSpeechBackend(SpeechBackend): + """Cloud speech-to-text using Deepgram API.""" + + backend_id = "deepgram" + + def __init__(self, api_key: Optional[str] = None) -> None: + self._api_key = api_key or os.environ.get("DEEPGRAM_API_KEY", "") + self._client = None + if self._api_key and DeepgramClient is not None: + self._client = DeepgramClient(self._api_key) + + def transcribe( + self, + audio: bytes, + *, + format: str = "wav", + language: Optional[str] = None, + ) -> TranscriptionResult: + """Transcribe audio using Deepgram's API.""" + if self._client is None: + raise RuntimeError("Deepgram client not initialized (missing API key?)") + + mime_map = { + "wav": "audio/wav", + "mp3": "audio/mpeg", + "ogg": "audio/ogg", + "flac": "audio/flac", + "webm": "audio/webm", + "m4a": "audio/mp4", + } + mime_type = mime_map.get(format, "audio/wav") + + options_kwargs: dict = {"model": "nova-2", "smart_format": True} + if language: + options_kwargs["language"] = language + else: + options_kwargs["detect_language"] = True + + payload = {"buffer": audio, "mimetype": mime_type} + + if PrerecordedOptions is not None: + options = PrerecordedOptions(**options_kwargs) + else: + options = options_kwargs + + response = self._client.listen.rest.v("1").transcribe_file( + payload, options, + ) + + # Extract transcript from response + channels = response.results.channels + if channels and channels[0].alternatives: + alt = channels[0].alternatives[0] + text = alt.transcript + confidence = getattr(alt, "confidence", None) + else: + text = "" + confidence = None + + detected_lang = None + if channels: + detected_lang = getattr(channels[0], "detected_language", None) + + duration = getattr(response.metadata, "duration", 0.0) + + return TranscriptionResult( + text=text, + language=detected_lang, + confidence=confidence, + duration_seconds=duration, + segments=[], + ) + + def health(self) -> bool: + return self._client is not None and bool(self._api_key) + + def supported_formats(self) -> List[str]: + return ["wav", "mp3", "ogg", "flac", "webm", "m4a"] +``` + +**Step 4: Run test to verify it passes** + +Run: `uv run pytest tests/speech/test_deepgram.py -v` +Expected: PASS + +**Step 5: Commit** + +```bash +git add src/openjarvis/speech/deepgram.py tests/speech/test_deepgram.py +git commit -m "feat(speech): implement DeepgramSpeechBackend (cloud STT)" +``` + +--- + +### Task 7: Speech discovery and package init + +**Files:** +- Create: `src/openjarvis/speech/_discovery.py` +- Modify: `src/openjarvis/speech/__init__.py` +- Test: `tests/speech/test_discovery.py` + +**Step 1: Write the failing test** + +Create `tests/speech/test_discovery.py`: + +```python +"""Tests for speech backend auto-discovery.""" + +from unittest.mock import patch + +from openjarvis.core.config import JarvisConfig, SpeechConfig + + +def test_get_speech_backend_explicit(): + """Explicit backend selection works.""" + from openjarvis.speech._discovery import get_speech_backend + + config = JarvisConfig() + config.speech.backend = "faster-whisper" + + with patch("openjarvis.speech._discovery._create_backend") as mock_create: + from openjarvis.speech._stubs import TranscriptionResult + + mock_backend = type("MockBackend", (), { + "backend_id": "faster-whisper", + "health": lambda self: True, + })() + mock_create.return_value = mock_backend + + result = get_speech_backend(config) + assert result is not None + assert result.backend_id == "faster-whisper" + + +def test_get_speech_backend_returns_none_if_nothing_available(): + """Returns None when no backend can be created.""" + from openjarvis.speech._discovery import get_speech_backend + + config = JarvisConfig() + config.speech.backend = "nonexistent" + + result = get_speech_backend(config) + assert result is None + + +def test_auto_discovery_priority(): + """Auto mode tries backends in priority order.""" + from openjarvis.speech._discovery import DISCOVERY_ORDER + + assert DISCOVERY_ORDER[0] == "faster-whisper" + assert "openai" in DISCOVERY_ORDER + assert "deepgram" in DISCOVERY_ORDER +``` + +**Step 2: Run test to verify it fails** + +Run: `uv run pytest tests/speech/test_discovery.py -v` +Expected: FAIL + +**Step 3: Implement discovery** + +Create `src/openjarvis/speech/_discovery.py`: + +```python +"""Auto-discover available speech-to-text backends.""" + +from __future__ import annotations + +import os +from typing import TYPE_CHECKING, Optional + +if TYPE_CHECKING: + from openjarvis.core.config import JarvisConfig + from openjarvis.speech._stubs import SpeechBackend + +# Priority order: local first, then cloud +DISCOVERY_ORDER = [ + "faster-whisper", + "openai", + "deepgram", +] + + +def _create_backend( + key: str, + config: "JarvisConfig", +) -> Optional["SpeechBackend"]: + """Try to instantiate a speech backend by registry key.""" + from openjarvis.core.registry import SpeechRegistry + + if not SpeechRegistry.contains(key): + return None + + try: + backend_cls = SpeechRegistry.get(key) + + if key == "faster-whisper": + return backend_cls( + model_size=config.speech.model, + device=config.speech.device, + compute_type=config.speech.compute_type, + ) + elif key == "openai": + api_key = os.environ.get("OPENAI_API_KEY", "") + if not api_key: + return None + return backend_cls(api_key=api_key) + elif key == "deepgram": + api_key = os.environ.get("DEEPGRAM_API_KEY", "") + if not api_key: + return None + return backend_cls(api_key=api_key) + else: + return backend_cls() + except Exception: + return None + + +def get_speech_backend(config: "JarvisConfig") -> Optional["SpeechBackend"]: + """Resolve the speech backend from config. + + If ``config.speech.backend`` is ``"auto"``, tries backends in + priority order and returns the first healthy one. + """ + # Trigger registration of built-in backends + import openjarvis.speech # noqa: F401 + + backend_key = config.speech.backend + + if backend_key != "auto": + return _create_backend(backend_key, config) + + # Auto-discovery: try each in priority order + for key in DISCOVERY_ORDER: + backend = _create_backend(key, config) + if backend is not None: + return backend + + return None +``` + +**Step 4: Update `src/openjarvis/speech/__init__.py`** + +```python +"""Speech subsystem — speech-to-text backends.""" + +import importlib + +# Optional backends — each registers itself via @SpeechRegistry.register() +for _mod in ("faster_whisper", "openai_whisper", "deepgram"): + try: + importlib.import_module(f".{_mod}", __name__) + except ImportError: + pass + +__all__ = ["SpeechBackend", "TranscriptionResult", "Segment", "get_speech_backend"] +``` + +**Step 5: Run test to verify it passes** + +Run: `uv run pytest tests/speech/test_discovery.py -v` +Expected: PASS + +**Step 6: Run all speech tests** + +Run: `uv run pytest tests/speech/ -v` +Expected: All PASS + +**Step 7: Commit** + +```bash +git add src/openjarvis/speech/ tests/speech/test_discovery.py +git commit -m "feat(speech): add auto-discovery and wire speech package init" +``` + +--- + +### Task 8: Wire speech into SystemBuilder + +**Files:** +- Modify: `src/openjarvis/system.py:16-43, 296-346, 447-471` +- Test: `tests/speech/test_system_integration.py` + +**Step 1: Write the failing test** + +Create `tests/speech/test_system_integration.py`: + +```python +"""Tests for speech integration in SystemBuilder/JarvisSystem.""" + +from unittest.mock import MagicMock, patch + +from openjarvis.system import JarvisSystem + + +def test_jarvis_system_has_speech_backend(): + """JarvisSystem has a speech_backend attribute.""" + system = JarvisSystem.__new__(JarvisSystem) + assert hasattr(JarvisSystem, "__dataclass_fields__") + assert "speech_backend" in JarvisSystem.__dataclass_fields__ +``` + +**Step 2: Run test to verify it fails** + +Run: `uv run pytest tests/speech/test_system_integration.py -v` +Expected: FAIL with `AssertionError` + +**Step 3: Add speech_backend to JarvisSystem** + +In `src/openjarvis/system.py`, add after line 42 (`operator_manager`): + +```python + speech_backend: Optional[Any] = None # SpeechBackend +``` + +In `SystemBuilder.__init__()`, add after line 306 (`self._sessions`): + +```python + self._speech: Optional[bool] = None +``` + +Add builder method after line 346 (`sessions`): + +```python + def speech(self, enabled: bool) -> SystemBuilder: + self._speech = enabled + return self +``` + +In the `build()` method, before the `system = JarvisSystem(` call (around line 449), add: + +```python + # Set up speech backend + speech_backend = None + speech_enabled = self._speech if self._speech is not None else True + if speech_enabled: + try: + from openjarvis.speech._discovery import get_speech_backend + + speech_backend = get_speech_backend(config) + except Exception: + pass +``` + +Add `speech_backend=speech_backend,` to the `JarvisSystem(...)` constructor call (after `operator_manager`-related line). + +**Step 4: Run test to verify it passes** + +Run: `uv run pytest tests/speech/test_system_integration.py -v` +Expected: PASS + +**Step 5: Run full test suite to verify nothing broke** + +Run: `uv run pytest tests/ -x -q` +Expected: All tests PASS + +**Step 6: Commit** + +```bash +git add src/openjarvis/system.py tests/speech/test_system_integration.py +git commit -m "feat(speech): wire speech backend into SystemBuilder and JarvisSystem" +``` + +--- + +### Task 9: Add speech API endpoints + +**Files:** +- Modify: `src/openjarvis/server/api_routes.py:571-599` +- Modify: `src/openjarvis/server/app.py:45-120` +- Test: `tests/server/test_speech_routes.py` + +**Step 1: Write the failing test** + +Create `tests/server/test_speech_routes.py`: + +```python +"""Tests for speech API endpoints.""" + +import pytest + +fastapi = pytest.importorskip("fastapi") + +from io import BytesIO +from unittest.mock import MagicMock + +from fastapi.testclient import TestClient + +from openjarvis.speech._stubs import TranscriptionResult + + +@pytest.fixture +def mock_speech_backend(): + backend = MagicMock() + backend.backend_id = "mock" + backend.health.return_value = True + backend.transcribe.return_value = TranscriptionResult( + text="Hello world", + language="en", + confidence=0.95, + duration_seconds=1.5, + segments=[], + ) + return backend + + +@pytest.fixture +def app_with_speech(mock_speech_backend): + from fastapi import FastAPI + from openjarvis.server.api_routes import speech_router + + app = FastAPI() + app.state.speech_backend = mock_speech_backend + app.include_router(speech_router) + return app + + +@pytest.fixture +def client(app_with_speech): + return TestClient(app_with_speech) + + +def test_transcribe_endpoint(client, mock_speech_backend): + response = client.post( + "/v1/speech/transcribe", + files={"file": ("test.wav", b"fake audio data", "audio/wav")}, + ) + assert response.status_code == 200 + data = response.json() + assert data["text"] == "Hello world" + assert data["language"] == "en" + assert data["confidence"] == 0.95 + assert data["duration_seconds"] == 1.5 + + +def test_transcribe_no_file(client): + response = client.post("/v1/speech/transcribe") + assert response.status_code == 422 # FastAPI validation error + + +def test_health_endpoint(client): + response = client.get("/v1/speech/health") + assert response.status_code == 200 + data = response.json() + assert data["available"] is True + assert data["backend"] == "mock" + + +def test_health_no_backend(): + from fastapi import FastAPI + from fastapi.testclient import TestClient + from openjarvis.server.api_routes import speech_router + + app = FastAPI() + app.state.speech_backend = None + app.include_router(speech_router) + client = TestClient(app) + + response = client.get("/v1/speech/health") + assert response.status_code == 200 + data = response.json() + assert data["available"] is False +``` + +**Step 2: Run test to verify it fails** + +Run: `uv run pytest tests/server/test_speech_routes.py -v` +Expected: FAIL with `ImportError: cannot import name 'speech_router'` + +**Step 3: Add speech routes to api_routes.py** + +In `src/openjarvis/server/api_routes.py`, add before the `include_all_routes` function (before line 573): + +```python +# ---- Speech routes ---- + +speech_router = APIRouter(prefix="/v1/speech", tags=["speech"]) + + +@speech_router.post("/transcribe") +async def transcribe_speech(request: Request): + """Transcribe uploaded audio to text.""" + backend = getattr(request.app.state, "speech_backend", None) + if backend is None: + raise HTTPException(status_code=501, detail="Speech backend not configured") + + form = await request.form() + audio_file = form.get("file") + if audio_file is None: + raise HTTPException(status_code=400, detail="Missing 'file' field") + + audio_bytes = await audio_file.read() + language = form.get("language") + + # Detect format from filename + filename = getattr(audio_file, "filename", "audio.wav") + ext = filename.rsplit(".", 1)[-1] if "." in filename else "wav" + + result = backend.transcribe(audio_bytes, format=ext, language=language or None) + return { + "text": result.text, + "language": result.language, + "confidence": result.confidence, + "duration_seconds": result.duration_seconds, + } + + +@speech_router.get("/health") +async def speech_health(request: Request): + """Check if a speech backend is available.""" + backend = getattr(request.app.state, "speech_backend", None) + if backend is None: + return {"available": False, "reason": "No speech backend configured"} + return { + "available": backend.health(), + "backend": backend.backend_id, + } +``` + +Add `speech_router` to `include_all_routes()`: + +```python + app.include_router(speech_router) +``` + +Add `"speech_router"` to `__all__`. + +**Step 4: Run test to verify it passes** + +Run: `uv run pytest tests/server/test_speech_routes.py -v` +Expected: PASS + +**Step 5: Wire speech backend into app state** + +In `src/openjarvis/server/app.py`, add `speech_backend=None` parameter to `create_app()`: + +```python +def create_app( + engine, + model: str, + *, + agent=None, + bus=None, + engine_name: str = "", + agent_name: str = "", + channel_bridge=None, + config=None, + speech_backend=None, +) -> FastAPI: +``` + +Add after line 119 (`app.state.session_start = time.time()`): + +```python + app.state.speech_backend = speech_backend +``` + +**Step 6: Wire speech into serve.py** + +In `src/openjarvis/cli/serve.py`, add after the channel setup block (around line 161), before `create_app`: + +```python + # Set up speech backend + speech_backend = None + try: + from openjarvis.speech._discovery import get_speech_backend + + speech_backend = get_speech_backend(config) + if speech_backend: + console.print(f" Speech: [cyan]{speech_backend.backend_id}[/cyan]") + except Exception: + pass +``` + +Add `speech_backend=speech_backend` to the `create_app()` call: + +```python + app = create_app( + engine, model_name, agent=agent, bus=bus, + engine_name=engine_name, agent_name=agent_key or "", + channel_bridge=channel_bridge, config=config, + speech_backend=speech_backend, + ) +``` + +**Step 7: Run all tests** + +Run: `uv run pytest tests/ -x -q` +Expected: All PASS + +**Step 8: Commit** + +```bash +git add src/openjarvis/server/api_routes.py src/openjarvis/server/app.py src/openjarvis/cli/serve.py tests/server/test_speech_routes.py +git commit -m "feat(speech): add /v1/speech/transcribe and /health API endpoints" +``` + +--- + +### Task 10: Add speech client function to frontend API layer + +**Files:** +- Modify: `frontend/src/api/client.ts` + +**Step 1: Add transcribe and speech health functions** + +In `frontend/src/api/client.ts`, add after the existing functions: + +```typescript +export interface TranscriptionResult { + text: string; + language: string | null; + confidence: number | null; + duration_seconds: number; +} + +export interface SpeechHealth { + available: boolean; + backend?: string; + reason?: string; +} + +export async function transcribeAudio(audioBlob: Blob, filename = 'recording.webm'): Promise { + const formData = new FormData(); + formData.append('file', audioBlob, filename); + const res = await fetch(`${BASE}/v1/speech/transcribe`, { + method: 'POST', + body: formData, + }); + if (!res.ok) throw new Error(`Transcription failed: ${res.status}`); + return res.json(); +} + +export async function fetchSpeechHealth(): Promise { + const res = await fetch(`${BASE}/v1/speech/health`); + if (!res.ok) return { available: false }; + return res.json(); +} +``` + +**Step 2: Commit** + +```bash +git add frontend/src/api/client.ts +git commit -m "feat(speech): add transcribeAudio and fetchSpeechHealth to frontend API" +``` + +--- + +### Task 11: Create useSpeech React hook + +**Files:** +- Create: `frontend/src/hooks/useSpeech.ts` + +**Step 1: Create the hook** + +Create `frontend/src/hooks/useSpeech.ts`: + +```typescript +import { useState, useCallback, useRef, useEffect } from 'react'; +import { transcribeAudio, fetchSpeechHealth } from '../api/client'; + +export type SpeechState = 'idle' | 'recording' | 'transcribing'; + +export function useSpeech() { + const [state, setState] = useState('idle'); + const [error, setError] = useState(null); + const [available, setAvailable] = useState(false); + const mediaRecorderRef = useRef(null); + const chunksRef = useRef([]); + const streamRef = useRef(null); + + // Check if speech backend is available on mount + useEffect(() => { + fetchSpeechHealth() + .then((health) => setAvailable(health.available)) + .catch(() => setAvailable(false)); + }, []); + + const startRecording = useCallback(async (): Promise => { + setError(null); + + if (!navigator.mediaDevices?.getUserMedia) { + setError('Microphone not supported in this browser'); + return; + } + + try { + const stream = await navigator.mediaDevices.getUserMedia({ audio: true }); + streamRef.current = stream; + + const recorder = new MediaRecorder(stream); + chunksRef.current = []; + + recorder.ondataavailable = (e) => { + if (e.data.size > 0) chunksRef.current.push(e.data); + }; + + recorder.start(); + mediaRecorderRef.current = recorder; + setState('recording'); + } catch (err) { + setError('Microphone access denied'); + setState('idle'); + } + }, []); + + const stopRecording = useCallback(async (): Promise => { + return new Promise((resolve, reject) => { + const recorder = mediaRecorderRef.current; + if (!recorder || recorder.state !== 'recording') { + reject(new Error('Not recording')); + return; + } + + recorder.onstop = async () => { + setState('transcribing'); + + // Stop all audio tracks + streamRef.current?.getTracks().forEach((track) => track.stop()); + streamRef.current = null; + + const blob = new Blob(chunksRef.current, { type: recorder.mimeType || 'audio/webm' }); + chunksRef.current = []; + + try { + const result = await transcribeAudio(blob); + setState('idle'); + resolve(result.text); + } catch (err) { + setState('idle'); + const msg = err instanceof Error ? err.message : 'Transcription failed'; + setError(msg); + reject(err); + } + }; + + recorder.stop(); + }); + }, []); + + return { + state, + error, + available, + startRecording, + stopRecording, + isRecording: state === 'recording', + isTranscribing: state === 'transcribing', + }; +} +``` + +**Step 2: Commit** + +```bash +git add frontend/src/hooks/useSpeech.ts +git commit -m "feat(speech): add useSpeech React hook for mic recording and transcription" +``` + +--- + +### Task 12: Create MicButton component and integrate into InputArea + +**Files:** +- Create: `frontend/src/components/Chat/MicButton.tsx` +- Modify: `frontend/src/components/Chat/InputArea.tsx` + +**Step 1: Create MicButton component** + +Create `frontend/src/components/Chat/MicButton.tsx`: + +```tsx +import type { SpeechState } from '../../hooks/useSpeech'; + +interface MicButtonProps { + state: SpeechState; + onClick: () => void; + disabled?: boolean; +} + +export function MicButton({ state, onClick, disabled }: MicButtonProps) { + const title = + state === 'recording' + ? 'Stop recording' + : state === 'transcribing' + ? 'Transcribing...' + : 'Voice input'; + + return ( + + ); +} +``` + +**Step 2: Integrate MicButton into InputArea** + +Replace `frontend/src/components/Chat/InputArea.tsx` with the version that includes the mic button. + +Add import at the top: + +```tsx +import { MicButton } from './MicButton'; +import { useSpeech } from '../../hooks/useSpeech'; +``` + +Inside the `InputArea` component, add the speech hook and handler: + +```tsx + const { state: speechState, available: speechAvailable, startRecording, stopRecording, error: speechError } = useSpeech(); + + const handleMicClick = useCallback(async () => { + if (speechState === 'recording') { + try { + const text = await stopRecording(); + if (text) { + setTyped((prev) => (prev ? prev + ' ' + text : text)); + } + } catch { + // Error is captured in speechError + } + } else { + await startRecording(); + } + }, [speechState, startRecording, stopRecording]); +``` + +In the button area (lines 143-155), replace with: + +```tsx + {isStreaming ? ( + + ) : ( +
+ {speechAvailable && ( + + )} + +
+ )} +``` + +**Step 3: Commit** + +```bash +git add frontend/src/components/Chat/MicButton.tsx frontend/src/components/Chat/InputArea.tsx +git commit -m "feat(speech): add MicButton component and integrate into InputArea" +``` + +--- + +### Task 13: Add Tauri command for speech transcription + +**Files:** +- Modify: `desktop/src-tauri/src/lib.rs:168-243` + +**Step 1: Add transcribe_audio command** + +In `desktop/src-tauri/src/lib.rs`, after line 168 (after `run_jarvis_command`), add: + +```rust +/// Transcribe audio via the speech API endpoint. +#[tauri::command] +async fn transcribe_audio( + api_url: String, + audio_data: Vec, + filename: String, +) -> Result { + let url = format!("{}/v1/speech/transcribe", api_url); + let client = reqwest::Client::new(); + + let part = reqwest::multipart::Part::bytes(audio_data) + .file_name(filename) + .mime_str("audio/webm") + .map_err(|e| format!("Failed to create multipart: {}", e))?; + + let form = reqwest::multipart::Form::new().part("file", part); + + let resp = client + .post(&url) + .multipart(form) + .send() + .await + .map_err(|e| format!("Connection failed: {}", e))?; + let body: serde_json::Value = resp + .json() + .await + .map_err(|e| format!("Invalid response: {}", e))?; + Ok(body) +} + +/// Check speech backend health. +#[tauri::command] +async fn speech_health(api_url: String) -> Result { + let url = format!("{}/v1/speech/health", api_url); + let resp = reqwest::get(&url) + .await + .map_err(|e| format!("Connection failed: {}", e))?; + let body: serde_json::Value = resp + .json() + .await + .map_err(|e| format!("Invalid response: {}", e))?; + Ok(body) +} +``` + +Add to the `invoke_handler` list (line 231-243): + +```rust + .invoke_handler(tauri::generate_handler![ + check_health, + fetch_energy, + fetch_telemetry, + fetch_traces, + fetch_trace, + fetch_learning_stats, + fetch_learning_policy, + fetch_memory_stats, + search_memory, + fetch_agents, + run_jarvis_command, + transcribe_audio, + speech_health, + ]) +``` + +**Step 2: Commit** + +```bash +git add desktop/src-tauri/src/lib.rs +git commit -m "feat(speech): add transcribe_audio and speech_health Tauri commands" +``` + +--- + +### Task 14: Add optional dependencies to pyproject.toml + +**Files:** +- Modify: `pyproject.toml:19-104` + +**Step 1: Add speech extras** + +In `pyproject.toml`, after line 99 (`dashboard = ["textual>=0.80"]`), add: + +```toml +speech = ["faster-whisper>=1.0"] +speech-deepgram = ["deepgram-sdk>=3.0"] +``` + +The OpenAI backend uses the existing `openai` dep from `inference-cloud`. + +**Step 2: Run tests to verify nothing broke** + +Run: `uv run pytest tests/ -x -q` +Expected: All PASS + +**Step 3: Commit** + +```bash +git add pyproject.toml +git commit -m "feat(speech): add speech and speech-deepgram optional dependencies" +``` + +--- + +### Task 15: Run full test suite and update CLAUDE.md + +**Files:** +- Modify: `CLAUDE.md` + +**Step 1: Run full test suite** + +Run: `uv run pytest tests/ -v --tb=short` +Expected: All pass, no regressions + +**Step 2: Run lint** + +Run: `uv run ruff check src/openjarvis/speech/ tests/speech/` +Expected: Clean or fix any issues + +**Step 3: Update CLAUDE.md** + +Add speech subsystem to the Architecture section, add `[speech]` config documentation, add `jarvis serve` speech output mention, update file counts. + +Key additions: +- Speech subsystem description in the Five Pillars or Cross-cutting Systems section +- `[speech]` config section documentation +- Speech API endpoints (`POST /v1/speech/transcribe`, `GET /v1/speech/health`) +- Speech optional dependencies (`openjarvis[speech]`, `openjarvis[speech-deepgram]`) + +**Step 4: Commit** + +```bash +git add CLAUDE.md +git commit -m "docs: update CLAUDE.md with speech subsystem documentation" +``` + +--- + +## Summary + +| Task | What | Files | Est. Lines | +|------|------|-------|-----------| +| 1 | SpeechRegistry | registry.py, conftest.py | ~10 | +| 2 | SpeechConfig | config.py, test | ~30 | +| 3 | SpeechBackend ABC | _stubs.py, test | ~70 | +| 4 | FasterWhisperBackend | faster_whisper.py, test | ~130 | +| 5 | OpenAIWhisperBackend | openai_whisper.py, test | ~100 | +| 6 | DeepgramBackend | deepgram.py, test | ~120 | +| 7 | Discovery + init | _discovery.py, __init__.py, test | ~90 | +| 8 | SystemBuilder wiring | system.py, test | ~30 | +| 9 | API endpoints | api_routes.py, app.py, serve.py, test | ~110 | +| 10 | Frontend API client | client.ts | ~30 | +| 11 | useSpeech hook | useSpeech.ts | ~90 | +| 12 | MicButton + InputArea | MicButton.tsx, InputArea.tsx | ~100 | +| 13 | Tauri commands | lib.rs | ~50 | +| 14 | pyproject.toml | pyproject.toml | ~5 | +| 15 | Tests + docs | CLAUDE.md | ~30 | +| **Total** | | **~20 files** | **~1000 lines** | diff --git a/frontend/src/components/Chat/InputArea.tsx b/frontend/src/components/Chat/InputArea.tsx index ca5b7e47..384d1f0a 100644 --- a/frontend/src/components/Chat/InputArea.tsx +++ b/frontend/src/components/Chat/InputArea.tsx @@ -3,6 +3,8 @@ import { Send, Square, Paperclip } from 'lucide-react'; import { useAppStore, generateId } from '../../lib/store'; import { streamChat } from '../../lib/sse'; import { fetchSavings } from '../../lib/api'; +import { MicButton } from './MicButton'; +import { useSpeech } from '../../hooks/useSpeech'; import type { ChatMessage, ToolCallInfo, TokenUsage } from '../../types'; export function InputArea() { @@ -21,7 +23,23 @@ export function InputArea() { const setStreamState = useAppStore((s) => s.setStreamState); const resetStream = useAppStore((s) => s.resetStream); - // Auto-resize textarea + const { state: speechState, available: speechAvailable, startRecording, stopRecording } = useSpeech(); + + const handleMicClick = useCallback(async () => { + if (speechState === 'recording') { + try { + const text = await stopRecording(); + if (text) { + setInput((prev) => (prev ? prev + ' ' + text : text)); + } + } catch { + // Error is captured in useSpeech + } + } else { + await startRecording(); + } + }, [speechState, startRecording, stopRecording]); + useEffect(() => { const el = textareaRef.current; if (!el) return; @@ -238,18 +256,27 @@ export function InputArea() { ) : ( - +
+ {speechAvailable && ( + + )} + +
)}
diff --git a/frontend/src/components/Chat/MicButton.tsx b/frontend/src/components/Chat/MicButton.tsx new file mode 100644 index 00000000..177bf2f3 --- /dev/null +++ b/frontend/src/components/Chat/MicButton.tsx @@ -0,0 +1,53 @@ +import type { SpeechState } from '../../hooks/useSpeech'; + +interface MicButtonProps { + state: SpeechState; + onClick: () => void; + disabled?: boolean; +} + +export function MicButton({ state, onClick, disabled }: MicButtonProps) { + const title = + state === 'recording' + ? 'Stop recording' + : state === 'transcribing' + ? 'Transcribing...' + : 'Voice input'; + + return ( + + ); +} diff --git a/frontend/src/hooks/useSpeech.ts b/frontend/src/hooks/useSpeech.ts new file mode 100644 index 00000000..6e16e258 --- /dev/null +++ b/frontend/src/hooks/useSpeech.ts @@ -0,0 +1,92 @@ +import { useState, useCallback, useRef, useEffect } from 'react'; +import { transcribeAudio, fetchSpeechHealth } from '../lib/api'; + +export type SpeechState = 'idle' | 'recording' | 'transcribing'; + +export function useSpeech() { + const [state, setState] = useState('idle'); + const [error, setError] = useState(null); + const [available, setAvailable] = useState(false); + const mediaRecorderRef = useRef(null); + const chunksRef = useRef([]); + const streamRef = useRef(null); + + // Check if speech backend is available on mount + useEffect(() => { + fetchSpeechHealth() + .then((health) => setAvailable(health.available)) + .catch(() => setAvailable(false)); + }, []); + + const startRecording = useCallback(async (): Promise => { + setError(null); + + if (!navigator.mediaDevices?.getUserMedia) { + setError('Microphone not supported in this browser'); + return; + } + + try { + const stream = await navigator.mediaDevices.getUserMedia({ audio: true }); + streamRef.current = stream; + + const recorder = new MediaRecorder(stream); + chunksRef.current = []; + + recorder.ondataavailable = (e) => { + if (e.data.size > 0) chunksRef.current.push(e.data); + }; + + recorder.start(); + mediaRecorderRef.current = recorder; + setState('recording'); + } catch (err) { + setError('Microphone access denied'); + setState('idle'); + } + }, []); + + const stopRecording = useCallback(async (): Promise => { + return new Promise((resolve, reject) => { + const recorder = mediaRecorderRef.current; + if (!recorder || recorder.state !== 'recording') { + reject(new Error('Not recording')); + return; + } + + recorder.onstop = async () => { + setState('transcribing'); + + // Stop all audio tracks + streamRef.current?.getTracks().forEach((track) => track.stop()); + streamRef.current = null; + + const blob = new Blob(chunksRef.current, { type: recorder.mimeType || 'audio/webm' }); + chunksRef.current = []; + + try { + const result = await transcribeAudio(blob); + setState('idle'); + resolve(result.text); + } catch (err) { + setState('idle'); + const msg = err instanceof Error ? err.message : 'Transcription failed'; + setError(msg); + reject(err); + } + }; + + recorder.stop(); + }); + }, []); + + return { + state, + error, + available, + startRecording, + stopRecording, + isRecording: state === 'recording', + isTranscribing: state === 'transcribing', + }; +} diff --git a/frontend/src/lib/api.ts b/frontend/src/lib/api.ts index f6252488..25a2ef28 100644 --- a/frontend/src/lib/api.ts +++ b/frontend/src/lib/api.ts @@ -125,3 +125,55 @@ export async function fetchTraces(limit: number = 50): Promise { if (!res.ok) throw new Error(`Failed: ${res.status}`); return res.json(); } + +// --------------------------------------------------------------------------- +// Speech +// --------------------------------------------------------------------------- + +export interface TranscriptionResult { + text: string; + language: string | null; + confidence: number | null; + duration_seconds: number; +} + +export interface SpeechHealth { + available: boolean; + backend?: string; + reason?: string; +} + +export async function transcribeAudio(audioBlob: Blob, filename = 'recording.webm'): Promise { + if (isTauri()) { + try { + const buffer = await audioBlob.arrayBuffer(); + return await tauriInvoke('transcribe_audio', { + audioData: Array.from(new Uint8Array(buffer)), + filename, + }); + } catch { + // Fall through to fetch + } + } + const formData = new FormData(); + formData.append('file', audioBlob, filename); + const res = await fetch(`${getBase()}/v1/speech/transcribe`, { + method: 'POST', + body: formData, + }); + if (!res.ok) throw new Error(`Transcription failed: ${res.status}`); + return res.json(); +} + +export async function fetchSpeechHealth(): Promise { + if (isTauri()) { + try { + return await tauriInvoke('speech_health'); + } catch { + return { available: false }; + } + } + const res = await fetch(`${getBase()}/v1/speech/health`); + if (!res.ok) return { available: false }; + return res.json(); +} diff --git a/pyproject.toml b/pyproject.toml index 08451dce..555b19bd 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -24,9 +24,6 @@ dev = [ "respx>=0.22", "ruff>=0.4", ] -inference-ollama = [] -inference-vllm = [] -inference-llamacpp = [] inference-mlx = ["mlx-lm>=0.19; sys_platform == 'darwin'"] inference-cloud = [ "openai>=1.30", @@ -55,31 +52,15 @@ server = [ "uvicorn>=0.30", "pydantic>=2.0", ] -agents = [] openhands = ["openhands-sdk>=1.0; python_version >= '3.12'"] -claude-code = [] gpu-metrics = ["pynvml>=12.0"] energy-amd = ["amdsmi>=6.1"] energy-apple = ["zeus-ml[apple]"] energy-all = ["pynvml>=12.0", "amdsmi>=6.1", "zeus-ml[apple]"] -learning = [] orchestrator-training = ["torch>=2.0", "transformers>=4.40"] channel-telegram = ["python-telegram-bot>=21.0"] channel-discord = ["discord.py>=2.3"] channel-slack = ["slack-sdk>=3.27"] -channel-webhook = [] -channel-email = [] -channel-whatsapp = [] -channel-signal = [] -channel-google-chat = [] -channel-irc = [] -channel-webchat = [] -channel-teams = [] -channel-matrix = [] -channel-mattermost = [] -channel-feishu = [] -channel-bluebubbles = [] -channel-whatsapp-baileys = [] channel-line = ["line-bot-sdk>=3.0"] channel-viber = ["viberbot>=1.0"] channel-messenger = ["pymessenger>=0.0.7"] @@ -97,6 +78,10 @@ scheduler = ["croniter>=2.0"] security-signing = ["cryptography>=43"] sandbox-wasm = ["wasmtime>=25"] dashboard = ["textual>=0.80"] +speech = ["faster-whisper>=1.0"] +speech-deepgram = ["deepgram-sdk>=3.0"] +eval-wandb = ["wandb>=0.17"] +eval-sheets = ["gspread>=6.0", "google-auth>=2.0"] docs = [ "mkdocs>=1.6", "mkdocs-material>=9.5", diff --git a/src/openjarvis/cli/__init__.py b/src/openjarvis/cli/__init__.py index 35532e03..20209f06 100644 --- a/src/openjarvis/cli/__init__.py +++ b/src/openjarvis/cli/__init__.py @@ -14,6 +14,7 @@ from openjarvis.cli.chat_cmd import chat from openjarvis.cli.daemon_cmd import restart, start, status, stop from openjarvis.cli.doctor_cmd import doctor from openjarvis.cli.eval_cmd import eval_group +from openjarvis.cli.host_cmd import host from openjarvis.cli.init_cmd import init from openjarvis.cli.memory_cmd import memory from openjarvis.cli.model import model @@ -64,6 +65,7 @@ cli.add_command(vault, "vault") cli.add_command(add, "add") cli.add_command(operators, "operators") cli.add_command(eval_group, "eval") +cli.add_command(host, "host") cli.add_command(quickstart, "quickstart") diff --git a/src/openjarvis/cli/ask.py b/src/openjarvis/cli/ask.py index 6b0cbdd8..96a1761e 100644 --- a/src/openjarvis/cli/ask.py +++ b/src/openjarvis/cli/ask.py @@ -4,13 +4,15 @@ from __future__ import annotations import json as json_mod import sys +import time import click from rich.console import Console +from rich.table import Table from openjarvis.cli.hints import hint_no_engine from openjarvis.core.config import load_config -from openjarvis.core.events import EventBus +from openjarvis.core.events import EventBus, EventType from openjarvis.core.types import Message, Role from openjarvis.engine import ( EngineConnectionError, @@ -147,6 +149,119 @@ def _run_agent( return agent.run(query_text, context=ctx) +def _print_profile( + bus: EventBus, + wall_seconds: float, + engine_name: str, + model_name: str, + console: Console, +) -> None: + """Print an inference telemetry profile table from EventBus history.""" + # Collect all INFERENCE_END events (agents may fire multiple) + inf_events = [ + e for e in bus.history if e.event_type == EventType.INFERENCE_END + ] + if not inf_events: + console.print("[dim]No inference telemetry recorded.[/dim]") + return + + total_calls = len(inf_events) + + # Aggregate across all inference calls + total_latency = sum(e.data.get("latency", 0.0) for e in inf_events) + total_tokens = sum( + e.data.get("usage", {}).get("completion_tokens", 0) + or e.data.get("completion_tokens", 0) + for e in inf_events + ) + total_prompt = sum( + e.data.get("usage", {}).get("prompt_tokens", 0) + for e in inf_events + ) + total_energy = sum(e.data.get("energy_joules", 0.0) for e in inf_events) + avg_power = 0.0 + power_vals = [e.data.get("power_watts", 0.0) for e in inf_events + if e.data.get("power_watts", 0.0) > 0] + if power_vals: + avg_power = sum(power_vals) / len(power_vals) + + throughput = total_tokens / total_latency if total_latency > 0 else 0.0 + energy_per_tok = total_energy / total_tokens if total_tokens > 0 else 0.0 + tpw = throughput / avg_power if avg_power > 0 else 0.0 + tok_per_j = total_tokens / total_energy if total_energy > 0 else 0.0 + + last = inf_events[-1].data + ttft = last.get("ttft", 0.0) + prefill_lat = last.get("prefill_latency_seconds", 0.0) + decode_lat = last.get("decode_latency_seconds", 0.0) + prefill_e = sum(e.data.get("prefill_energy_joules", 0.0) for e in inf_events) + decode_e = sum(e.data.get("decode_energy_joules", 0.0) for e in inf_events) + gpu_util = last.get("gpu_utilization_pct", 0.0) + gpu_mem = last.get("gpu_memory_used_gb", 0.0) + gpu_temp = last.get("gpu_temperature_c", 0.0) + mean_itl = last.get("mean_itl_ms", 0.0) + e_method = last.get("energy_method", "") + e_vendor = last.get("energy_vendor", "") + + # Build the profile table + table = Table( + title=f"Inference Profile ({engine_name} / {model_name})", + show_header=True, + header_style="bold bright_white", + border_style="bright_blue", + title_style="bold cyan", + ) + table.add_column("Metric", style="cyan", no_wrap=True) + table.add_column("Value", justify="right", style="green") + + def _row(label: str, val: str) -> None: + table.add_row(label, val) + + _row("Wall time", f"{wall_seconds:.3f} s") + _row("Inference calls", str(total_calls)) + _row("Total latency", f"{total_latency:.3f} s") + if ttft > 0: + _row("TTFT", f"{ttft * 1000:.1f} ms") + if prefill_lat > 0: + _row("Prefill latency", f"{prefill_lat * 1000:.1f} ms") + if decode_lat > 0: + _row("Decode latency", f"{decode_lat:.3f} s") + if mean_itl > 0: + _row("Mean ITL", f"{mean_itl:.2f} ms") + _row("Prompt tokens", str(total_prompt)) + _row("Completion tokens", str(total_tokens)) + _row("Throughput", f"{throughput:.1f} tok/s") + + if total_energy > 0: + _row("", "") # separator + _row("Energy", f"{total_energy:.4f} J") + if prefill_e > 0: + _row(" Prefill energy", f"{prefill_e:.4f} J") + if decode_e > 0: + _row(" Decode energy", f"{decode_e:.4f} J") + _row("Energy / output token", f"{energy_per_tok:.6f} J") + _row("Tokens / joule", f"{tok_per_j:.1f}") + _row("Throughput / watt (IPW)", f"{tpw:.2f} tok/s/W") + if avg_power > 0: + _row("Avg power draw", f"{avg_power:.1f} W") + if e_vendor: + _row("Energy vendor", e_vendor) + if e_method: + _row("Energy method", e_method) + + if gpu_util > 0 or gpu_mem > 0 or gpu_temp > 0: + _row("", "") # separator + if gpu_util > 0: + _row("GPU utilization", f"{gpu_util:.1f} %") + if gpu_mem > 0: + _row("GPU memory used", f"{gpu_mem:.2f} GB") + if gpu_temp > 0: + _row("GPU temperature", f"{gpu_temp:.0f} °C") + + console.print() + console.print(table) + + @click.command() @click.argument("query", nargs=-1, required=True) @click.option("-m", "--model", "model_name", default=None, help="Model to use.") @@ -173,6 +288,10 @@ def _run_agent( "--tools", "tool_names", default=None, help="Comma-separated tool names to enable (e.g. calculator,think).", ) +@click.option( + "--profile", "enable_profile", is_flag=True, + help="Print inference telemetry profile (latency, tokens, energy, IPW).", +) def ask( query: tuple[str, ...], model_name: str | None, @@ -184,11 +303,14 @@ def ask( no_context: bool, agent_name: str | None, tool_names: str | None, + enable_profile: bool, ) -> None: """Ask Jarvis a question.""" console = Console(stderr=True) query_text = " ".join(query) + wall_start = time.monotonic() if enable_profile else None + # Load config config = load_config() @@ -228,7 +350,8 @@ def ask( # Wrap engine with InstrumentedEngine for telemetry (energy + GPU metrics) energy_monitor = None - if config.telemetry.gpu_metrics: + want_energy = config.telemetry.gpu_metrics or enable_profile + if want_energy: try: from openjarvis.telemetry.energy_monitor import create_energy_monitor @@ -288,6 +411,12 @@ def ask( else: click.echo(result.content) + if enable_profile: + _print_profile( + bus, time.monotonic() - wall_start, + engine_name, model_name, console, + ) + if telem_store is not None: try: telem_store.close() @@ -341,6 +470,12 @@ def ask( else: click.echo(result.get("content", "")) + if enable_profile: + _print_profile( + bus, time.monotonic() - wall_start, + engine_name, model_name, console, + ) + # Cleanup if energy_monitor is not None: try: diff --git a/src/openjarvis/cli/eval_cmd.py b/src/openjarvis/cli/eval_cmd.py index ce5d116b..3a9dc8dc 100644 --- a/src/openjarvis/cli/eval_cmd.py +++ b/src/openjarvis/cli/eval_cmd.py @@ -113,6 +113,34 @@ def eval_list() -> None: "-o", "--output", "output_path", default=None, type=click.Path(), help="Output JSONL path.", ) +@click.option( + "--wandb-project", "wandb_project", default="", + help="W&B project name (enables W&B tracking).", +) +@click.option( + "--wandb-entity", "wandb_entity", default="", + help="W&B entity (team or user).", +) +@click.option( + "--wandb-tags", "wandb_tags", default="", + help="Comma-separated W&B tags.", +) +@click.option( + "--wandb-group", "wandb_group", default="", + help="W&B run group.", +) +@click.option( + "--sheets-id", "sheets_spreadsheet_id", default="", + help="Google Sheets spreadsheet ID.", +) +@click.option( + "--sheets-worksheet", "sheets_worksheet", default="Results", + help="Google Sheets worksheet name.", +) +@click.option( + "--sheets-creds", "sheets_credentials_path", default="", + help="Path to Google service account JSON.", +) @click.option( "-v", "--verbose", "verbose", is_flag=True, default=False, help="Verbose logging.", @@ -127,6 +155,13 @@ def eval_run( tools: str, telemetry: bool, output_path: Optional[str], + wandb_project: str, + wandb_entity: str, + wandb_tags: str, + wandb_group: str, + sheets_spreadsheet_id: str, + sheets_worksheet: str, + sheets_credentials_path: str, verbose: bool, ) -> None: """Run evaluation benchmarks.""" @@ -216,6 +251,13 @@ def eval_run( tools=tool_list, output_path=output_path, telemetry=telemetry, + wandb_project=wandb_project, + wandb_entity=wandb_entity, + wandb_tags=wandb_tags, + wandb_group=wandb_group, + sheets_spreadsheet_id=sheets_spreadsheet_id, + sheets_worksheet=sheets_worksheet, + sheets_credentials_path=sheets_credentials_path, ) try: diff --git a/src/openjarvis/cli/host_cmd.py b/src/openjarvis/cli/host_cmd.py new file mode 100644 index 00000000..7bdf1f62 --- /dev/null +++ b/src/openjarvis/cli/host_cmd.py @@ -0,0 +1,366 @@ +"""``jarvis host`` — download and serve a model locally with auto backend setup.""" + +from __future__ import annotations + +import os +import shutil +import signal +import subprocess +import sys +from typing import Optional + +import click +from rich.console import Console +from rich.panel import Panel +from rich.table import Table + +_BACKENDS = { + "mlx": { + "display": "MLX (Apple Silicon)", + "package": "mlx-lm", + "import_check": "mlx_lm", + "pip_spec": "mlx-lm>=0.19", + "uv_extra": "inference-mlx", + "platform": "darwin", + "default_port": 8080, + }, + "vllm": { + "display": "vLLM (NVIDIA GPU)", + "package": "vllm", + "import_check": "vllm", + "pip_spec": "vllm", + "uv_extra": None, + "platform": "linux", + "default_port": 8000, + }, + "sglang": { + "display": "SGLang (NVIDIA GPU)", + "package": "sglang", + "import_check": "sglang", + "pip_spec": "sglang[all]", + "uv_extra": None, + "platform": None, + "default_port": 30000, + }, + "ollama": { + "display": "Ollama", + "package": "ollama", + "import_check": None, + "pip_spec": None, + "uv_extra": None, + "platform": None, + "default_port": 11434, + "binary": "ollama", + }, + "llamacpp": { + "display": "llama.cpp", + "package": "llama.cpp", + "import_check": None, + "pip_spec": None, + "uv_extra": None, + "platform": None, + "default_port": 8080, + "binary": "llama-server", + }, +} + + +def _is_package_available(backend: str) -> bool: + """Check whether the backend's Python package or binary is importable/available.""" + info = _BACKENDS[backend] + + if info.get("import_check"): + try: + __import__(info["import_check"]) + return True + except ImportError: + return False + + binary = info.get("binary") + if binary: + return shutil.which(binary) is not None + + return False + + +def _detect_backend() -> str | None: + """Auto-detect the best backend for the current platform.""" + import platform + + system = platform.system().lower() + + if system == "darwin": + try: + result = subprocess.run( + ["sysctl", "-n", "machdep.cpu.brand_string"], + capture_output=True, + text=True, + ) + if "Apple" in result.stdout: + return "mlx" + except FileNotFoundError: + pass + return "ollama" + + if system == "linux": + if shutil.which("nvidia-smi"): + return "vllm" + return "ollama" + + return "ollama" + + +def _install_backend(backend: str, console: Console) -> bool: + """Prompt the user and install the backend package. Returns True on success.""" + info = _BACKENDS[backend] + display = info["display"] + + console.print() + console.print( + f"[yellow]Backend [bold]{display}[/bold] is not installed.[/yellow]" + ) + + uv_available = shutil.which("uv") is not None + in_uv_project = os.path.exists("pyproject.toml") and os.path.exists("uv.lock") + + if info.get("binary") and not info.get("pip_spec"): + return _install_binary_backend(backend, console) + + pip_spec = info["pip_spec"] + uv_extra = info.get("uv_extra") + + if uv_available and in_uv_project and uv_extra: + install_cmd = ["uv", "pip", "install", pip_spec] + install_label = f"uv pip install {pip_spec}" + elif uv_available: + install_cmd = ["uv", "pip", "install", pip_spec] + install_label = f"uv pip install {pip_spec}" + else: + install_cmd = [sys.executable, "-m", "pip", "install", pip_spec] + install_label = f"pip install {pip_spec}" + + console.print(f"\n Install command: [cyan]{install_label}[/cyan]\n") + if not click.confirm("Install now?", default=True): + console.print("[dim]Skipped. Install manually and retry.[/dim]") + return False + + console.print(f"[bold]Running:[/bold] {install_label}") + result = subprocess.run(install_cmd) + if result.returncode != 0: + console.print( + f"[red]Installation failed (exit {result.returncode}).[/red]" + ) + return False + + console.print(f"[green]{display} installed successfully.[/green]\n") + return True + + +def _install_binary_backend(backend: str, console: Console) -> bool: + """Guide the user through installing a binary backend (Ollama, llama.cpp).""" + info = _BACKENDS[backend] + binary = info["binary"] + + instructions = { + "ollama": ( + "Install Ollama:\n" + "\n" + " macOS / Linux:\n" + " curl -fsSL https://ollama.com/install.sh | sh\n" + "\n" + " Or download from: https://ollama.com/download" + ), + "llamacpp": ( + "Install llama.cpp:\n" + "\n" + " macOS:\n" + " brew install llama.cpp\n" + "\n" + " From source:\n" + " https://github.com/ggerganov/llama.cpp" + ), + } + + console.print() + console.print( + Panel( + instructions.get( + backend, + f"Install {binary} and ensure it's on PATH.", + ), + title=f"{info['display']} Installation", + border_style="yellow", + ) + ) + + if backend == "ollama": + import platform + + system = platform.system().lower() + if system in ("linux", "darwin"): + if click.confirm("Run the Ollama install script now?", default=True): + console.print("[bold]Running Ollama installer...[/bold]") + result = subprocess.run( + ["sh", "-c", "curl -fsSL https://ollama.com/install.sh | sh"], + ) + if result.returncode == 0 and shutil.which("ollama"): + console.print("[green]Ollama installed successfully.[/green]\n") + return True + console.print( + "[red]Installation may have failed. " + "Check above for errors.[/red]" + ) + return False + + console.print("[dim]Install manually and retry.[/dim]") + return False + + +def _build_serve_command(backend: str, model: str, port: int) -> list[str]: + """Build the subprocess command list to start the inference server.""" + if backend == "mlx": + return [ + sys.executable, + "-m", + "mlx_lm.server", + "--model", + model, + "--port", + str(port), + ] + + if backend == "vllm": + return ["vllm", "serve", model, "--port", str(port)] + + if backend == "sglang": + return [ + sys.executable, + "-m", + "sglang.launch_server", + "--model-path", + model, + "--port", + str(port), + ] + + if backend == "ollama": + return ["ollama", "run", model] + + if backend == "llamacpp": + return ["llama-server", "-m", model, "--port", str(port)] + + raise ValueError(f"Unknown backend: {backend}") + + +@click.command() +@click.argument("model") +@click.option( + "-b", + "--backend", + type=click.Choice(list(_BACKENDS.keys()), case_sensitive=False), + default=None, + help="Inference backend to use. Auto-detected if omitted.", +) +@click.option( + "-p", + "--port", + type=int, + default=None, + help="Port to serve on (default depends on backend).", +) +@click.option( + "--trust-remote-code", + is_flag=True, + default=False, + help="Pass --trust-remote-code to the backend.", +) +def host( + model: str, + backend: Optional[str], + port: Optional[int], + trust_remote_code: bool, +) -> None: + """Download (if needed) and serve a model locally. + + Examples: + + \b + jarvis host mlx-community/Qwen2.5-7B-4bit --backend mlx + jarvis host Qwen/Qwen3-8B --backend vllm + jarvis host qwen3:8b --backend ollama + jarvis host meta-llama/Llama-3-8B -b sglang + """ + console = Console() + + if backend is None: + detected = _detect_backend() + if detected is None: + console.print("[red]Could not auto-detect a suitable backend.[/red]") + console.print("Specify one with [cyan]--backend[/cyan].") + raise SystemExit(1) + backend = detected + name = _BACKENDS[backend]["display"] + console.print( + f"Auto-detected backend: [bold cyan]{name}[/bold cyan]" + ) + + info = _BACKENDS[backend] + + if not _is_package_available(backend): + if not _install_backend(backend, console): + raise SystemExit(1) + if not _is_package_available(backend): + console.print( + f"[red]{info['display']} still not available after install.[/red]" + ) + console.print( + "You may need to restart your shell or " + "activate the correct environment." + ) + raise SystemExit(1) + + serve_port = port or info["default_port"] + cmd = _build_serve_command(backend, model, serve_port) + + if trust_remote_code: + if backend in ("vllm", "sglang"): + cmd.append("--trust-remote-code") + elif backend == "mlx": + cmd.extend(["--trust-remote-code", "True"]) + + host_url = f"http://localhost:{serve_port}" + + table = Table.grid(padding=(0, 2)) + table.add_row("[bold]Backend:[/bold]", info["display"]) + table.add_row("[bold]Model:[/bold]", model) + table.add_row("[bold]Endpoint:[/bold]", host_url) + table.add_row("[bold]Command:[/bold]", " ".join(cmd)) + + console.print() + console.print(Panel(table, title="Hosting Model", border_style="green")) + console.print() + + if backend != "ollama": + console.print( + f"[dim]The model server will be available at {host_url}[/dim]" + ) + console.print( + "[dim]OpenJarvis will auto-discover it. " + "Press Ctrl+C to stop.[/dim]\n" + ) + + try: + proc = subprocess.Popen(cmd) + proc.wait() + except KeyboardInterrupt: + console.print("\n[yellow]Shutting down model server...[/yellow]") + proc.send_signal(signal.SIGTERM) + try: + proc.wait(timeout=10) + except subprocess.TimeoutExpired: + proc.kill() + console.print("[green]Server stopped.[/green]") + except FileNotFoundError: + console.print(f"[red]Command not found:[/red] {cmd[0]}") + console.print(f"Make sure {info['display']} is installed and on your PATH.") + raise SystemExit(1) diff --git a/src/openjarvis/cli/init_cmd.py b/src/openjarvis/cli/init_cmd.py index 0eeaf3de..f489f285 100644 --- a/src/openjarvis/cli/init_cmd.py +++ b/src/openjarvis/cli/init_cmd.py @@ -5,6 +5,8 @@ from __future__ import annotations import click from rich.console import Console from rich.panel import Panel +from pathlib import Path +from typing import Optional from openjarvis.core.config import ( DEFAULT_CONFIG_DIR, @@ -113,7 +115,12 @@ def _next_steps_text(engine: str) -> str: @click.option( "--force", is_flag=True, help="Overwrite existing config without prompting." ) -def init(force: bool) -> None: +@click.option( + "--config", + type=click.Path(exists=True), + help="Path to config file to use.", +) +def init(force: bool, config: Optional[Path]) -> None: """Detect hardware and generate ~/.openjarvis/config.toml.""" console = Console() @@ -137,10 +144,16 @@ def init(force: bool) -> None: else: console.print(" GPU : none detected") - toml_content = generate_default_toml(hw) + if config: + toml_content = config.read_text() + else: + toml_content = generate_default_toml(hw) DEFAULT_CONFIG_DIR.mkdir(parents=True, exist_ok=True) - DEFAULT_CONFIG_PATH.write_text(toml_content) + if config: + config.write_text(toml_content) + else: + DEFAULT_CONFIG_PATH.write_text(toml_content) console.print() console.print( @@ -157,3 +170,4 @@ def init(force: bool) -> None: border_style="cyan", ) ) + diff --git a/src/openjarvis/cli/serve.py b/src/openjarvis/cli/serve.py index bba728a3..baa3a1f4 100644 --- a/src/openjarvis/cli/serve.py +++ b/src/openjarvis/cli/serve.py @@ -185,6 +185,16 @@ def serve( console.print(f"[yellow]Channel failed to start: {exc}[/yellow]") channel_bridge = None + # Set up speech backend + speech_backend = None + try: + from openjarvis.speech._discovery import get_speech_backend + speech_backend = get_speech_backend(config) + if speech_backend: + console.print(f" Speech: [cyan]{speech_backend.backend_id}[/cyan]") + except Exception: + pass + # Create app from openjarvis.server.app import create_app @@ -192,6 +202,7 @@ def serve( engine, model_name, agent=agent, bus=bus, engine_name=engine_name, agent_name=agent_key or "", channel_bridge=channel_bridge, config=config, + speech_backend=speech_backend, ) console.print( diff --git a/src/openjarvis/core/config.py b/src/openjarvis/core/config.py index 1d16f373..8f1fa07b 100644 --- a/src/openjarvis/core/config.py +++ b/src/openjarvis/core/config.py @@ -830,6 +830,17 @@ class OperatorsConfig: auto_activate: str = "" # Comma-separated operator IDs +@dataclass(slots=True) +class SpeechConfig: + """Speech-to-text settings.""" + + backend: str = "auto" # "auto", "faster-whisper", "openai", "deepgram" + model: str = "base" # Whisper model size: tiny, base, small, medium, large-v3 + language: str = "" # Empty = auto-detect + device: str = "auto" # "auto", "cpu", "cuda" + compute_type: str = "float16" # "float16", "int8", "float32" + + @dataclass class JarvisConfig: """Top-level configuration for OpenJarvis.""" @@ -851,6 +862,7 @@ class JarvisConfig: sessions: SessionConfig = field(default_factory=SessionConfig) a2a: A2AConfig = field(default_factory=A2AConfig) operators: OperatorsConfig = field(default_factory=OperatorsConfig) + speech: SpeechConfig = field(default_factory=SpeechConfig) @property def memory(self) -> StorageConfig: @@ -945,6 +957,7 @@ def load_config(path: Optional[Path] = None) -> JarvisConfig: "server", "telemetry", "traces", "security", "channel", "tools", "sandbox", "scheduler", "workflow", "sessions", "a2a", "operators", + "speech", ) for section_name in top_sections: if section_name in data: @@ -1196,6 +1209,7 @@ __all__ = [ "SessionConfig", "SignalChannelConfig", "SlackChannelConfig", + "SpeechConfig", "StorageConfig", "TeamsChannelConfig", "TelegramChannelConfig", diff --git a/src/openjarvis/core/registry.py b/src/openjarvis/core/registry.py index 63ba014c..cbbc1868 100644 --- a/src/openjarvis/core/registry.py +++ b/src/openjarvis/core/registry.py @@ -137,6 +137,10 @@ class SkillRegistry(RegistryBase[Any]): """Registry for skill manifests.""" +class SpeechRegistry(RegistryBase[Any]): + """Registry for speech backend implementations.""" + + __all__ = [ "AgentRegistry", "BenchmarkRegistry", @@ -148,5 +152,6 @@ __all__ = [ "RegistryBase", "RouterPolicyRegistry", "SkillRegistry", + "SpeechRegistry", "ToolRegistry", ] diff --git a/src/openjarvis/evals/cli.py b/src/openjarvis/evals/cli.py index 1318bc04..7395bfd1 100644 --- a/src/openjarvis/evals/cli.py +++ b/src/openjarvis/evals/cli.py @@ -217,6 +217,39 @@ def _print_summary( print_completion(console, summary, output_path, traces_dir) +def _build_trackers(config) -> list: + """Build tracker instances from RunConfig fields.""" + trackers = [] + if getattr(config, "wandb_project", ""): + try: + from openjarvis.evals.trackers.wandb_tracker import WandbTracker + trackers.append(WandbTracker( + project=config.wandb_project, + entity=getattr(config, "wandb_entity", ""), + tags=getattr(config, "wandb_tags", ""), + group=getattr(config, "wandb_group", ""), + )) + except ImportError as exc: + raise click.ClickException( + f"wandb not installed: {exc}\n" + "Install with: pip install 'openjarvis[eval-wandb]'" + ) from exc + if getattr(config, "sheets_spreadsheet_id", ""): + try: + from openjarvis.evals.trackers.sheets_tracker import SheetsTracker + trackers.append(SheetsTracker( + spreadsheet_id=config.sheets_spreadsheet_id, + worksheet=getattr(config, "sheets_worksheet", "Results"), + credentials_path=getattr(config, "sheets_credentials_path", ""), + )) + except ImportError as exc: + raise click.ClickException( + f"gspread not installed: {exc}\n" + "Install with: pip install 'openjarvis[eval-sheets]'" + ) from exc + return trackers + + def _run_single(config, console: Optional[Console] = None) -> object: """Run a single eval from a RunConfig and return the summary.""" from openjarvis.evals.core.runner import EvalRunner @@ -236,7 +269,8 @@ def _run_single(config, console: Optional[Console] = None) -> object: judge_backend = _build_judge_backend(config.judge_model) scorer = _build_scorer(config.benchmark, judge_backend, config.judge_model) - runner = EvalRunner(config, dataset, eval_backend, scorer) + trackers = _build_trackers(config) + runner = EvalRunner(config, dataset, eval_backend, scorer, trackers=trackers) try: num_samples = config.max_samples or 0 # Use progress bar if we know the sample count @@ -292,6 +326,11 @@ def _run_from_config(config_path: str, verbose: bool) -> None: output_dir = Path(suite.run.output_dir) output_dir.mkdir(parents=True, exist_ok=True) + # Auto-set wandb_group to suite name if W&B enabled and no explicit group + for rc in run_configs: + if rc.wandb_project and not rc.wandb_group: + rc.wandb_group = suite_name + summaries = [] for i, rc in enumerate(run_configs, 1): print_section( @@ -355,12 +394,30 @@ def main(): help="Enable GPU metrics collection") @click.option("--compact", is_flag=True, default=False, help="Dense single-table output") @click.option("--trace-detail", is_flag=True, default=False, help="Full per-step trace listing") +@click.option("--wandb-project", default="", + help="W&B project name (enables tracking)") +@click.option("--wandb-entity", default="", + help="W&B entity (team or user)") +@click.option("--wandb-tags", default="", + help="Comma-separated W&B tags") +@click.option("--wandb-group", default="", + help="W&B run group") +@click.option("--sheets-id", "sheets_spreadsheet_id", default="", + help="Google Sheets spreadsheet ID") +@click.option("--sheets-worksheet", default="Results", + help="Google Sheets worksheet name") +@click.option("--sheets-creds", "sheets_credentials_path", + default="", + help="Service account JSON path") @click.option("-v", "--verbose", is_flag=True, help="Verbose logging") @click.pass_context def run(ctx, config_path, benchmark, backend, model, engine_key, agent_name, tools, max_samples, max_workers, judge_model, output_path, seed, dataset_split, temperature, max_tokens, telemetry, gpu_metrics, - compact, trace_detail, verbose): + compact, trace_detail, + wandb_project, wandb_entity, wandb_tags, wandb_group, + sheets_spreadsheet_id, sheets_worksheet, sheets_credentials_path, + verbose): """Run a single benchmark evaluation, or a full suite from a TOML config.""" _setup_logging(verbose) @@ -404,6 +461,13 @@ def run(ctx, config_path, benchmark, backend, model, engine_key, agent_name, dataset_split=dataset_split, telemetry=telemetry, gpu_metrics=gpu_metrics, + wandb_project=wandb_project, + wandb_entity=wandb_entity, + wandb_tags=wandb_tags, + wandb_group=wandb_group, + sheets_spreadsheet_id=sheets_spreadsheet_id, + sheets_worksheet=sheets_worksheet, + sheets_credentials_path=sheets_credentials_path, ) # Banner + config @@ -493,7 +557,8 @@ def run_all(model, engine_key, max_samples, max_workers, judge_model, judge_backend = _build_judge_backend(judge_model) scorer = _build_scorer(bench_name, judge_backend, judge_model) - runner = EvalRunner(config, dataset, eval_backend, scorer) + trackers = _build_trackers(config) + runner = EvalRunner(config, dataset, eval_backend, scorer, trackers=trackers) try: if max_samples and max_samples > 0: with Progress( diff --git a/src/openjarvis/evals/core/config.py b/src/openjarvis/evals/core/config.py index 966d4fdc..af09a9de 100644 --- a/src/openjarvis/evals/core/config.py +++ b/src/openjarvis/evals/core/config.py @@ -99,6 +99,13 @@ def load_eval_config(path: str | Path) -> EvalSuiteConfig: gpu_metrics=bool(run_raw.get("gpu_metrics", False)), warmup_samples=int(run_raw.get("warmup_samples", 0)), energy_vendor=run_raw.get("energy_vendor", ""), + wandb_project=run_raw.get("wandb_project", ""), + wandb_entity=run_raw.get("wandb_entity", ""), + wandb_tags=run_raw.get("wandb_tags", ""), + wandb_group=run_raw.get("wandb_group", ""), + sheets_spreadsheet_id=run_raw.get("sheets_spreadsheet_id", ""), + sheets_worksheet=run_raw.get("sheets_worksheet", "Results"), + sheets_credentials_path=run_raw.get("sheets_credentials_path", ""), ) # Parse [[models]] @@ -243,6 +250,13 @@ def expand_suite(suite: EvalSuiteConfig) -> List[RunConfig]: gpu_metrics=suite.run.gpu_metrics, metadata=model_meta, warmup_samples=suite.run.warmup_samples, + wandb_project=suite.run.wandb_project, + wandb_entity=suite.run.wandb_entity, + wandb_tags=suite.run.wandb_tags, + wandb_group=suite.run.wandb_group, + sheets_spreadsheet_id=suite.run.sheets_spreadsheet_id, + sheets_worksheet=suite.run.sheets_worksheet, + sheets_credentials_path=suite.run.sheets_credentials_path, )) return configs diff --git a/src/openjarvis/evals/core/runner.py b/src/openjarvis/evals/core/runner.py index 07a9a0eb..66ec57f1 100644 --- a/src/openjarvis/evals/core/runner.py +++ b/src/openjarvis/evals/core/runner.py @@ -14,7 +14,14 @@ from typing import Any, Callable, Dict, List, Optional from openjarvis.evals.core.backend import InferenceBackend from openjarvis.evals.core.dataset import DatasetProvider from openjarvis.evals.core.scorer import Scorer -from openjarvis.evals.core.types import EvalRecord, EvalResult, MetricStats, RunConfig, RunSummary +from openjarvis.evals.core.tracker import ResultTracker +from openjarvis.evals.core.types import ( + EvalRecord, + EvalResult, + MetricStats, + RunConfig, + RunSummary, +) try: from openjarvis.telemetry.efficiency import compute_efficiency @@ -33,11 +40,13 @@ class EvalRunner: dataset: DatasetProvider, backend: InferenceBackend, scorer: Scorer, + trackers: Optional[List[ResultTracker]] = None, ) -> None: self._config = config self._dataset = dataset self._backend = backend self._scorer = scorer + self._trackers: List[ResultTracker] = trackers or [] self._results: List[EvalResult] = [] self._output_file: Optional[Any] = None @@ -79,6 +88,16 @@ class EvalRunner: output_path.parent.mkdir(parents=True, exist_ok=True) self._output_file = open(output_path, "w") + # Notify trackers of run start + for tracker in self._trackers: + try: + tracker.on_run_start(cfg) + except Exception as exc: + LOGGER.warning( + "Tracker %s.on_run_start failed: %s", + type(tracker).__name__, exc, + ) + total = len(records) try: with ThreadPoolExecutor(max_workers=cfg.max_workers) as pool: @@ -99,6 +118,23 @@ class EvalRunner: ended_at = time.time() summary = self._compute_summary(records, started_at, ended_at) + # Notify trackers of summary and run end + for tracker in self._trackers: + try: + tracker.on_summary(summary) + except Exception as exc: + LOGGER.warning( + "Tracker %s.on_summary failed: %s", + type(tracker).__name__, exc, + ) + try: + tracker.on_run_end() + except Exception as exc: + LOGGER.warning( + "Tracker %s.on_run_end failed: %s", + type(tracker).__name__, exc, + ) + # Write summary JSON alongside JSONL traces_dir: Optional[Path] = None if output_path: @@ -255,6 +291,16 @@ class EvalRunner: self._output_file.write(json.dumps(record_dict) + "\n") self._output_file.flush() + # Notify trackers of each result + for tracker in self._trackers: + try: + tracker.on_result(result, self._config) + except Exception as exc: + LOGGER.warning( + "Tracker %s.on_result failed: %s", + type(tracker).__name__, exc, + ) + def _resolve_output_path(self) -> Optional[Path]: """Determine the output file path.""" if self._config.output_path: diff --git a/src/openjarvis/evals/core/tracker.py b/src/openjarvis/evals/core/tracker.py new file mode 100644 index 00000000..83b62deb --- /dev/null +++ b/src/openjarvis/evals/core/tracker.py @@ -0,0 +1,34 @@ +"""ResultTracker ABC for external experiment tracking.""" + +from __future__ import annotations + +from abc import ABC, abstractmethod + +from openjarvis.evals.core.types import EvalResult, RunConfig, RunSummary + + +class ResultTracker(ABC): + """Abstract base class for experiment result trackers. + + Lifecycle: on_run_start -> on_result (per sample) + -> on_summary -> on_run_end. + """ + + @abstractmethod + def on_run_start(self, config: RunConfig) -> None: + """Called once before evaluation begins.""" + + @abstractmethod + def on_result(self, result: EvalResult, config: RunConfig) -> None: + """Called after each sample is evaluated.""" + + @abstractmethod + def on_summary(self, summary: RunSummary) -> None: + """Called after all samples are evaluated with aggregate stats.""" + + @abstractmethod + def on_run_end(self) -> None: + """Called at the very end of a run for cleanup.""" + + +__all__ = ["ResultTracker"] diff --git a/src/openjarvis/evals/core/types.py b/src/openjarvis/evals/core/types.py index 22702dd0..6f4c10ff 100644 --- a/src/openjarvis/evals/core/types.py +++ b/src/openjarvis/evals/core/types.py @@ -71,6 +71,13 @@ class RunConfig: gpu_metrics: bool = False metadata: Dict[str, Any] = field(default_factory=dict) warmup_samples: int = 0 + wandb_project: str = "" + wandb_entity: str = "" + wandb_tags: str = "" + wandb_group: str = "" + sheets_spreadsheet_id: str = "" + sheets_worksheet: str = "Results" + sheets_credentials_path: str = "" @dataclass(slots=True) @@ -176,6 +183,13 @@ class ExecutionConfig: gpu_metrics: bool = False warmup_samples: int = 0 energy_vendor: str = "" + wandb_project: str = "" + wandb_entity: str = "" + wandb_tags: str = "" + wandb_group: str = "" + sheets_spreadsheet_id: str = "" + sheets_worksheet: str = "Results" + sheets_credentials_path: str = "" @dataclass(slots=True) diff --git a/src/openjarvis/evals/trackers/__init__.py b/src/openjarvis/evals/trackers/__init__.py new file mode 100644 index 00000000..61306ba6 --- /dev/null +++ b/src/openjarvis/evals/trackers/__init__.py @@ -0,0 +1,23 @@ +"""External experiment trackers for the eval framework. + +Trackers are lazily imported to avoid mandatory dependencies on wandb/gspread. +""" + +from __future__ import annotations + + +def WandbTracker(*args, **kwargs): # noqa: N802 + """Lazy constructor — imports the real class on first use.""" + from openjarvis.evals.trackers.wandb_tracker import WandbTracker as _Cls + + return _Cls(*args, **kwargs) + + +def SheetsTracker(*args, **kwargs): # noqa: N802 + """Lazy constructor — imports the real class on first use.""" + from openjarvis.evals.trackers.sheets_tracker import SheetsTracker as _Cls + + return _Cls(*args, **kwargs) + + +__all__ = ["WandbTracker", "SheetsTracker"] diff --git a/src/openjarvis/evals/trackers/sheets_tracker.py b/src/openjarvis/evals/trackers/sheets_tracker.py new file mode 100644 index 00000000..85c9e37b --- /dev/null +++ b/src/openjarvis/evals/trackers/sheets_tracker.py @@ -0,0 +1,162 @@ +"""Google Sheets experiment tracker for the eval framework.""" + +from __future__ import annotations + +import logging +import time +from typing import Any, List, Optional + +from openjarvis.evals.core.tracker import ResultTracker +from openjarvis.evals.core.types import EvalResult, MetricStats, RunConfig, RunSummary + +try: + import gspread + from google.oauth2.service_account import Credentials +except ImportError: + gspread = None # type: ignore[assignment] + Credentials = None # type: ignore[assignment,misc] + +LOGGER = logging.getLogger(__name__) + +# Canonical column order for the summary row. +SHEET_COLUMNS: List[str] = [ + "timestamp", + "benchmark", + "model", + "backend", + "total_samples", + "scored_samples", + "correct", + "accuracy", + "errors", + "mean_latency_seconds", + "total_cost_usd", + "total_energy_joules", + "avg_power_watts", + "total_input_tokens", + "total_output_tokens", + "latency_mean", + "latency_p90", + "latency_p95", + "energy_mean", + "energy_p90", + "throughput_mean", + "throughput_p90", + "ipw_mean", + "ipj_mean", + "mfu_mean", + "mbu_mean", + "ttft_mean", + "ttft_p90", + "gpu_utilization_mean", +] + + +def _stat_val(ms: Optional[MetricStats], attr: str) -> Any: + """Safely extract a stat value from a MetricStats, returning '' if None.""" + if ms is None: + return "" + return getattr(ms, attr, "") + + +class SheetsTracker(ResultTracker): + """Appends a summary row to a Google Sheet after each eval run.""" + + def __init__( + self, + spreadsheet_id: str, + worksheet: str = "Results", + credentials_path: str = "", + ) -> None: + if gspread is None: + raise ImportError( + "gspread is not installed. " + "Install it with: pip install 'openjarvis[eval-sheets]'" + ) + self._spreadsheet_id = spreadsheet_id + self._worksheet_name = worksheet + self._credentials_path = credentials_path + + def on_run_start(self, config: RunConfig) -> None: + pass + + def on_result(self, result: EvalResult, config: RunConfig) -> None: + # No-op: summary-only to avoid excessive API calls. + pass + + def on_summary(self, summary: RunSummary) -> None: + row = self._build_row(summary) + try: + gc = self._authorize() + spreadsheet = gc.open_by_key(self._spreadsheet_id) + try: + ws = spreadsheet.worksheet(self._worksheet_name) + except gspread.exceptions.WorksheetNotFound: + ws = spreadsheet.add_worksheet( + title=self._worksheet_name, rows=1000, cols=len(SHEET_COLUMNS), + ) + # Ensure header row exists (idempotent) + existing = ws.row_values(1) + if not existing or existing[0] != SHEET_COLUMNS[0]: + ws.update(range_name="A1", values=[SHEET_COLUMNS]) + ws.append_row(row, value_input_option="RAW") + LOGGER.info("Appended summary row to Google Sheet") + except Exception as exc: + LOGGER.warning("SheetsTracker.on_summary failed: %s", exc) + + def on_run_end(self) -> None: + pass + + def _authorize(self): + """Authenticate with Google Sheets API.""" + scopes = [ + "https://www.googleapis.com/auth/spreadsheets", + "https://www.googleapis.com/auth/drive", + ] + if self._credentials_path: + creds = Credentials.from_service_account_file( + self._credentials_path, scopes=scopes, + ) + else: + # Fall back to Application Default Credentials + import google.auth + + creds, _ = google.auth.default(scopes=scopes) + return gspread.authorize(creds) + + def _build_row(self, s: RunSummary) -> List[Any]: + """Build a flat row matching SHEET_COLUMNS order.""" + return [ + time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()), + s.benchmark, + s.model, + s.backend, + s.total_samples, + s.scored_samples, + s.correct, + s.accuracy, + s.errors, + s.mean_latency_seconds, + s.total_cost_usd, + s.total_energy_joules, + s.avg_power_watts, + s.total_input_tokens, + s.total_output_tokens, + _stat_val(s.latency_stats, "mean"), + _stat_val(s.latency_stats, "p90"), + _stat_val(s.latency_stats, "p95"), + _stat_val(s.energy_stats, "mean"), + _stat_val(s.energy_stats, "p90"), + _stat_val(s.throughput_stats, "mean"), + _stat_val(s.throughput_stats, "p90"), + _stat_val(s.ipw_stats, "mean"), + _stat_val(s.ipj_stats, "mean"), + _stat_val(s.mfu_stats, "mean"), + _stat_val(s.mbu_stats, "mean"), + _stat_val(s.ttft_stats, "mean"), + _stat_val(s.ttft_stats, "p90"), + _stat_val(s.gpu_utilization_stats, "mean"), + ] + + +__all__ = ["SheetsTracker", "SHEET_COLUMNS"] diff --git a/src/openjarvis/evals/trackers/wandb_tracker.py b/src/openjarvis/evals/trackers/wandb_tracker.py new file mode 100644 index 00000000..d613f03d --- /dev/null +++ b/src/openjarvis/evals/trackers/wandb_tracker.py @@ -0,0 +1,154 @@ +"""W&B experiment tracker for the eval framework.""" + +from __future__ import annotations + +import logging +from typing import Any, Dict, List, Optional + +from openjarvis.evals.core.tracker import ResultTracker +from openjarvis.evals.core.types import EvalResult, MetricStats, RunConfig, RunSummary + +try: + import wandb +except ImportError: + wandb = None # type: ignore[assignment] + +LOGGER = logging.getLogger(__name__) + + +def _flatten_metric_stats(prefix: str, ms: Optional[MetricStats]) -> Dict[str, float]: + """Flatten a MetricStats into a dict with prefixed keys.""" + if ms is None: + return {} + return { + f"{prefix}_mean": ms.mean, + f"{prefix}_median": ms.median, + f"{prefix}_min": ms.min, + f"{prefix}_max": ms.max, + f"{prefix}_std": ms.std, + f"{prefix}_p90": ms.p90, + f"{prefix}_p95": ms.p95, + f"{prefix}_p99": ms.p99, + } + + +class WandbTracker(ResultTracker): + """Streams per-sample metrics to Weights & Biases.""" + + def __init__( + self, + project: str, + entity: str = "", + tags: str = "", + group: str = "", + ) -> None: + if wandb is None: + raise ImportError( + "wandb is not installed. " + "Install it with: pip install 'openjarvis[eval-wandb]'" + ) + self._project = project + self._entity = entity or None + self._tags: List[str] = [ + t.strip() for t in tags.split(",") if t.strip() + ] if tags else [] + self._group = group or None + self._run: Any = None + self._step = 0 + + def on_run_start(self, config: RunConfig) -> None: + run_config = { + "benchmark": config.benchmark, + "model": config.model, + "backend": config.backend, + "max_samples": config.max_samples, + "max_workers": config.max_workers, + "temperature": config.temperature, + "max_tokens": config.max_tokens, + "seed": config.seed, + } + if config.agent_name: + run_config["agent_name"] = config.agent_name + if config.tools: + run_config["tools"] = ",".join(config.tools) + if config.engine_key: + run_config["engine_key"] = config.engine_key + + self._run = wandb.init( + project=self._project, + entity=self._entity, + tags=self._tags or None, + group=self._group, + config=run_config, + reinit=True, + ) + self._step = 0 + + def on_result(self, result: EvalResult, config: RunConfig) -> None: + if self._run is None: + return + self._step += 1 + log_data: Dict[str, Any] = { + "sample/is_correct": 1.0 if result.is_correct else 0.0, + "sample/latency_seconds": result.latency_seconds, + "sample/prompt_tokens": result.prompt_tokens, + "sample/completion_tokens": result.completion_tokens, + "sample/cost_usd": result.cost_usd, + "sample/ttft": result.ttft, + "sample/energy_joules": result.energy_joules, + "sample/power_watts": result.power_watts, + "sample/throughput_tok_per_sec": result.throughput_tok_per_sec, + "sample/ipw": result.ipw, + "sample/ipj": result.ipj, + } + if result.error: + log_data["sample/has_error"] = 1.0 + wandb.log(log_data, step=self._step) + + def on_summary(self, summary: RunSummary) -> None: + if self._run is None: + return + flat: Dict[str, Any] = { + "accuracy": summary.accuracy, + "total_samples": summary.total_samples, + "scored_samples": summary.scored_samples, + "correct": summary.correct, + "errors": summary.errors, + "mean_latency_seconds": summary.mean_latency_seconds, + "total_cost_usd": summary.total_cost_usd, + "total_energy_joules": summary.total_energy_joules, + "avg_power_watts": summary.avg_power_watts, + "total_input_tokens": summary.total_input_tokens, + "total_output_tokens": summary.total_output_tokens, + } + flat.update(_flatten_metric_stats("accuracy", summary.accuracy_stats)) + flat.update(_flatten_metric_stats("latency", summary.latency_stats)) + flat.update(_flatten_metric_stats("ttft", summary.ttft_stats)) + flat.update(_flatten_metric_stats("energy", summary.energy_stats)) + flat.update(_flatten_metric_stats("power", summary.power_stats)) + flat.update( + _flatten_metric_stats("gpu_utilization", summary.gpu_utilization_stats) + ) + flat.update(_flatten_metric_stats("throughput", summary.throughput_stats)) + flat.update(_flatten_metric_stats("mfu", summary.mfu_stats)) + flat.update(_flatten_metric_stats("mbu", summary.mbu_stats)) + flat.update(_flatten_metric_stats("ipw", summary.ipw_stats)) + flat.update(_flatten_metric_stats("ipj", summary.ipj_stats)) + flat.update(_flatten_metric_stats( + "energy_per_output_token", + summary.energy_per_output_token_stats, + )) + flat.update(_flatten_metric_stats( + "throughput_per_watt", + summary.throughput_per_watt_stats, + )) + flat.update(_flatten_metric_stats("itl", summary.itl_stats)) + wandb.run.summary.update(flat) + + def on_run_end(self) -> None: + if self._run is not None: + self._run.finish() + self._run = None + + +__all__ = ["WandbTracker"] diff --git a/src/openjarvis/sdk.py b/src/openjarvis/sdk.py index dfb9e0fa..95d3f497 100644 --- a/src/openjarvis/sdk.py +++ b/src/openjarvis/sdk.py @@ -108,6 +108,12 @@ class MemoryHandle: self._backend.close() self._backend = None + def __enter__(self) -> MemoryHandle: + return self + + def __exit__(self, *exc: Any) -> None: + self.close() + class Jarvis: """High-level OpenJarvis SDK. @@ -116,9 +122,13 @@ class Jarvis: from openjarvis import Jarvis + with Jarvis() as j: + response = j.ask("Hello, what can you do?") + print(response) + + # Or without context manager: j = Jarvis() - response = j.ask("Hello, what can you do?") - print(response) + response = j.ask("Hello") j.close() """ @@ -479,5 +489,11 @@ class Jarvis: self._audit_logger = None self._engine = None + def __enter__(self) -> Jarvis: + return self + + def __exit__(self, *exc: Any) -> None: + self.close() + __all__ = ["Jarvis", "JarvisSystem", "MemoryHandle", "SystemBuilder"] diff --git a/src/openjarvis/server/api_routes.py b/src/openjarvis/server/api_routes.py index 0ae84ec4..7f73c10b 100644 --- a/src/openjarvis/server/api_routes.py +++ b/src/openjarvis/server/api_routes.py @@ -618,6 +618,51 @@ async def learning_policy(request: Request): return result +# ---- Speech routes ---- + +speech_router = APIRouter(prefix="/v1/speech", tags=["speech"]) + + +@speech_router.post("/transcribe") +async def transcribe_speech(request: Request): + """Transcribe uploaded audio to text.""" + backend = getattr(request.app.state, "speech_backend", None) + if backend is None: + raise HTTPException(status_code=501, detail="Speech backend not configured") + + form = await request.form() + audio_file = form.get("file") + if audio_file is None: + raise HTTPException(status_code=400, detail="Missing 'file' field") + + audio_bytes = await audio_file.read() + language = form.get("language") + + # Detect format from filename + filename = getattr(audio_file, "filename", "audio.wav") + ext = filename.rsplit(".", 1)[-1] if "." in filename else "wav" + + result = backend.transcribe(audio_bytes, format=ext, language=language or None) + return { + "text": result.text, + "language": result.language, + "confidence": result.confidence, + "duration_seconds": result.duration_seconds, + } + + +@speech_router.get("/health") +async def speech_health(request: Request): + """Check if a speech backend is available.""" + backend = getattr(request.app.state, "speech_backend", None) + if backend is None: + return {"available": False, "reason": "No speech backend configured"} + return { + "available": backend.health(), + "backend": backend.backend_id, + } + + def include_all_routes(app) -> None: """Include all extended API routers in a FastAPI app.""" app.include_router(agents_router) @@ -630,6 +675,7 @@ def include_all_routes(app) -> None: app.include_router(metrics_router) app.include_router(websocket_router) app.include_router(learning_router) + app.include_router(speech_router) __all__ = [ @@ -644,4 +690,5 @@ __all__ = [ "metrics_router", "websocket_router", "learning_router", + "speech_router", ] diff --git a/src/openjarvis/server/app.py b/src/openjarvis/server/app.py index 2f49ce46..806f06f6 100644 --- a/src/openjarvis/server/app.py +++ b/src/openjarvis/server/app.py @@ -52,6 +52,7 @@ def create_app( agent_name: str = "", channel_bridge=None, config=None, + speech_backend=None, ) -> FastAPI: """Create and configure the FastAPI application. @@ -116,6 +117,7 @@ def create_app( getattr(agent, "agent_id", None) if agent else None ) app.state.channel_bridge = channel_bridge + app.state.speech_backend = speech_backend app.state.session_start = time.time() app.include_router(router) diff --git a/src/openjarvis/speech/__init__.py b/src/openjarvis/speech/__init__.py new file mode 100644 index 00000000..7fa6ea36 --- /dev/null +++ b/src/openjarvis/speech/__init__.py @@ -0,0 +1,10 @@ +"""Speech subsystem — speech-to-text backends.""" + +import importlib + +# Optional backends — each registers itself via @SpeechRegistry.register() +for _mod in ("faster_whisper", "openai_whisper", "deepgram"): + try: + importlib.import_module(f".{_mod}", __name__) + except ImportError: + pass diff --git a/src/openjarvis/speech/_discovery.py b/src/openjarvis/speech/_discovery.py new file mode 100644 index 00000000..0319193b --- /dev/null +++ b/src/openjarvis/speech/_discovery.py @@ -0,0 +1,75 @@ +"""Auto-discover available speech-to-text backends.""" + +from __future__ import annotations + +import os +from typing import TYPE_CHECKING, Optional + +if TYPE_CHECKING: + from openjarvis.core.config import JarvisConfig + from openjarvis.speech._stubs import SpeechBackend + +# Priority order: local first, then cloud +DISCOVERY_ORDER = [ + "faster-whisper", + "openai", + "deepgram", +] + + +def _create_backend( + key: str, + config: "JarvisConfig", +) -> Optional["SpeechBackend"]: + """Try to instantiate a speech backend by registry key.""" + from openjarvis.core.registry import SpeechRegistry + + if not SpeechRegistry.contains(key): + return None + + try: + backend_cls = SpeechRegistry.get(key) + + if key == "faster-whisper": + return backend_cls( + model_size=config.speech.model, + device=config.speech.device, + compute_type=config.speech.compute_type, + ) + elif key == "openai": + api_key = os.environ.get("OPENAI_API_KEY", "") + if not api_key: + return None + return backend_cls(api_key=api_key) + elif key == "deepgram": + api_key = os.environ.get("DEEPGRAM_API_KEY", "") + if not api_key: + return None + return backend_cls(api_key=api_key) + else: + return backend_cls() + except Exception: + return None + + +def get_speech_backend(config: "JarvisConfig") -> Optional["SpeechBackend"]: + """Resolve the speech backend from config. + + If ``config.speech.backend`` is ``"auto"``, tries backends in + priority order and returns the first healthy one. + """ + # Trigger registration of built-in backends + import openjarvis.speech # noqa: F401 + + backend_key = config.speech.backend + + if backend_key != "auto": + return _create_backend(backend_key, config) + + # Auto-discovery: try each in priority order + for key in DISCOVERY_ORDER: + backend = _create_backend(key, config) + if backend is not None: + return backend + + return None diff --git a/src/openjarvis/speech/_stubs.py b/src/openjarvis/speech/_stubs.py new file mode 100644 index 00000000..239a7414 --- /dev/null +++ b/src/openjarvis/speech/_stubs.py @@ -0,0 +1,55 @@ +"""Abstract base classes and data types for the speech subsystem.""" + +from __future__ import annotations + +from abc import ABC, abstractmethod +from dataclasses import dataclass, field +from typing import List, Optional + + +@dataclass +class Segment: + """A timed segment of transcribed text.""" + + text: str + start: float # Start time in seconds + end: float # End time in seconds + confidence: Optional[float] = None + + +@dataclass +class TranscriptionResult: + """Result of a speech-to-text transcription.""" + + text: str + language: Optional[str] = None + confidence: Optional[float] = None + duration_seconds: float = 0.0 + segments: List[Segment] = field(default_factory=list) + + +class SpeechBackend(ABC): + """Abstract base class for speech-to-text backends.""" + + backend_id: str = "" + + @abstractmethod + def transcribe( + self, + audio: bytes, + *, + format: str = "wav", + language: Optional[str] = None, + ) -> TranscriptionResult: + """Transcribe audio bytes to text.""" + + @abstractmethod + def health(self) -> bool: + """Check if the backend is ready.""" + + @abstractmethod + def supported_formats(self) -> List[str]: + """Return list of supported audio formats.""" + + +__all__ = ["Segment", "SpeechBackend", "TranscriptionResult"] diff --git a/src/openjarvis/speech/deepgram.py b/src/openjarvis/speech/deepgram.py new file mode 100644 index 00000000..124e1cd8 --- /dev/null +++ b/src/openjarvis/speech/deepgram.py @@ -0,0 +1,96 @@ +"""Deepgram speech-to-text backend (cloud).""" + +from __future__ import annotations + +import os +from typing import List, Optional + +from openjarvis.core.registry import SpeechRegistry +from openjarvis.speech._stubs import SpeechBackend, TranscriptionResult + +try: + from deepgram import DeepgramClient, PrerecordedOptions +except ImportError: + DeepgramClient = None # type: ignore[assignment, misc] + PrerecordedOptions = None # type: ignore[assignment, misc] + + +@SpeechRegistry.register("deepgram") +class DeepgramSpeechBackend(SpeechBackend): + """Cloud speech-to-text using Deepgram API.""" + + backend_id = "deepgram" + + def __init__(self, api_key: Optional[str] = None) -> None: + self._api_key = api_key or os.environ.get("DEEPGRAM_API_KEY", "") + self._client = None + if self._api_key and DeepgramClient is not None: + self._client = DeepgramClient(self._api_key) + + def transcribe( + self, + audio: bytes, + *, + format: str = "wav", + language: Optional[str] = None, + ) -> TranscriptionResult: + """Transcribe audio using Deepgram's API.""" + if self._client is None: + raise RuntimeError("Deepgram client not initialized (missing API key?)") + + mime_map = { + "wav": "audio/wav", + "mp3": "audio/mpeg", + "ogg": "audio/ogg", + "flac": "audio/flac", + "webm": "audio/webm", + "m4a": "audio/mp4", + } + mime_type = mime_map.get(format, "audio/wav") + + options_kwargs: dict = {"model": "nova-2", "smart_format": True} + if language: + options_kwargs["language"] = language + else: + options_kwargs["detect_language"] = True + + payload = {"buffer": audio, "mimetype": mime_type} + + if PrerecordedOptions is not None: + options = PrerecordedOptions(**options_kwargs) + else: + options = options_kwargs + + response = self._client.listen.rest.v("1").transcribe_file( + payload, options, + ) + + # Extract transcript from response + channels = response.results.channels + if channels and channels[0].alternatives: + alt = channels[0].alternatives[0] + text = alt.transcript + confidence = getattr(alt, "confidence", None) + else: + text = "" + confidence = None + + detected_lang = None + if channels: + detected_lang = getattr(channels[0], "detected_language", None) + + duration = getattr(response.metadata, "duration", 0.0) + + return TranscriptionResult( + text=text, + language=detected_lang, + confidence=confidence, + duration_seconds=duration, + segments=[], + ) + + def health(self) -> bool: + return self._client is not None and bool(self._api_key) + + def supported_formats(self) -> List[str]: + return ["wav", "mp3", "ogg", "flac", "webm", "m4a"] diff --git a/src/openjarvis/speech/faster_whisper.py b/src/openjarvis/speech/faster_whisper.py new file mode 100644 index 00000000..e3d02843 --- /dev/null +++ b/src/openjarvis/speech/faster_whisper.py @@ -0,0 +1,100 @@ +"""Faster-Whisper speech-to-text backend (local, CTranslate2-based).""" + +from __future__ import annotations + +import tempfile +from typing import List, Optional + +from openjarvis.core.registry import SpeechRegistry +from openjarvis.speech._stubs import Segment, SpeechBackend, TranscriptionResult + +try: + from faster_whisper import WhisperModel +except ImportError: + WhisperModel = None # type: ignore[assignment, misc] + + +@SpeechRegistry.register("faster-whisper") +class FasterWhisperBackend(SpeechBackend): + """Local speech-to-text using Faster-Whisper (CTranslate2).""" + + backend_id = "faster-whisper" + + def __init__( + self, + model_size: str = "base", + device: str = "auto", + compute_type: str = "float16", + ) -> None: + self._model_size = model_size + self._device = device + self._compute_type = compute_type + self._model: Optional[WhisperModel] = None + + def _ensure_model(self) -> WhisperModel: + """Lazy-load the Whisper model on first use.""" + if self._model is None: + if WhisperModel is None: + raise ImportError( + "faster-whisper is not installed. " + "Install with: pip install 'openjarvis[speech]'" + ) + self._model = WhisperModel( + self._model_size, + device=self._device, + compute_type=self._compute_type, + ) + return self._model + + def transcribe( + self, + audio: bytes, + *, + format: str = "wav", + language: Optional[str] = None, + ) -> TranscriptionResult: + """Transcribe audio bytes using Faster-Whisper.""" + model = self._ensure_model() + + # Write audio to a temp file (faster-whisper needs a file path) + suffix = f".{format}" if not format.startswith(".") else format + with tempfile.NamedTemporaryFile(suffix=suffix, delete=True) as tmp: + tmp.write(audio) + tmp.flush() + + kwargs = {} + if language: + kwargs["language"] = language + + segments_iter, info = model.transcribe(tmp.name, **kwargs) + segments_list = list(segments_iter) + + # Build result + text = "".join(seg.text for seg in segments_list).strip() + segments = [ + Segment( + text=seg.text.strip(), + start=seg.start, + end=seg.end, + confidence=None, + ) + for seg in segments_list + ] + + return TranscriptionResult( + text=text, + language=getattr(info, "language", None), + confidence=getattr(info, "language_probability", None), + duration_seconds=getattr(info, "duration", 0.0), + segments=segments, + ) + + def health(self) -> bool: + """Check if model is loaded or loadable.""" + if self._model is not None: + return True + return WhisperModel is not None + + def supported_formats(self) -> List[str]: + """Supported audio formats (same as ffmpeg/Whisper).""" + return ["wav", "mp3", "m4a", "ogg", "flac", "webm"] diff --git a/src/openjarvis/speech/openai_whisper.py b/src/openjarvis/speech/openai_whisper.py new file mode 100644 index 00000000..32a2e315 --- /dev/null +++ b/src/openjarvis/speech/openai_whisper.py @@ -0,0 +1,64 @@ +"""OpenAI Whisper API speech-to-text backend (cloud).""" + +from __future__ import annotations + +import io +import os +from typing import List, Optional + +from openjarvis.core.registry import SpeechRegistry +from openjarvis.speech._stubs import SpeechBackend, TranscriptionResult + +try: + from openai import OpenAI +except ImportError: + OpenAI = None # type: ignore[assignment, misc] + + +@SpeechRegistry.register("openai") +class OpenAIWhisperBackend(SpeechBackend): + """Cloud speech-to-text using OpenAI Whisper API.""" + + backend_id = "openai" + + def __init__(self, api_key: Optional[str] = None) -> None: + self._api_key = api_key or os.environ.get("OPENAI_API_KEY", "") + self._client: Optional[OpenAI] = None + if self._api_key and OpenAI is not None: + self._client = OpenAI(api_key=self._api_key) + + def transcribe( + self, + audio: bytes, + *, + format: str = "wav", + language: Optional[str] = None, + ) -> TranscriptionResult: + """Transcribe audio using OpenAI's Whisper API.""" + if self._client is None: + raise RuntimeError("OpenAI client not initialized (missing API key?)") + + ext = format if not format.startswith(".") else format[1:] + audio_file = io.BytesIO(audio) + audio_file.name = f"audio.{ext}" + + kwargs: dict = {"model": "whisper-1", "file": audio_file} + if language: + kwargs["language"] = language + kwargs["response_format"] = "verbose_json" + + response = self._client.audio.transcriptions.create(**kwargs) + + return TranscriptionResult( + text=getattr(response, "text", str(response)), + language=getattr(response, "language", None), + confidence=None, + duration_seconds=getattr(response, "duration", 0.0), + segments=[], + ) + + def health(self) -> bool: + return self._client is not None and bool(self._api_key) + + def supported_formats(self) -> List[str]: + return ["mp3", "mp4", "mpeg", "mpga", "m4a", "wav", "webm"] diff --git a/src/openjarvis/system.py b/src/openjarvis/system.py index 3046ec7a..0fbc6917 100644 --- a/src/openjarvis/system.py +++ b/src/openjarvis/system.py @@ -40,6 +40,7 @@ class JarvisSystem: session_store: Optional[Any] = None # SessionStore capability_policy: Optional[Any] = None # CapabilityPolicy operator_manager: Optional[Any] = None # OperatorManager + speech_backend: Optional[Any] = None # SpeechBackend _learning_orchestrator: Optional[Any] = None # LearningOrchestrator def ask( @@ -274,6 +275,12 @@ class JarvisSystem: if self.trace_store and hasattr(self.trace_store, "close"): self.trace_store.close() + def __enter__(self) -> JarvisSystem: + return self + + def __exit__(self, *exc: Any) -> None: + self.close() + class SystemBuilder: """Config-driven fluent builder for JarvisSystem.""" @@ -304,6 +311,7 @@ class SystemBuilder: self._scheduler: Optional[bool] = None self._workflow: Optional[bool] = None self._sessions: Optional[bool] = None + self._speech: Optional[bool] = None def engine(self, key: str) -> SystemBuilder: self._engine_key = key @@ -345,6 +353,10 @@ class SystemBuilder: self._sessions = enabled return self + def speech(self, enabled: bool) -> SystemBuilder: + self._speech = enabled + return self + def event_bus(self, bus: EventBus) -> SystemBuilder: self._bus = bus return self @@ -447,6 +459,16 @@ class SystemBuilder: # Set up learning orchestrator (when training is enabled) learning_orchestrator = self._setup_learning_orchestrator(config) + # Set up speech backend + speech_backend = None + speech_enabled = self._speech if self._speech is not None else True + if speech_enabled: + try: + from openjarvis.speech._discovery import get_speech_backend + speech_backend = get_speech_backend(config) + except Exception: + pass + system = JarvisSystem( config=config, bus=bus, @@ -466,6 +488,7 @@ class SystemBuilder: workflow_engine=workflow_engine, session_store=session_store, capability_policy=capability_policy, + speech_backend=speech_backend, ) system._learning_orchestrator = learning_orchestrator return system diff --git a/tests/conftest.py b/tests/conftest.py index 1f6dd4b1..d8d04e24 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -17,6 +17,7 @@ from openjarvis.core.registry import ( MemoryRegistry, ModelRegistry, RouterPolicyRegistry, + SpeechRegistry, ToolRegistry, ) @@ -32,6 +33,7 @@ def _clean_registries() -> None: RouterPolicyRegistry.clear() BenchmarkRegistry.clear() ChannelRegistry.clear() + SpeechRegistry.clear() reset_event_bus() diff --git a/tests/evals/test_trackers.py b/tests/evals/test_trackers.py new file mode 100644 index 00000000..9481fade --- /dev/null +++ b/tests/evals/test_trackers.py @@ -0,0 +1,333 @@ +"""Tests for eval result trackers (W&B + Google Sheets).""" + +from __future__ import annotations + +import sys +from typing import List +from unittest.mock import MagicMock, patch + +import pytest + +from openjarvis.evals.core.tracker import ResultTracker +from openjarvis.evals.core.types import EvalResult, RunConfig, RunSummary + +# --------------------------------------------------------------------------- +# Test double +# --------------------------------------------------------------------------- + +class RecordingTracker(ResultTracker): + """Records all lifecycle calls for testing.""" + + def __init__(self) -> None: + self.calls: List[str] = [] + self.results: List[EvalResult] = [] + self.summary: RunSummary | None = None + + def on_run_start(self, config: RunConfig) -> None: + self.calls.append("on_run_start") + + def on_result(self, result: EvalResult, config: RunConfig) -> None: + self.calls.append("on_result") + self.results.append(result) + + def on_summary(self, summary: RunSummary) -> None: + self.calls.append("on_summary") + self.summary = summary + + def on_run_end(self) -> None: + self.calls.append("on_run_end") + + +class CrashingTracker(ResultTracker): + """Raises on every lifecycle call.""" + + def on_run_start(self, config: RunConfig) -> None: + raise RuntimeError("boom start") + + def on_result(self, result: EvalResult, config: RunConfig) -> None: + raise RuntimeError("boom result") + + def on_summary(self, summary: RunSummary) -> None: + raise RuntimeError("boom summary") + + def on_run_end(self) -> None: + raise RuntimeError("boom end") + + +# --------------------------------------------------------------------------- +# Helpers +# --------------------------------------------------------------------------- + +def _make_config(**overrides) -> RunConfig: + defaults = dict(benchmark="test", backend="jarvis-direct", model="test-model") + defaults.update(overrides) + return RunConfig(**defaults) + + +def _make_summary(**overrides) -> RunSummary: + defaults = dict( + benchmark="test", + category="chat", + backend="jarvis-direct", + model="test-model", + total_samples=10, + scored_samples=10, + correct=8, + accuracy=0.8, + errors=0, + mean_latency_seconds=1.0, + total_cost_usd=0.01, + ) + defaults.update(overrides) + return RunSummary(**defaults) + + +def _make_result(**overrides) -> EvalResult: + defaults = dict(record_id="r1", model_answer="answer", is_correct=True) + defaults.update(overrides) + return EvalResult(**defaults) + + +# --------------------------------------------------------------------------- +# RecordingTracker through EvalRunner lifecycle +# --------------------------------------------------------------------------- + +class TestRecordingTrackerIntegration: + """Test that trackers receive all lifecycle calls through EvalRunner.""" + + def test_tracker_lifecycle(self, tmp_path): + """RecordingTracker receives start, result, summary, end calls.""" + from openjarvis.evals.core.runner import EvalRunner + + # Minimal stubs + dataset = MagicMock() + record = MagicMock() + record.record_id = "r1" + record.problem = "What is 1+1?" + record.reference = "2" + record.category = "chat" + record.subject = "" + dataset.load = MagicMock() + dataset.iter_records = MagicMock(return_value=[record]) + + backend = MagicMock() + backend.generate_full = MagicMock(return_value={ + "content": "2", + "usage": {"prompt_tokens": 10, "completion_tokens": 5}, + "latency_seconds": 0.5, + }) + + scorer = MagicMock() + scorer.score = MagicMock(return_value=(True, {})) + + tracker = RecordingTracker() + config = _make_config(output_path=str(tmp_path / "out.jsonl")) + + runner = EvalRunner(config, dataset, backend, scorer, trackers=[tracker]) + runner.run() + + assert "on_run_start" in tracker.calls + assert "on_result" in tracker.calls + assert "on_summary" in tracker.calls + assert "on_run_end" in tracker.calls + # Order matters + assert tracker.calls.index("on_run_start") < tracker.calls.index("on_result") + assert tracker.calls.index("on_result") < tracker.calls.index("on_summary") + assert tracker.calls.index("on_summary") < tracker.calls.index("on_run_end") + assert len(tracker.results) == 1 + assert tracker.summary is not None + + def test_crashing_tracker_does_not_abort(self, tmp_path): + """A tracker that raises exceptions must not prevent JSONL output.""" + from openjarvis.evals.core.runner import EvalRunner + + dataset = MagicMock() + record = MagicMock() + record.record_id = "r1" + record.problem = "What?" + record.reference = "yes" + record.category = "chat" + record.subject = "" + dataset.load = MagicMock() + dataset.iter_records = MagicMock(return_value=[record]) + + backend = MagicMock() + backend.generate_full = MagicMock(return_value={ + "content": "yes", + "usage": {}, + "latency_seconds": 0.1, + }) + + scorer = MagicMock() + scorer.score = MagicMock(return_value=(True, {})) + + output = tmp_path / "out.jsonl" + config = _make_config(output_path=str(output)) + + crasher = CrashingTracker() + runner = EvalRunner(config, dataset, backend, scorer, trackers=[crasher]) + summary = runner.run() + + # Run completed, JSONL written despite crashing tracker + assert summary.total_samples == 1 + assert output.exists() + assert output.read_text().strip() != "" + + +# --------------------------------------------------------------------------- +# WandbTracker unit tests +# --------------------------------------------------------------------------- + +class TestWandbTracker: + """Unit tests for WandbTracker (mocked wandb module).""" + + def test_import_error_when_wandb_missing(self): + """WandbTracker raises ImportError when wandb is not installed.""" + with patch.dict(sys.modules, {"wandb": None}): + import openjarvis.evals.trackers.wandb_tracker as wt_mod + original = wt_mod.wandb + wt_mod.wandb = None + try: + with pytest.raises(ImportError, match="wandb is not installed"): + wt_mod.WandbTracker(project="test") + finally: + wt_mod.wandb = original + + def test_on_result_calls_wandb_log(self): + """on_result calls wandb.log with sample/ prefixed keys.""" + import openjarvis.evals.trackers.wandb_tracker as wt_mod + + mock_wandb = MagicMock() + mock_run = MagicMock() + mock_wandb.init = MagicMock(return_value=mock_run) + original = wt_mod.wandb + wt_mod.wandb = mock_wandb + try: + tracker = wt_mod.WandbTracker(project="test-proj") + config = _make_config() + tracker.on_run_start(config) + + result = _make_result(latency_seconds=0.5, energy_joules=1.0) + tracker.on_result(result, config) + + mock_wandb.log.assert_called_once() + call_args = mock_wandb.log.call_args + log_data = call_args[0][0] + assert "sample/is_correct" in log_data + assert "sample/latency_seconds" in log_data + assert log_data["sample/is_correct"] == 1.0 + assert call_args[1]["step"] == 1 + + tracker.on_run_end() + finally: + wt_mod.wandb = original + + def test_on_summary_updates_run_summary(self): + """on_summary calls wandb.run.summary.update with flat dict.""" + import openjarvis.evals.trackers.wandb_tracker as wt_mod + + mock_wandb = MagicMock() + mock_run = MagicMock() + mock_wandb.init = MagicMock(return_value=mock_run) + mock_wandb.run = mock_run + original = wt_mod.wandb + wt_mod.wandb = mock_wandb + try: + tracker = wt_mod.WandbTracker(project="test-proj") + config = _make_config() + tracker.on_run_start(config) + + summary = _make_summary() + tracker.on_summary(summary) + + mock_run.summary.update.assert_called_once() + flat = mock_run.summary.update.call_args[0][0] + assert flat["accuracy"] == 0.8 + assert flat["total_samples"] == 10 + + tracker.on_run_end() + finally: + wt_mod.wandb = original + + def test_reinit_true_for_suite_mode(self): + """wandb.init is called with reinit=True.""" + import openjarvis.evals.trackers.wandb_tracker as wt_mod + + mock_wandb = MagicMock() + mock_run = MagicMock() + mock_wandb.init = MagicMock(return_value=mock_run) + original = wt_mod.wandb + wt_mod.wandb = mock_wandb + try: + tracker = wt_mod.WandbTracker(project="test-proj", entity="team") + config = _make_config() + tracker.on_run_start(config) + + call_kwargs = mock_wandb.init.call_args[1] + assert call_kwargs["reinit"] is True + assert call_kwargs["project"] == "test-proj" + assert call_kwargs["entity"] == "team" + + tracker.on_run_end() + finally: + wt_mod.wandb = original + + +# --------------------------------------------------------------------------- +# SheetsTracker unit tests +# --------------------------------------------------------------------------- + +class TestSheetsTracker: + """Unit tests for SheetsTracker.""" + + def test_import_error_when_gspread_missing(self): + """SheetsTracker raises ImportError when gspread not installed.""" + import openjarvis.evals.trackers.sheets_tracker as st_mod + original = st_mod.gspread + st_mod.gspread = None + try: + with pytest.raises(ImportError, match="gspread is not installed"): + st_mod.SheetsTracker(spreadsheet_id="abc123") + finally: + st_mod.gspread = original + + def test_on_result_is_noop(self): + """on_result does nothing (no API calls for individual samples).""" + import openjarvis.evals.trackers.sheets_tracker as st_mod + + mock_gspread = MagicMock() + original = st_mod.gspread + st_mod.gspread = mock_gspread + original_creds = st_mod.Credentials + st_mod.Credentials = MagicMock() + try: + tracker = st_mod.SheetsTracker(spreadsheet_id="abc123") + result = _make_result() + config = _make_config() + + # on_result should not call any external API + tracker.on_result(result, config) + mock_gspread.authorize.assert_not_called() + finally: + st_mod.gspread = original + st_mod.Credentials = original_creds + + def test_build_row_matches_columns(self): + """_build_row returns a list matching SHEET_COLUMNS length.""" + import openjarvis.evals.trackers.sheets_tracker as st_mod + + mock_gspread = MagicMock() + original = st_mod.gspread + st_mod.gspread = mock_gspread + original_creds = st_mod.Credentials + st_mod.Credentials = MagicMock() + try: + tracker = st_mod.SheetsTracker(spreadsheet_id="abc123") + summary = _make_summary() + row = tracker._build_row(summary) + assert len(row) == len(st_mod.SHEET_COLUMNS), ( + f"Row length {len(row)} != columns length {len(st_mod.SHEET_COLUMNS)}" + ) + finally: + st_mod.gspread = original + st_mod.Credentials = original_creds diff --git a/tests/server/test_speech_routes.py b/tests/server/test_speech_routes.py new file mode 100644 index 00000000..cb6f452f --- /dev/null +++ b/tests/server/test_speech_routes.py @@ -0,0 +1,84 @@ +"""Tests for speech API endpoints.""" + +import pytest + +fastapi = pytest.importorskip("fastapi") + +from unittest.mock import MagicMock + +from fastapi.testclient import TestClient + +from openjarvis.speech._stubs import TranscriptionResult + + +@pytest.fixture +def mock_speech_backend(): + backend = MagicMock() + backend.backend_id = "mock" + backend.health.return_value = True + backend.transcribe.return_value = TranscriptionResult( + text="Hello world", + language="en", + confidence=0.95, + duration_seconds=1.5, + segments=[], + ) + return backend + + +@pytest.fixture +def app_with_speech(mock_speech_backend): + from fastapi import FastAPI + from openjarvis.server.api_routes import speech_router + + app = FastAPI() + app.state.speech_backend = mock_speech_backend + app.include_router(speech_router) + return app + + +@pytest.fixture +def client(app_with_speech): + return TestClient(app_with_speech) + + +def test_transcribe_endpoint(client, mock_speech_backend): + response = client.post( + "/v1/speech/transcribe", + files={"file": ("test.wav", b"fake audio data", "audio/wav")}, + ) + assert response.status_code == 200 + data = response.json() + assert data["text"] == "Hello world" + assert data["language"] == "en" + assert data["confidence"] == 0.95 + assert data["duration_seconds"] == 1.5 + + +def test_transcribe_no_file(client): + response = client.post("/v1/speech/transcribe") + assert response.status_code == 400 or response.status_code == 422 + + +def test_health_endpoint(client): + response = client.get("/v1/speech/health") + assert response.status_code == 200 + data = response.json() + assert data["available"] is True + assert data["backend"] == "mock" + + +def test_health_no_backend(): + from fastapi import FastAPI + from fastapi.testclient import TestClient + from openjarvis.server.api_routes import speech_router + + app = FastAPI() + app.state.speech_backend = None + app.include_router(speech_router) + client = TestClient(app) + + response = client.get("/v1/speech/health") + assert response.status_code == 200 + data = response.json() + assert data["available"] is False diff --git a/tests/speech/__init__.py b/tests/speech/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/tests/speech/test_config.py b/tests/speech/test_config.py new file mode 100644 index 00000000..804af199 --- /dev/null +++ b/tests/speech/test_config.py @@ -0,0 +1,19 @@ +"""Tests for speech configuration.""" + +from openjarvis.core.config import JarvisConfig, SpeechConfig + + +def test_speech_config_defaults(): + cfg = SpeechConfig() + assert cfg.backend == "auto" + assert cfg.model == "base" + assert cfg.language == "" + assert cfg.device == "auto" + assert cfg.compute_type == "float16" + + +def test_jarvis_config_has_speech(): + cfg = JarvisConfig() + assert hasattr(cfg, "speech") + assert isinstance(cfg.speech, SpeechConfig) + assert cfg.speech.backend == "auto" diff --git a/tests/speech/test_deepgram.py b/tests/speech/test_deepgram.py new file mode 100644 index 00000000..3dca37e4 --- /dev/null +++ b/tests/speech/test_deepgram.py @@ -0,0 +1,61 @@ +"""Tests for Deepgram speech backend.""" + +from unittest.mock import MagicMock, patch + +import pytest + +from openjarvis.core.registry import SpeechRegistry +from openjarvis.speech._stubs import TranscriptionResult +from openjarvis.speech.deepgram import DeepgramSpeechBackend + + +@pytest.fixture(autouse=True) +def _register_deepgram(): + """Re-register after any registry clear.""" + if not SpeechRegistry.contains("deepgram"): + SpeechRegistry.register_value("deepgram", DeepgramSpeechBackend) + + +def test_deepgram_registers(): + assert SpeechRegistry.contains("deepgram") + + +def test_deepgram_transcribe(): + mock_client = MagicMock() + mock_result = MagicMock() + mock_channel = MagicMock() + mock_alternative = MagicMock() + mock_alternative.transcript = "Hello from Deepgram" + mock_alternative.confidence = 0.92 + mock_channel.alternatives = [mock_alternative] + mock_channel.detected_language = "en" + mock_result.results.channels = [mock_channel] + mock_result.metadata.duration = 1.8 + mock_client.listen.rest.v.return_value.transcribe_file.return_value = mock_result + + with patch("openjarvis.speech.deepgram.DeepgramClient", return_value=mock_client): + from openjarvis.speech.deepgram import DeepgramSpeechBackend + + backend = DeepgramSpeechBackend(api_key="test-key") + result = backend.transcribe(b"fake audio", format="wav") + + assert isinstance(result, TranscriptionResult) + assert result.text == "Hello from Deepgram" + + +def test_deepgram_health(): + with patch("openjarvis.speech.deepgram.DeepgramClient"): + from openjarvis.speech.deepgram import DeepgramSpeechBackend + + backend = DeepgramSpeechBackend(api_key="test-key") + assert backend.health() is True + + +def test_deepgram_health_no_key(): + with patch("openjarvis.speech.deepgram.DeepgramClient"): + from openjarvis.speech.deepgram import DeepgramSpeechBackend + + backend = DeepgramSpeechBackend.__new__(DeepgramSpeechBackend) + backend._client = None + backend._api_key = "" + assert backend.health() is False diff --git a/tests/speech/test_discovery.py b/tests/speech/test_discovery.py new file mode 100644 index 00000000..523b73cc --- /dev/null +++ b/tests/speech/test_discovery.py @@ -0,0 +1,44 @@ +"""Tests for speech backend auto-discovery.""" + +from unittest.mock import patch + +from openjarvis.core.config import JarvisConfig + + +def test_get_speech_backend_explicit(): + """Explicit backend selection works.""" + from openjarvis.speech._discovery import get_speech_backend + + config = JarvisConfig() + config.speech.backend = "faster-whisper" + + with patch("openjarvis.speech._discovery._create_backend") as mock_create: + mock_backend = type("MockBackend", (), { + "backend_id": "faster-whisper", + "health": lambda self: True, + })() + mock_create.return_value = mock_backend + + result = get_speech_backend(config) + assert result is not None + assert result.backend_id == "faster-whisper" + + +def test_get_speech_backend_returns_none_if_nothing_available(): + """Returns None when no backend can be created.""" + from openjarvis.speech._discovery import get_speech_backend + + config = JarvisConfig() + config.speech.backend = "nonexistent" + + result = get_speech_backend(config) + assert result is None + + +def test_auto_discovery_priority(): + """Auto mode tries backends in priority order.""" + from openjarvis.speech._discovery import DISCOVERY_ORDER + + assert DISCOVERY_ORDER[0] == "faster-whisper" + assert "openai" in DISCOVERY_ORDER + assert "deepgram" in DISCOVERY_ORDER diff --git a/tests/speech/test_faster_whisper.py b/tests/speech/test_faster_whisper.py new file mode 100644 index 00000000..4a463671 --- /dev/null +++ b/tests/speech/test_faster_whisper.py @@ -0,0 +1,78 @@ +"""Tests for Faster-Whisper speech backend.""" + +from unittest.mock import MagicMock, patch + +import pytest + +from openjarvis.core.registry import SpeechRegistry +from openjarvis.speech.faster_whisper import FasterWhisperBackend + + +@pytest.fixture(autouse=True) +def _register_faster_whisper(): + """Re-register after any registry clear.""" + if not SpeechRegistry.contains("faster-whisper"): + SpeechRegistry.register_value("faster-whisper", FasterWhisperBackend) + + +def test_faster_whisper_backend_registers(): + """Backend registers itself in SpeechRegistry.""" + assert SpeechRegistry.contains("faster-whisper") + + +def test_faster_whisper_transcribe(): + """Transcribe returns a TranscriptionResult.""" + from openjarvis.speech._stubs import TranscriptionResult + + mock_model = MagicMock() + mock_segment = MagicMock() + mock_segment.text = " Hello world" + mock_segment.start = 0.0 + mock_segment.end = 1.2 + mock_segment.avg_logprob = -0.3 + + mock_info = MagicMock() + mock_info.language = "en" + mock_info.language_probability = 0.95 + mock_info.duration = 1.5 + + mock_model.transcribe.return_value = ([mock_segment], mock_info) + + with patch( + "openjarvis.speech.faster_whisper.WhisperModel", + return_value=mock_model, + ): + from openjarvis.speech.faster_whisper import FasterWhisperBackend + + backend = FasterWhisperBackend(model_size="base", device="cpu") + result = backend.transcribe(b"fake audio bytes") + + assert isinstance(result, TranscriptionResult) + assert result.text == "Hello world" + assert result.language == "en" + assert result.duration_seconds == 1.5 + + +def test_faster_whisper_health_no_model(): + """Health returns False before model is loaded.""" + with patch( + "openjarvis.speech.faster_whisper.WhisperModel", + new=None, + ): + from openjarvis.speech.faster_whisper import FasterWhisperBackend + + backend = FasterWhisperBackend.__new__(FasterWhisperBackend) + backend._model = None + assert backend.health() is False + + +def test_faster_whisper_supported_formats(): + """Backend supports standard audio formats.""" + with patch("openjarvis.speech.faster_whisper.WhisperModel"): + from openjarvis.speech.faster_whisper import FasterWhisperBackend + + backend = FasterWhisperBackend.__new__(FasterWhisperBackend) + formats = backend.supported_formats() + assert "wav" in formats + assert "mp3" in formats + assert "webm" in formats diff --git a/tests/speech/test_openai_whisper.py b/tests/speech/test_openai_whisper.py new file mode 100644 index 00000000..c4a7a5bc --- /dev/null +++ b/tests/speech/test_openai_whisper.py @@ -0,0 +1,57 @@ +"""Tests for OpenAI Whisper API speech backend.""" + +from unittest.mock import MagicMock, patch + +import pytest + +from openjarvis.core.registry import SpeechRegistry +from openjarvis.speech._stubs import TranscriptionResult +from openjarvis.speech.openai_whisper import OpenAIWhisperBackend + + +@pytest.fixture(autouse=True) +def _register_openai_whisper(): + """Re-register after any registry clear.""" + if not SpeechRegistry.contains("openai"): + SpeechRegistry.register_value("openai", OpenAIWhisperBackend) + + +def test_openai_whisper_registers(): + assert SpeechRegistry.contains("openai") + + +def test_openai_whisper_transcribe(): + mock_client = MagicMock() + mock_response = MagicMock() + mock_response.text = "Hello from OpenAI" + mock_response.language = "en" + mock_response.duration = 2.0 + mock_client.audio.transcriptions.create.return_value = mock_response + + with patch("openjarvis.speech.openai_whisper.OpenAI", return_value=mock_client): + from openjarvis.speech.openai_whisper import OpenAIWhisperBackend + + backend = OpenAIWhisperBackend(api_key="test-key") + result = backend.transcribe(b"fake audio", format="wav") + + assert isinstance(result, TranscriptionResult) + assert result.text == "Hello from OpenAI" + assert result.language == "en" + + +def test_openai_whisper_health(): + with patch("openjarvis.speech.openai_whisper.OpenAI"): + from openjarvis.speech.openai_whisper import OpenAIWhisperBackend + + backend = OpenAIWhisperBackend(api_key="test-key") + assert backend.health() is True + + +def test_openai_whisper_health_no_key(): + with patch("openjarvis.speech.openai_whisper.OpenAI"): + from openjarvis.speech.openai_whisper import OpenAIWhisperBackend + + backend = OpenAIWhisperBackend.__new__(OpenAIWhisperBackend) + backend._client = None + backend._api_key = "" + assert backend.health() is False diff --git a/tests/speech/test_stubs.py b/tests/speech/test_stubs.py new file mode 100644 index 00000000..86af23fc --- /dev/null +++ b/tests/speech/test_stubs.py @@ -0,0 +1,32 @@ +"""Tests for speech ABC and data types.""" + +from openjarvis.speech._stubs import Segment, SpeechBackend, TranscriptionResult + + +def test_transcription_result(): + result = TranscriptionResult( + text="Hello world", + language="en", + confidence=0.95, + duration_seconds=1.5, + segments=[], + ) + assert result.text == "Hello world" + assert result.language == "en" + assert result.confidence == 0.95 + assert result.duration_seconds == 1.5 + assert result.segments == [] + + +def test_segment(): + seg = Segment(text="Hello", start=0.0, end=0.5, confidence=0.98) + assert seg.text == "Hello" + assert seg.start == 0.0 + assert seg.end == 0.5 + + +def test_speech_backend_is_abstract(): + import pytest + + with pytest.raises(TypeError): + SpeechBackend() diff --git a/tests/speech/test_system_integration.py b/tests/speech/test_system_integration.py new file mode 100644 index 00000000..e0ef811a --- /dev/null +++ b/tests/speech/test_system_integration.py @@ -0,0 +1,8 @@ +"""Tests for speech integration in SystemBuilder/JarvisSystem.""" + +from openjarvis.system import JarvisSystem + + +def test_jarvis_system_has_speech_backend(): + """JarvisSystem has a speech_backend attribute.""" + assert "speech_backend" in JarvisSystem.__dataclass_fields__ diff --git a/uv.lock b/uv.lock index f5150c2a..230605aa 100644 --- a/uv.lock +++ b/uv.lock @@ -4,17 +4,22 @@ requires-python = ">=3.10" resolution-markers = [ "python_full_version >= '3.14' and sys_platform == 'win32'", "python_full_version >= '3.14' and sys_platform == 'emscripten'", - "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'", + "python_full_version >= '3.14' and sys_platform == 'linux'", + "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'", "python_full_version == '3.13.*' and sys_platform == 'win32'", "python_full_version == '3.12.*' and sys_platform == 'win32'", "python_full_version == '3.13.*' and sys_platform == 'emscripten'", "python_full_version == '3.12.*' and sys_platform == 'emscripten'", - "python_full_version == '3.13.*' and sys_platform != 'emscripten' and sys_platform != 'win32'", - "python_full_version == '3.12.*' and sys_platform != 'emscripten' and sys_platform != 'win32'", + "python_full_version == '3.13.*' and sys_platform == 'linux'", + "python_full_version == '3.13.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'", + "python_full_version == '3.12.*' and sys_platform == 'linux'", + "python_full_version == '3.12.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'", "python_full_version == '3.11.*' and sys_platform == 'win32'", "python_full_version == '3.11.*' and sys_platform == 'emscripten'", - "python_full_version == '3.11.*' and sys_platform != 'emscripten' and sys_platform != 'win32'", - "python_full_version < '3.11'", + "python_full_version == '3.11.*' and sys_platform == 'linux'", + "python_full_version == '3.11.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'", + "python_full_version < '3.11' and sys_platform == 'linux'", + "python_full_version < '3.11' and sys_platform != 'linux'", ] [[package]] @@ -338,6 +343,63 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/9b/73/f7084bf12755113cd535ae586782ff3a6e710bfbe6a0d13d1c2f81ffbbfa/authlib-1.6.8-py2.py3-none-any.whl", hash = "sha256:97286fd7a15e6cfefc32771c8ef9c54f0ed58028f1322de6a2a7c969c3817888", size = 244116, upload-time = "2026-02-14T04:02:15.579Z" }, ] +[[package]] +name = "av" +version = "16.1.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/78/cd/3a83ffbc3cc25b39721d174487fb0d51a76582f4a1703f98e46170ce83d4/av-16.1.0.tar.gz", hash = "sha256:a094b4fd87a3721dacf02794d3d2c82b8d712c85b9534437e82a8a978c175ffd", size = 4285203, upload-time = "2026-01-11T07:31:33.772Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/97/51/2217a9249409d2e88e16e3f16f7c0def9fd3e7ffc4238b2ec211f9935bdb/av-16.1.0-cp310-cp310-macosx_11_0_x86_64.whl", hash = "sha256:2395748b0c34fe3a150a1721e4f3d4487b939520991b13e7b36f8926b3b12295", size = 26942590, upload-time = "2026-01-09T20:17:58.588Z" }, + { url = "https://files.pythonhosted.org/packages/bf/cd/a7070f4febc76a327c38808e01e2ff6b94531fe0b321af54ea3915165338/av-16.1.0-cp310-cp310-macosx_14_0_arm64.whl", hash = "sha256:72d7ac832710a158eeb7a93242370aa024a7646516291c562ee7f14a7ea881fd", size = 21507910, upload-time = "2026-01-09T20:18:02.309Z" }, + { url = "https://files.pythonhosted.org/packages/ae/30/ec812418cd9b297f0238fe20eb0747d8a8b68d82c5f73c56fe519a274143/av-16.1.0-cp310-cp310-manylinux_2_28_aarch64.whl", hash = "sha256:6cbac833092e66b6b0ac4d81ab077970b8ca874951e9c3974d41d922aaa653ed", size = 38738309, upload-time = "2026-01-09T20:18:04.701Z" }, + { url = "https://files.pythonhosted.org/packages/3a/b8/6c5795bf1f05f45c5261f8bce6154e0e5e86b158a6676650ddd77c28805e/av-16.1.0-cp310-cp310-manylinux_2_28_x86_64.whl", hash = "sha256:eb990672d97c18f99c02f31c8d5750236f770ffe354b5a52c5f4d16c5e65f619", size = 40293006, upload-time = "2026-01-09T20:18:07.238Z" }, + { url = "https://files.pythonhosted.org/packages/a7/44/5e183bcb9333fc3372ee6e683be8b0c9b515a506894b2d32ff465430c074/av-16.1.0-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:05ad70933ac3b8ef896a820ea64b33b6cca91a5fac5259cb9ba7fa010435be15", size = 40123516, upload-time = "2026-01-09T20:18:09.955Z" }, + { url = "https://files.pythonhosted.org/packages/12/1d/b5346d582a3c3d958b4d26a2cc63ce607233582d956121eb20d2bbe55c2e/av-16.1.0-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:d831a1062a3c47520bf99de6ec682bd1d64a40dfa958e5457bb613c5270e7ce3", size = 41463289, upload-time = "2026-01-09T20:18:12.459Z" }, + { url = "https://files.pythonhosted.org/packages/fa/31/acc946c0545f72b8d0d74584cb2a0ade9b7dfe2190af3ef9aa52a2e3c0b1/av-16.1.0-cp310-cp310-win_amd64.whl", hash = "sha256:358ab910fef3c5a806c55176f2b27e5663b33c4d0a692dafeb049c6ed71f8aff", size = 31754959, upload-time = "2026-01-09T20:18:14.718Z" }, + { url = "https://files.pythonhosted.org/packages/48/d0/b71b65d1b36520dcb8291a2307d98b7fc12329a45614a303ff92ada4d723/av-16.1.0-cp311-cp311-macosx_11_0_x86_64.whl", hash = "sha256:e88ad64ee9d2b9c4c5d891f16c22ae78e725188b8926eb88187538d9dd0b232f", size = 26927747, upload-time = "2026-01-09T20:18:16.976Z" }, + { url = "https://files.pythonhosted.org/packages/2f/79/720a5a6ccdee06eafa211b945b0a450e3a0b8fc3d12922f0f3c454d870d2/av-16.1.0-cp311-cp311-macosx_14_0_arm64.whl", hash = "sha256:cb296073fa6935724de72593800ba86ae49ed48af03960a4aee34f8a611f442b", size = 21492232, upload-time = "2026-01-09T20:18:19.266Z" }, + { url = "https://files.pythonhosted.org/packages/8e/4f/a1ba8d922f2f6d1a3d52419463ef26dd6c4d43ee364164a71b424b5ae204/av-16.1.0-cp311-cp311-manylinux_2_28_aarch64.whl", hash = "sha256:720edd4d25aa73723c1532bb0597806d7b9af5ee34fc02358782c358cfe2f879", size = 39291737, upload-time = "2026-01-09T20:18:21.513Z" }, + { url = "https://files.pythonhosted.org/packages/1a/31/fc62b9fe8738d2693e18d99f040b219e26e8df894c10d065f27c6b4f07e3/av-16.1.0-cp311-cp311-manylinux_2_28_x86_64.whl", hash = "sha256:c7f2bc703d0df260a1fdf4de4253c7f5500ca9fc57772ea241b0cb241bcf972e", size = 40846822, upload-time = "2026-01-09T20:18:24.275Z" }, + { url = "https://files.pythonhosted.org/packages/53/10/ab446583dbce730000e8e6beec6ec3c2753e628c7f78f334a35cad0317f4/av-16.1.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:d69c393809babada7d54964d56099e4b30a3e1f8b5736ca5e27bd7be0e0f3c83", size = 40675604, upload-time = "2026-01-09T20:18:26.866Z" }, + { url = "https://files.pythonhosted.org/packages/31/d7/1003be685277005f6d63fd9e64904ee222fe1f7a0ea70af313468bb597db/av-16.1.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:441892be28582356d53f282873c5a951592daaf71642c7f20165e3ddcb0b4c63", size = 42015955, upload-time = "2026-01-09T20:18:29.461Z" }, + { url = "https://files.pythonhosted.org/packages/2f/4a/fa2a38ee9306bf4579f556f94ecbc757520652eb91294d2a99c7cf7623b9/av-16.1.0-cp311-cp311-win_amd64.whl", hash = "sha256:273a3e32de64819e4a1cd96341824299fe06f70c46f2288b5dc4173944f0fd62", size = 31750339, upload-time = "2026-01-09T20:18:32.249Z" }, + { url = "https://files.pythonhosted.org/packages/9c/84/2535f55edcd426cebec02eb37b811b1b0c163f26b8d3f53b059e2ec32665/av-16.1.0-cp312-cp312-macosx_11_0_x86_64.whl", hash = "sha256:640f57b93f927fba8689f6966c956737ee95388a91bd0b8c8b5e0481f73513d6", size = 26945785, upload-time = "2026-01-09T20:18:34.486Z" }, + { url = "https://files.pythonhosted.org/packages/b6/17/ffb940c9e490bf42e86db4db1ff426ee1559cd355a69609ec1efe4d3a9eb/av-16.1.0-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:ae3fb658eec00852ebd7412fdc141f17f3ddce8afee2d2e1cf366263ad2a3b35", size = 21481147, upload-time = "2026-01-09T20:18:36.716Z" }, + { url = "https://files.pythonhosted.org/packages/15/c1/e0d58003d2d83c3921887d5c8c9b8f5f7de9b58dc2194356a2656a45cfdc/av-16.1.0-cp312-cp312-manylinux_2_28_aarch64.whl", hash = "sha256:27ee558d9c02a142eebcbe55578a6d817fedfde42ff5676275504e16d07a7f86", size = 39517197, upload-time = "2026-01-11T09:57:31.937Z" }, + { url = "https://files.pythonhosted.org/packages/32/77/787797b43475d1b90626af76f80bfb0c12cfec5e11eafcfc4151b8c80218/av-16.1.0-cp312-cp312-manylinux_2_28_x86_64.whl", hash = "sha256:7ae547f6d5fa31763f73900d43901e8c5fa6367bb9a9840978d57b5a7ae14ed2", size = 41174337, upload-time = "2026-01-11T09:57:35.792Z" }, + { url = "https://files.pythonhosted.org/packages/8e/ac/d90df7f1e3b97fc5554cf45076df5045f1e0a6adf13899e10121229b826c/av-16.1.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:8cf065f9d438e1921dc31fc7aa045790b58aee71736897866420d80b5450f62a", size = 40817720, upload-time = "2026-01-11T09:57:39.039Z" }, + { url = "https://files.pythonhosted.org/packages/80/6f/13c3a35f9dbcebafd03fe0c4cbd075d71ac8968ec849a3cfce406c35a9d2/av-16.1.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:a345877a9d3cc0f08e2bc4ec163ee83176864b92587afb9d08dff50f37a9a829", size = 42267396, upload-time = "2026-01-11T09:57:42.115Z" }, + { url = "https://files.pythonhosted.org/packages/c8/b9/275df9607f7fb44317ccb1d4be74827185c0d410f52b6e2cd770fe209118/av-16.1.0-cp312-cp312-win_amd64.whl", hash = "sha256:f49243b1d27c91cd8c66fdba90a674e344eb8eb917264f36117bf2b6879118fd", size = 31752045, upload-time = "2026-01-11T09:57:45.106Z" }, + { url = "https://files.pythonhosted.org/packages/75/2a/63797a4dde34283dd8054219fcb29294ba1c25d68ba8c8c8a6ae53c62c45/av-16.1.0-cp313-cp313-macosx_11_0_x86_64.whl", hash = "sha256:ce2a1b3d8bf619f6c47a9f28cfa7518ff75ddd516c234a4ee351037b05e6a587", size = 26916715, upload-time = "2026-01-11T09:57:47.682Z" }, + { url = "https://files.pythonhosted.org/packages/d2/c4/0b49cf730d0ae8cda925402f18ae814aef351f5772d14da72dd87ff66448/av-16.1.0-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:408dbe6a2573ca58a855eb8cd854112b33ea598651902c36709f5f84c991ed8e", size = 21452167, upload-time = "2026-01-11T09:57:50.606Z" }, + { url = "https://files.pythonhosted.org/packages/51/23/408806503e8d5d840975aad5699b153aaa21eb6de41ade75248a79b7a37f/av-16.1.0-cp313-cp313-manylinux_2_28_aarch64.whl", hash = "sha256:57f657f86652a160a8a01887aaab82282f9e629abf94c780bbdbb01595d6f0f7", size = 39215659, upload-time = "2026-01-11T09:57:53.757Z" }, + { url = "https://files.pythonhosted.org/packages/c4/19/a8528d5bba592b3903f44c28dab9cc653c95fcf7393f382d2751a1d1523e/av-16.1.0-cp313-cp313-manylinux_2_28_x86_64.whl", hash = "sha256:adbad2b355c2ee4552cac59762809d791bda90586d134a33c6f13727fb86cb3a", size = 40874970, upload-time = "2026-01-11T09:57:56.802Z" }, + { url = "https://files.pythonhosted.org/packages/e8/24/2dbcdf0e929ad56b7df078e514e7bd4ca0d45cba798aff3c8caac097d2f7/av-16.1.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:f42e1a68ec2aebd21f7eb6895be69efa6aa27eec1670536876399725bbda4b99", size = 40530345, upload-time = "2026-01-11T09:58:00.421Z" }, + { url = "https://files.pythonhosted.org/packages/54/27/ae91b41207f34e99602d1c72ab6ffd9c51d7c67e3fbcd4e3a6c0e54f882c/av-16.1.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:58fe47aeaef0f100c40ec8a5de9abbd37f118d3ca03829a1009cf288e9aef67c", size = 41972163, upload-time = "2026-01-11T09:58:03.756Z" }, + { url = "https://files.pythonhosted.org/packages/fc/7a/22158fb923b2a9a00dfab0e96ef2e8a1763a94dd89e666a5858412383d46/av-16.1.0-cp313-cp313-win_amd64.whl", hash = "sha256:565093ebc93b2f4b76782589564869dadfa83af5b852edebedd8fee746457d06", size = 31729230, upload-time = "2026-01-11T09:58:07.254Z" }, + { url = "https://files.pythonhosted.org/packages/7f/f1/878f8687d801d6c4565d57ebec08449c46f75126ebca8e0fed6986599627/av-16.1.0-cp313-cp313t-macosx_11_0_x86_64.whl", hash = "sha256:574081a24edb98343fd9f473e21ae155bf61443d4ec9d7708987fa597d6b04b2", size = 27008769, upload-time = "2026-01-11T09:58:10.266Z" }, + { url = "https://files.pythonhosted.org/packages/30/f1/bd4ce8c8b5cbf1d43e27048e436cbc9de628d48ede088a1d0a993768eb86/av-16.1.0-cp313-cp313t-macosx_14_0_arm64.whl", hash = "sha256:9ab00ea29c25ebf2ea1d1e928d7babb3532d562481c5d96c0829212b70756ad0", size = 21590588, upload-time = "2026-01-11T09:58:12.629Z" }, + { url = "https://files.pythonhosted.org/packages/1d/dd/c81f6f9209201ff0b5d5bed6da6c6e641eef52d8fbc930d738c3f4f6f75d/av-16.1.0-cp313-cp313t-manylinux_2_28_aarch64.whl", hash = "sha256:a84a91188c1071f238a9523fd42dbe567fb2e2607b22b779851b2ce0eac1b560", size = 40638029, upload-time = "2026-01-11T09:58:15.399Z" }, + { url = "https://files.pythonhosted.org/packages/15/4d/07edff82b78d0459a6e807e01cd280d3180ce832efc1543de80d77676722/av-16.1.0-cp313-cp313t-manylinux_2_28_x86_64.whl", hash = "sha256:c2cd0de4dd022a7225ff224fde8e7971496d700be41c50adaaa26c07bb50bf97", size = 41970776, upload-time = "2026-01-11T09:58:19.075Z" }, + { url = "https://files.pythonhosted.org/packages/da/9d/1f48b354b82fa135d388477cd1b11b81bdd4384bd6a42a60808e2ec2d66b/av-16.1.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:0816143530624a5a93bc5494f8c6eeaf77549b9366709c2ac8566c1e9bff6df5", size = 41764751, upload-time = "2026-01-11T09:58:22.788Z" }, + { url = "https://files.pythonhosted.org/packages/2f/c7/a509801e98db35ec552dd79da7bdbcff7104044bfeb4c7d196c1ce121593/av-16.1.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:e3a28053af29644696d0c007e897d19b1197585834660a54773e12a40b16974c", size = 43034355, upload-time = "2026-01-11T09:58:26.125Z" }, + { url = "https://files.pythonhosted.org/packages/36/8b/e5f530d9e8f640da5f5c5f681a424c65f9dd171c871cd255d8a861785a6e/av-16.1.0-cp313-cp313t-win_amd64.whl", hash = "sha256:2e3e67144a202b95ed299d165232533989390a9ea3119d37eccec697dc6dbb0c", size = 31947047, upload-time = "2026-01-11T09:58:31.867Z" }, + { url = "https://files.pythonhosted.org/packages/df/18/8812221108c27d19f7e5f486a82c827923061edf55f906824ee0fcaadf50/av-16.1.0-cp314-cp314-macosx_11_0_x86_64.whl", hash = "sha256:39a634d8e5a87e78ea80772774bfd20c0721f0d633837ff185f36c9d14ffede4", size = 26916179, upload-time = "2026-01-11T09:58:36.506Z" }, + { url = "https://files.pythonhosted.org/packages/38/ef/49d128a9ddce42a2766fe2b6595bd9c49e067ad8937a560f7838a541464e/av-16.1.0-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:0ba32fb9e9300948a7fa9f8a3fc686e6f7f77599a665c71eb2118fdfd2c743f9", size = 21460168, upload-time = "2026-01-11T09:58:39.231Z" }, + { url = "https://files.pythonhosted.org/packages/e6/a9/b310d390844656fa74eeb8c2750e98030877c75b97551a23a77d3f982741/av-16.1.0-cp314-cp314-manylinux_2_28_aarch64.whl", hash = "sha256:ca04d17815182d34ce3edc53cbda78a4f36e956c0fd73e3bab249872a831c4d7", size = 39210194, upload-time = "2026-01-11T09:58:42.138Z" }, + { url = "https://files.pythonhosted.org/packages/0c/7b/e65aae179929d0f173af6e474ad1489b5b5ad4c968a62c42758d619e54cf/av-16.1.0-cp314-cp314-manylinux_2_28_x86_64.whl", hash = "sha256:ee0e8de2e124a9ef53c955fe2add6ee7c56cc8fd83318265549e44057db77142", size = 40811675, upload-time = "2026-01-11T09:58:45.871Z" }, + { url = "https://files.pythonhosted.org/packages/54/3f/5d7edefd26b6a5187d6fac0f5065ee286109934f3dea607ef05e53f05b31/av-16.1.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:22bf77a2f658827043a1e184b479c3bf25c4c43ab32353677df2d119f080e28f", size = 40543942, upload-time = "2026-01-11T09:58:49.759Z" }, + { url = "https://files.pythonhosted.org/packages/1b/24/f8b17897b67be0900a211142f5646a99d896168f54d57c81f3e018853796/av-16.1.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:2dd419d262e6a71cab206d80bbf28e0a10d0f227b671cdf5e854c028faa2d043", size = 41924336, upload-time = "2026-01-11T09:58:53.344Z" }, + { url = "https://files.pythonhosted.org/packages/1c/cf/d32bc6bbbcf60b65f6510c54690ed3ae1c4ca5d9fafbce835b6056858686/av-16.1.0-cp314-cp314-win_amd64.whl", hash = "sha256:53585986fd431cd436f290fba662cfb44d9494fbc2949a183de00acc5b33fa88", size = 31735077, upload-time = "2026-01-11T09:58:56.684Z" }, + { url = "https://files.pythonhosted.org/packages/53/f4/9b63dc70af8636399bd933e9df4f3025a0294609510239782c1b746fc796/av-16.1.0-cp314-cp314t-macosx_11_0_x86_64.whl", hash = "sha256:76f5ed8495cf41e1209a5775d3699dc63fdc1740b94a095e2485f13586593205", size = 27014423, upload-time = "2026-01-11T09:58:59.703Z" }, + { url = "https://files.pythonhosted.org/packages/d1/da/787a07a0d6ed35a0888d7e5cfb8c2ffa202f38b7ad2c657299fac08eb046/av-16.1.0-cp314-cp314t-macosx_14_0_arm64.whl", hash = "sha256:8d55397190f12a1a3ae7538be58c356cceb2bf50df1b33523817587748ce89e5", size = 21595536, upload-time = "2026-01-11T09:59:02.508Z" }, + { url = "https://files.pythonhosted.org/packages/d8/f4/9a7d8651a611be6e7e3ab7b30bb43779899c8cac5f7293b9fb634c44a3f3/av-16.1.0-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:9d51d9037437218261b4bbf9df78a95e216f83d7774fbfe8d289230b5b2e28e2", size = 40642490, upload-time = "2026-01-11T09:59:05.842Z" }, + { url = "https://files.pythonhosted.org/packages/6b/e4/eb79bc538a94b4ff93cd4237d00939cba797579f3272490dd0144c165a21/av-16.1.0-cp314-cp314t-manylinux_2_28_x86_64.whl", hash = "sha256:0ce07a89c15644407f49d942111ca046e323bbab0a9078ff43ee57c9b4a50dad", size = 41976905, upload-time = "2026-01-11T09:59:09.169Z" }, + { url = "https://files.pythonhosted.org/packages/5e/f5/f6db0dd86b70167a4d55ee0d9d9640983c570d25504f2bde42599f38241e/av-16.1.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:cac0c074892ea97113b53556ff41c99562db7b9f09f098adac1f08318c2acad5", size = 41770481, upload-time = "2026-01-11T09:59:12.74Z" }, + { url = "https://files.pythonhosted.org/packages/9e/8b/33651d658e45e16ab7671ea5fcf3d20980ea7983234f4d8d0c63c65581a5/av-16.1.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:7dec3dcbc35a187ce450f65a2e0dda820d5a9e6553eea8344a1459af11c98649", size = 43036824, upload-time = "2026-01-11T09:59:16.507Z" }, + { url = "https://files.pythonhosted.org/packages/83/41/7f13361db54d7e02f11552575c0384dadaf0918138f4eaa82ea03a9f9580/av-16.1.0-cp314-cp314t-win_amd64.whl", hash = "sha256:6f90dc082ff2068ddbe77618400b44d698d25d9c4edac57459e250c16b33d700", size = 31948164, upload-time = "2026-01-11T09:59:19.501Z" }, +] + [[package]] name = "babel" version = "2.18.0" @@ -973,12 +1035,55 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/bc/58/6b3d24e6b9bc474a2dcdee65dfd1f008867015408a271562e4b690561a4d/cryptography-46.0.5-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:8456928655f856c6e1533ff59d5be76578a7157224dbd9ce6872f25055ab9ab7", size = 3407605, upload-time = "2026-02-10T19:18:29.233Z" }, ] +[[package]] +name = "ctranslate2" +version = "4.7.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, + { name = "numpy", version = "2.4.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "pyyaml" }, + { name = "setuptools" }, +] +wheels = [ + { url = "https://files.pythonhosted.org/packages/cb/e0/b69c40c3d739b213a78d327071240590792071b4f890e34088b03b95bb1e/ctranslate2-4.7.1-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:9017a355dd7c6d29dc3bca6e9fc74827306c61b702c66bb1f6b939655e7de3fa", size = 1255773, upload-time = "2026-02-04T06:11:04.769Z" }, + { url = "https://files.pythonhosted.org/packages/51/29/e5c2fc1253e3fb9b2c86997f36524bba182a8ed77fb4f8fe8444a5649191/ctranslate2-4.7.1-cp310-cp310-macosx_11_0_x86_64.whl", hash = "sha256:6abcd0552285e7173475836f9d133e04dfc3e42ca8e6930f65eaa4b8b13a47fa", size = 11914945, upload-time = "2026-02-04T06:11:06.853Z" }, + { url = "https://files.pythonhosted.org/packages/03/25/e7fe847d3f02c84d2e9c5e8312434fbeab5af3d8916b6c8e2bdbe860d052/ctranslate2-4.7.1-cp310-cp310-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:8492cba605319e0d7f2760180957d5a2a435dfdebcef1a75d2ade740e6b9fb0b", size = 16547973, upload-time = "2026-02-04T06:11:09.021Z" }, + { url = "https://files.pythonhosted.org/packages/68/75/074ed22bc340c2e26c09af6bf85859b586516e4e2d753b20189936d0dcf7/ctranslate2-4.7.1-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:688bd82482b5d057eff5bc1e727f11bb9a1277b7e4fce8ab01fd3bb70e69294b", size = 38636471, upload-time = "2026-02-04T06:11:12.146Z" }, + { url = "https://files.pythonhosted.org/packages/76/b6/9baf8a565f6dcdbfbc9cfd179dd6214529838cda4e91e89b616045a670f0/ctranslate2-4.7.1-cp310-cp310-win_amd64.whl", hash = "sha256:3b39a5f4e3c87ac91976996458a64ba08a7cbf974dc0be4e6df83a9e040d4bd2", size = 18842389, upload-time = "2026-02-04T06:11:15.154Z" }, + { url = "https://files.pythonhosted.org/packages/da/25/41920ccee68e91cb6fa0fc9e8078ab2b7839f2c668f750dc123144cb7c6e/ctranslate2-4.7.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:f74200bab9996b14a57cf6f7cb27d0921ceedc4acc1e905598e3e85b4d75b1ec", size = 1256943, upload-time = "2026-02-04T06:11:17.781Z" }, + { url = "https://files.pythonhosted.org/packages/79/22/bc81fcc9f10ba4da3ffd1a9adec15cfb73cb700b3bbe69c6c8b55d333316/ctranslate2-4.7.1-cp311-cp311-macosx_11_0_x86_64.whl", hash = "sha256:59b427eb3ac999a746315b03a63942fddd351f511db82ba1a66880d4dea98e25", size = 11916445, upload-time = "2026-02-04T06:11:19.938Z" }, + { url = "https://files.pythonhosted.org/packages/0a/a7/494a66bb02c7926331cadfff51d5ce81f5abfb1e8d05d7f2459082f31b48/ctranslate2-4.7.1-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:95f0c1051c180669d2a83a44b44b518b2d1683de125f623bbc81ad5dd6f6141c", size = 16696997, upload-time = "2026-02-04T06:11:22.697Z" }, + { url = "https://files.pythonhosted.org/packages/ed/4e/b48f79fd36e5d3c7e12db383aa49814c340921a618ef7364bd0ced670644/ctranslate2-4.7.1-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0ed92d9ab0ac6bc7005942be83d68714c80adb0897ab17f98157294ee0374347", size = 38836379, upload-time = "2026-02-04T06:11:26.325Z" }, + { url = "https://files.pythonhosted.org/packages/d2/23/8c01ac52e1f26fc4dbe985a35222ae7cd365bbf7ee5db5fd5545d8926f91/ctranslate2-4.7.1-cp311-cp311-win_amd64.whl", hash = "sha256:67d9ad9b69933fbfeee7dcec899b2cd9341d5dca4fdfb53e8ba8c109dc332ee1", size = 18843315, upload-time = "2026-02-04T06:11:29.441Z" }, + { url = "https://files.pythonhosted.org/packages/fc/0f/581de94b64c5f2327a736270bc7e7a5f8fe5cf1ed56a2203b52de4d8986a/ctranslate2-4.7.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:4c0cbd46a23b8dc37ccdbd9b447cb5f7fadc361c90e9df17d82ca84b1f019986", size = 1257089, upload-time = "2026-02-04T06:11:32.442Z" }, + { url = "https://files.pythonhosted.org/packages/3d/e9/d55b0e436362f9fe26bd98fefd2dd5d81926121f1d7f799c805e6035bb26/ctranslate2-4.7.1-cp312-cp312-macosx_11_0_x86_64.whl", hash = "sha256:5b141ddad1da5f84cf3c2a569a56227a37de649a555d376cbd9b80e8f0373dd8", size = 11918502, upload-time = "2026-02-04T06:11:33.986Z" }, + { url = "https://files.pythonhosted.org/packages/ec/ce/9f29f0b0bb4280c2ebafb3ddb6cdff8ef1c2e185ee020c0ec0ecba7dc934/ctranslate2-4.7.1-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d00a62544db4a3caaa58a3c50d39b25613c042b430053ae32384d94eb1d40990", size = 16859601, upload-time = "2026-02-04T06:11:36.227Z" }, + { url = "https://files.pythonhosted.org/packages/b3/86/428d270fd72117d19fb48ed3211aa8a3c8bd7577373252962cb634e0fd01/ctranslate2-4.7.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:722b93a89647974cbd182b4c7f87fefc7794fff7fc9cbd0303b6447905cc157e", size = 38995338, upload-time = "2026-02-04T06:11:42.789Z" }, + { url = "https://files.pythonhosted.org/packages/4a/f4/d23dbfb9c62cb642c114a30f05d753ba61d6ffbfd8a3a4012fe85a073bcb/ctranslate2-4.7.1-cp312-cp312-win_amd64.whl", hash = "sha256:d0f734dc3757118094663bdaaf713f5090c55c1927fb330a76bb8b84173940e8", size = 18844949, upload-time = "2026-02-04T06:11:45.436Z" }, + { url = "https://files.pythonhosted.org/packages/34/6d/eb49ba05db286b4ea9d5d3fcf5f5cd0a9a5e218d46349618d5041001e303/ctranslate2-4.7.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:6b2abf2929756e3ec6246057b56df379995661560a2d776af05f9d97f63afcf5", size = 1256960, upload-time = "2026-02-04T06:11:47.487Z" }, + { url = "https://files.pythonhosted.org/packages/45/5a/b9cce7b00d89fc6fdeaf27587aa52d0597b465058563e93ff50910553bdd/ctranslate2-4.7.1-cp313-cp313-macosx_11_0_x86_64.whl", hash = "sha256:857ef3959d6b1c40dc227c715a36db33db2d097164996d6c75b6db8e30828f52", size = 11918645, upload-time = "2026-02-04T06:11:49.599Z" }, + { url = "https://files.pythonhosted.org/packages/ea/03/c0db0a5276599fb44ceafa2f2cb1afd5628808ec406fe036060a39693680/ctranslate2-4.7.1-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:393a9e7e989034660526a2c0e8bb65d1924f43d9a5c77d336494a353d16ba2a4", size = 16860452, upload-time = "2026-02-04T06:11:52.276Z" }, + { url = "https://files.pythonhosted.org/packages/0b/03/4e3728ce29d192ee75ed9a2d8589bf4f19edafe5bed3845187de51b179a3/ctranslate2-4.7.1-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5a3d0682f2b9082e31c73d75b45f16cde77355ab76d7e8356a24c3cb2480a6d3", size = 38995174, upload-time = "2026-02-04T06:11:55.477Z" }, + { url = "https://files.pythonhosted.org/packages/9b/15/6e8e87c6a201d69803a79ac2e29623ce7c2cc9cd1df9db99810cca714373/ctranslate2-4.7.1-cp313-cp313-win_amd64.whl", hash = "sha256:baa6d2b10f57933d8c11791e8522659217918722d07bbef2389a443801125fe7", size = 18844953, upload-time = "2026-02-04T06:11:58.519Z" }, + { url = "https://files.pythonhosted.org/packages/fd/73/8a6b7ba18cad0c8667ee221ddab8c361cb70926440e5b8dd0e81924c28ac/ctranslate2-4.7.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:d5dfb076566551f4959dfd0706f94c923c1931def9b7bb249a2caa6ab23353a0", size = 1257560, upload-time = "2026-02-04T06:12:00.926Z" }, + { url = "https://files.pythonhosted.org/packages/70/c2/8817ca5d6c1b175b23a12f7c8b91484652f8718a76353317e5919b038733/ctranslate2-4.7.1-cp314-cp314-macosx_11_0_x86_64.whl", hash = "sha256:eecdb4ed934b384f16e8c01b185b082d6b5ffc7dcbb0b6a6eb48cd465282d957", size = 11918995, upload-time = "2026-02-04T06:12:02.875Z" }, + { url = "https://files.pythonhosted.org/packages/ac/33/b8eb3acc67bbca4d9872fc9ff94db78e6167a7ba5cd932f585d1560effc7/ctranslate2-4.7.1-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1aa6796edcc3c8d163c9e39c429d50076d266d68980fed9d1b2443f617c67e9e", size = 16844162, upload-time = "2026-02-04T06:12:05.099Z" }, + { url = "https://files.pythonhosted.org/packages/80/11/6474893b07121057035069a0a483fe1cd8c47878213f282afb4c0c6fc275/ctranslate2-4.7.1-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:24c0482c51726430fb83724451921c0e539d769c8618dcfd46b1645e7f75960d", size = 38966728, upload-time = "2026-02-04T06:12:07.923Z" }, + { url = "https://files.pythonhosted.org/packages/94/88/8fc7ff435c5e783e5fad9586d839d463e023988dbbbad949d442092d01f1/ctranslate2-4.7.1-cp314-cp314-win_amd64.whl", hash = "sha256:76db234c0446a23d20dd8eeaa7a789cc87d1d05283f48bf3152bae9fa0a69844", size = 19100788, upload-time = "2026-02-04T06:12:10.592Z" }, + { url = "https://files.pythonhosted.org/packages/d9/b3/f100013a76a98d64e67c721bd4559ea4eeb54be3e4ac45f4d801769899af/ctranslate2-4.7.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:058c9db2277dc8b19ecc86c7937628f69022f341844b9081d2ab642965d88fc6", size = 1280179, upload-time = "2026-02-04T06:12:12.596Z" }, + { url = "https://files.pythonhosted.org/packages/39/22/b77f748015667a5e2ca54a5ee080d7016fce34314f0e8cf904784549305a/ctranslate2-4.7.1-cp314-cp314t-macosx_11_0_x86_64.whl", hash = "sha256:5abcf885062c7f28a3f9a46be8d185795e8706ac6230ad086cae0bc82917df31", size = 11940166, upload-time = "2026-02-04T06:12:14.054Z" }, + { url = "https://files.pythonhosted.org/packages/7d/78/6d7fd52f646c6ba3343f71277a9bbef33734632949d1651231948b0f0359/ctranslate2-4.7.1-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9950acb04a002d5c60ae90a1ddceead1a803af1f00cadd9b1a1dc76e1f017481", size = 16849483, upload-time = "2026-02-04T06:12:17.082Z" }, + { url = "https://files.pythonhosted.org/packages/40/27/58769ff15ac31b44205bd7a8aeca80cf7357c657ea5df1b94ce0f5c83771/ctranslate2-4.7.1-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1dcc734e92e3f1ceeaa0c42bbfd009352857be179ecd4a7ed6cccc086a202f58", size = 38949393, upload-time = "2026-02-04T06:12:21.302Z" }, + { url = "https://files.pythonhosted.org/packages/0e/5c/9fa0ad6462b62efd0fb5ac1100eee47bc96ecc198ff4e237c731e5473616/ctranslate2-4.7.1-cp314-cp314t-win_amd64.whl", hash = "sha256:dfb7657bdb7b8211c8f9ecb6f3b70bc0db0e0384d01a8b1808cb66fe7199df59", size = 19123451, upload-time = "2026-02-04T06:12:24.115Z" }, +] + [[package]] name = "cuda-bindings" version = "12.9.4" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "cuda-pathfinder", marker = "(python_full_version < '3.11' and sys_platform == 'emscripten') or (python_full_version < '3.11' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'win32')" }, + { name = "cuda-pathfinder", marker = "sys_platform == 'linux'" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/7a/d8/b546104b8da3f562c1ff8ab36d130c8fe1dd6a045ced80b4f6ad74f7d4e1/cuda_bindings-12.9.4-cp310-cp310-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:4d3c842c2a4303b2a580fe955018e31aea30278be19795ae05226235268032e5", size = 12148218, upload-time = "2025-10-21T14:51:28.855Z" }, @@ -1049,6 +1154,22 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/4e/8c/f3147f5c4b73e7550fe5f9352eaa956ae838d5c51eb58e7a25b9f3e2643b/decorator-5.2.1-py3-none-any.whl", hash = "sha256:d316bb415a2d9e2d2b3abcc4084c6502fc09240e292cd76a76afc106a1c8e04a", size = 9190, upload-time = "2025-02-24T04:41:32.565Z" }, ] +[[package]] +name = "deepgram-sdk" +version = "6.0.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "httpx" }, + { name = "pydantic" }, + { name = "pydantic-core" }, + { name = "typing-extensions" }, + { name = "websockets" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/73/46/6dc45de574d766a20853452d7beccf17cb0cfeb685a0f03460f1fe49b48e/deepgram_sdk-6.0.1.tar.gz", hash = "sha256:88558a43d6173a861c8b6d6491b9ee8805679fb09fb81ef51eeb6871dad77767", size = 176743, upload-time = "2026-02-24T13:52:17.163Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/58/a4/53b9075816edc566694aed014d9864febedf232677b74f5d30bdde64b5de/deepgram_sdk-6.0.1-py3-none-any.whl", hash = "sha256:1b33d621b1c0b1d7a6a7b46fdc393aef4212e670521fada99764f5fb3f9d55fd", size = 490751, upload-time = "2026-02-24T13:52:15.998Z" }, +] + [[package]] name = "deprecated" version = "1.3.1" @@ -1199,6 +1320,22 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/9e/dd/d0ee25348ac58245ee9f90b6f3cbb666bf01f69be7e0911f9851bddbda16/fastapi-0.129.0-py3-none-any.whl", hash = "sha256:b4946880e48f462692b31c083be0432275cbfb6e2274566b1be91479cc1a84ec", size = 102950, upload-time = "2026-02-12T13:54:54.528Z" }, ] +[[package]] +name = "faster-whisper" +version = "1.2.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "av" }, + { name = "ctranslate2" }, + { name = "huggingface-hub" }, + { name = "onnxruntime" }, + { name = "tokenizers" }, + { name = "tqdm" }, +] +wheels = [ + { url = "https://files.pythonhosted.org/packages/05/99/49ee85903dee060d9f08297b4a342e5e0bcfca2f027a07b4ee0a38ab13f9/faster_whisper-1.2.1-py3-none-any.whl", hash = "sha256:79a66ad50688c0b794dd501dc340a736992a6342f7f95e5811be60b5224a26a7", size = 1118909, upload-time = "2025-10-31T11:35:47.794Z" }, +] + [[package]] name = "fastmcp" version = "3.0.2" @@ -1319,6 +1456,14 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/ec/f9/7f9263c5695f4bd0023734af91bedb2ff8209e8de6ead162f35d8dc762fd/flask-3.1.2-py3-none-any.whl", hash = "sha256:ca1d8112ec8a6158cc29ea4858963350011b5c846a414cdb7a954aa9e967d03c", size = 103308, upload-time = "2025-08-19T21:03:19.499Z" }, ] +[[package]] +name = "flatbuffers" +version = "25.12.19" +source = { registry = "https://pypi.org/simple" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e8/2d/d2a548598be01649e2d46231d151a6c56d10b964d94043a335ae56ea2d92/flatbuffers-25.12.19-py2.py3-none-any.whl", hash = "sha256:7634f50c427838bb021c2d66a3d1168e9d199b0607e6329399f04846d42e20b4", size = 26661, upload-time = "2025-12-19T23:16:13.622Z" }, +] + [[package]] name = "frozenlist" version = "1.8.0" @@ -1518,6 +1663,19 @@ requests = [ { name = "requests" }, ] +[[package]] +name = "google-auth-oauthlib" +version = "1.3.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "google-auth" }, + { name = "requests-oauthlib" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/ac/b4/1b19567e4c567b796f5c593d89895f3cfae5a38e04f27c6af87618fd0942/google_auth_oauthlib-1.3.0.tar.gz", hash = "sha256:cd39e807ac7229d6b8b9c1e297321d36fcc8a9e4857dff4301870985df51a528", size = 21777, upload-time = "2026-02-27T14:13:01.489Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/2f/56/909fd5632226d3fba31d7aeffd4754410735d49362f5809956fe3e9af344/google_auth_oauthlib-1.3.0-py3-none-any.whl", hash = "sha256:386b3fb85cf4a5b819c6ad23e3128d975216b4cac76324de1d90b128aaf38f29", size = 19308, upload-time = "2026-02-27T14:12:47.865Z" }, +] + [[package]] name = "google-genai" version = "1.64.0" @@ -1681,6 +1839,19 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/1c/ce/adfe7e5f701d503be7778291757452e3fab6b19acf51917c79f5d1cf7f8a/grpcio-1.78.1-cp314-cp314-win_amd64.whl", hash = "sha256:e2a6b33d1050dce2c6f563c5caf7f7cbeebf7fba8cde37ffe3803d50526900d1", size = 4932000, upload-time = "2026-02-20T01:15:36.127Z" }, ] +[[package]] +name = "gspread" +version = "6.2.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "google-auth" }, + { name = "google-auth-oauthlib" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/91/83/42d1d813822ed016d77aabadc99b09de3b5bd68532fd6bae23fd62347c41/gspread-6.2.1.tar.gz", hash = "sha256:2c7c99f7c32ebea6ec0d36f2d5cbe8a2be5e8f2a48bde87ad1ea203eff32bd03", size = 82590, upload-time = "2025-05-14T15:56:25.254Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/27/76/563fb20dedd0e12794d9a12cfe0198458cc0501fdc7b034eee2166d035d5/gspread-6.2.1-py3-none-any.whl", hash = "sha256:6d4ec9f1c23ae3c704a9219026dac01f2b328ac70b96f1495055d453c4c184db", size = 59977, upload-time = "2025-05-14T15:56:24.014Z" }, +] + [[package]] name = "h11" version = "0.16.0" @@ -2674,7 +2845,8 @@ name = "networkx" version = "3.4.2" source = { registry = "https://pypi.org/simple" } resolution-markers = [ - "python_full_version < '3.11'", + "python_full_version < '3.11' and sys_platform == 'linux'", + "python_full_version < '3.11' and sys_platform != 'linux'", ] sdist = { url = "https://files.pythonhosted.org/packages/fd/1d/06475e1cd5264c0b870ea2cc6fdb3e37177c1e565c43f56ff17a10e3937f/networkx-3.4.2.tar.gz", hash = "sha256:307c3669428c5362aab27c8a1260aa8f47c4e91d3891f48be0141738d8d053e1", size = 2151368, upload-time = "2024-10-21T12:39:38.695Z" } wheels = [ @@ -2688,16 +2860,20 @@ source = { registry = "https://pypi.org/simple" } resolution-markers = [ "python_full_version >= '3.14' and sys_platform == 'win32'", "python_full_version >= '3.14' and sys_platform == 'emscripten'", - "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'", + "python_full_version >= '3.14' and sys_platform == 'linux'", + "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'", "python_full_version == '3.13.*' and sys_platform == 'win32'", "python_full_version == '3.12.*' and sys_platform == 'win32'", "python_full_version == '3.13.*' and sys_platform == 'emscripten'", "python_full_version == '3.12.*' and sys_platform == 'emscripten'", - "python_full_version == '3.13.*' and sys_platform != 'emscripten' and sys_platform != 'win32'", - "python_full_version == '3.12.*' and sys_platform != 'emscripten' and sys_platform != 'win32'", + "python_full_version == '3.13.*' and sys_platform == 'linux'", + "python_full_version == '3.13.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'", + "python_full_version == '3.12.*' and sys_platform == 'linux'", + "python_full_version == '3.12.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'", "python_full_version == '3.11.*' and sys_platform == 'win32'", "python_full_version == '3.11.*' and sys_platform == 'emscripten'", - "python_full_version == '3.11.*' and sys_platform != 'emscripten' and sys_platform != 'win32'", + "python_full_version == '3.11.*' and sys_platform == 'linux'", + "python_full_version == '3.11.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'", ] sdist = { url = "https://files.pythonhosted.org/packages/6a/51/63fe664f3908c97be9d2e4f1158eb633317598cfa6e1fc14af5383f17512/networkx-3.6.1.tar.gz", hash = "sha256:26b7c357accc0c8cde558ad486283728b65b6a95d85ee1cd66bafab4c8168509", size = 2517025, upload-time = "2025-12-08T17:02:39.908Z" } wheels = [ @@ -2735,7 +2911,8 @@ name = "numpy" version = "2.2.6" source = { registry = "https://pypi.org/simple" } resolution-markers = [ - "python_full_version < '3.11'", + "python_full_version < '3.11' and sys_platform == 'linux'", + "python_full_version < '3.11' and sys_platform != 'linux'", ] sdist = { url = "https://files.pythonhosted.org/packages/76/21/7d2a95e4bba9dc13d043ee156a356c0a8f0c6309dff6b21b4d71a073b8a8/numpy-2.2.6.tar.gz", hash = "sha256:e29554e2bef54a90aa5cc07da6ce955accb83f21ab5de01a62c8478897b264fd", size = 20276440, upload-time = "2025-05-17T22:38:04.611Z" } wheels = [ @@ -2802,16 +2979,20 @@ source = { registry = "https://pypi.org/simple" } resolution-markers = [ "python_full_version >= '3.14' and sys_platform == 'win32'", "python_full_version >= '3.14' and sys_platform == 'emscripten'", - "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'", + "python_full_version >= '3.14' and sys_platform == 'linux'", + "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'", "python_full_version == '3.13.*' and sys_platform == 'win32'", "python_full_version == '3.12.*' and sys_platform == 'win32'", "python_full_version == '3.13.*' and sys_platform == 'emscripten'", "python_full_version == '3.12.*' and sys_platform == 'emscripten'", - "python_full_version == '3.13.*' and sys_platform != 'emscripten' and sys_platform != 'win32'", - "python_full_version == '3.12.*' and sys_platform != 'emscripten' and sys_platform != 'win32'", + "python_full_version == '3.13.*' and sys_platform == 'linux'", + "python_full_version == '3.13.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'", + "python_full_version == '3.12.*' and sys_platform == 'linux'", + "python_full_version == '3.12.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'", "python_full_version == '3.11.*' and sys_platform == 'win32'", "python_full_version == '3.11.*' and sys_platform == 'emscripten'", - "python_full_version == '3.11.*' and sys_platform != 'emscripten' and sys_platform != 'win32'", + "python_full_version == '3.11.*' and sys_platform == 'linux'", + "python_full_version == '3.11.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'", ] sdist = { url = "https://files.pythonhosted.org/packages/57/fd/0005efbd0af48e55eb3c7208af93f2862d4b1a56cd78e84309a2d959208d/numpy-2.4.2.tar.gz", hash = "sha256:659a6107e31a83c4e33f763942275fd278b21d095094044eb35569e86a21ddae", size = 20723651, upload-time = "2026-01-31T23:13:10.135Z" } wheels = [ @@ -2925,7 +3106,7 @@ name = "nvidia-cudnn-cu12" version = "9.10.2.21" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "nvidia-cublas-cu12", marker = "(python_full_version < '3.11' and sys_platform == 'emscripten') or (python_full_version < '3.11' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'win32')" }, + { name = "nvidia-cublas-cu12", marker = "sys_platform == 'linux'" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/ba/51/e123d997aa098c61d029f76663dedbfb9bc8dcf8c60cbd6adbe42f76d049/nvidia_cudnn_cu12-9.10.2.21-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:949452be657fa16687d0930933f032835951ef0892b37d2d53824d1a84dc97a8", size = 706758467, upload-time = "2025-06-06T21:54:08.597Z" }, @@ -2936,7 +3117,7 @@ name = "nvidia-cufft-cu12" version = "11.3.3.83" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "nvidia-nvjitlink-cu12", marker = "(python_full_version < '3.11' and sys_platform == 'emscripten') or (python_full_version < '3.11' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'win32')" }, + { name = "nvidia-nvjitlink-cu12", marker = "sys_platform == 'linux'" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/1f/13/ee4e00f30e676b66ae65b4f08cb5bcbb8392c03f54f2d5413ea99a5d1c80/nvidia_cufft_cu12-11.3.3.83-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:4d2dd21ec0b88cf61b62e6b43564355e5222e4a3fb394cac0db101f2dd0d4f74", size = 193118695, upload-time = "2025-03-07T01:45:27.821Z" }, @@ -2963,9 +3144,9 @@ name = "nvidia-cusolver-cu12" version = "11.7.3.90" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "nvidia-cublas-cu12", marker = "(python_full_version < '3.11' and sys_platform == 'emscripten') or (python_full_version < '3.11' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'win32')" }, - { name = "nvidia-cusparse-cu12", marker = "(python_full_version < '3.11' and sys_platform == 'emscripten') or (python_full_version < '3.11' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'win32')" }, - { name = "nvidia-nvjitlink-cu12", marker = "(python_full_version < '3.11' and sys_platform == 'emscripten') or (python_full_version < '3.11' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'win32')" }, + { name = "nvidia-cublas-cu12", marker = "sys_platform == 'linux'" }, + { name = "nvidia-cusparse-cu12", marker = "sys_platform == 'linux'" }, + { name = "nvidia-nvjitlink-cu12", marker = "sys_platform == 'linux'" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/85/48/9a13d2975803e8cf2777d5ed57b87a0b6ca2cc795f9a4f59796a910bfb80/nvidia_cusolver_cu12-11.7.3.90-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:4376c11ad263152bd50ea295c05370360776f8c3427b30991df774f9fb26c450", size = 267506905, upload-time = "2025-03-07T01:47:16.273Z" }, @@ -2976,7 +3157,7 @@ name = "nvidia-cusparse-cu12" version = "12.5.8.93" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "nvidia-nvjitlink-cu12", marker = "(python_full_version < '3.11' and sys_platform == 'emscripten') or (python_full_version < '3.11' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'win32')" }, + { name = "nvidia-nvjitlink-cu12", marker = "sys_platform == 'linux'" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/c2/f5/e1854cb2f2bcd4280c44736c93550cc300ff4b8c95ebe370d0aa7d2b473d/nvidia_cusparse_cu12-12.5.8.93-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:1ec05d76bbbd8b61b06a80e1eaf8cf4959c3d4ce8e711b65ebd0443bb0ebb13b", size = 288216466, upload-time = "2025-03-07T01:48:13.779Z" }, @@ -3031,6 +3212,54 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/a2/eb/86626c1bbc2edb86323022371c39aa48df6fd8b0a1647bc274577f72e90b/nvidia_nvtx_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:5b17e2001cc0d751a5bc2c6ec6d26ad95913324a4adb86788c944f8ce9ba441f", size = 89954, upload-time = "2025-03-07T01:42:44.131Z" }, ] +[[package]] +name = "oauthlib" +version = "3.3.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/0b/5f/19930f824ffeb0ad4372da4812c50edbd1434f678c90c2733e1188edfc63/oauthlib-3.3.1.tar.gz", hash = "sha256:0f0f8aa759826a193cf66c12ea1af1637f87b9b4622d46e866952bb022e538c9", size = 185918, upload-time = "2025-06-19T22:48:08.269Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/be/9c/92789c596b8df838baa98fa71844d84283302f7604ed565dafe5a6b5041a/oauthlib-3.3.1-py3-none-any.whl", hash = "sha256:88119c938d2b8fb88561af5f6ee0eec8cc8d552b7bb1f712743136eb7523b7a1", size = 160065, upload-time = "2025-06-19T22:48:06.508Z" }, +] + +[[package]] +name = "onnxruntime" +version = "1.24.2" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "flatbuffers" }, + { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, + { name = "numpy", version = "2.4.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "packaging" }, + { name = "protobuf" }, + { name = "sympy" }, +] +wheels = [ + { url = "https://files.pythonhosted.org/packages/2c/4e/050c947924ffd8ff856d219d8f83ee3d4e7dc52d5a6770ff34a15675c437/onnxruntime-1.24.2-cp311-cp311-macosx_14_0_arm64.whl", hash = "sha256:69d1c75997276106d24e65da2e69ec4302af1b117fef414e2154740cde0f6214", size = 17217298, upload-time = "2026-02-19T17:15:09.891Z" }, + { url = "https://files.pythonhosted.org/packages/30/17/c814121dff4de962476ced979c402c3cce72d5d46e87099610b47a1f2622/onnxruntime-1.24.2-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:670d7e671af2dbd17638472f9b9ff98041889efd7150718406b9ea989312d064", size = 15027128, upload-time = "2026-02-19T17:13:19.367Z" }, + { url = "https://files.pythonhosted.org/packages/2c/32/4e5921ba8b82ac37cad45f1108ca6effd430f49c7f20577d53f317d166ed/onnxruntime-1.24.2-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:93fe190ee555ae8e9c1214bcfcf13af85cd06dd835e8d835ce5a8d01056844fe", size = 17107440, upload-time = "2026-02-19T17:14:02.932Z" }, + { url = "https://files.pythonhosted.org/packages/48/55/9d13c97d912db81e81c9b369a49b36f2804fa3bb8de64462e5e6bd412d0b/onnxruntime-1.24.2-cp311-cp311-win_amd64.whl", hash = "sha256:04a3a80b28dd39739463cb1e34081eed668929ba0b8e1bc861885dcdf66b7601", size = 12506375, upload-time = "2026-02-19T17:14:57.049Z" }, + { url = "https://files.pythonhosted.org/packages/b0/d4/cf0e0b3bd84e7b68fe911810f7098f414936d1ffb612faa569a3fb8a76a5/onnxruntime-1.24.2-cp311-cp311-win_arm64.whl", hash = "sha256:a845096277444670b0b52855bb4aad706003540bd34986b50868e9f29606c142", size = 12167758, upload-time = "2026-02-19T17:14:47.386Z" }, + { url = "https://files.pythonhosted.org/packages/23/1c/38af1cfe82c75d2b205eb5019834b0f2b0b6647ec8a20a3086168e413570/onnxruntime-1.24.2-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:d8a50b422d45c0144864c0977d04ad4fa50a8a48e5153056ab1f7d06ea9fc3e2", size = 17217857, upload-time = "2026-02-19T17:15:14.297Z" }, + { url = "https://files.pythonhosted.org/packages/01/8a/e2d4332ae18d6383376e75141cd914256bee12c3cc439f42260eb176ceb9/onnxruntime-1.24.2-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:76c44fc9a89dcefcd5a4ab5c6bbbb9ff1604325ab2d5d0bc9ff5a9cba7b37f4a", size = 15027167, upload-time = "2026-02-19T17:13:21.92Z" }, + { url = "https://files.pythonhosted.org/packages/35/af/ad86cfbfd65d5a86204b3a30893e92c0cf3f1a56280efc5a12e69d81f52d/onnxruntime-1.24.2-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:09aa6f8d766b4afc3cfba68dd10be39586b49f9462fbd1386c5d5644239461ca", size = 17106547, upload-time = "2026-02-19T17:14:05.758Z" }, + { url = "https://files.pythonhosted.org/packages/ee/62/9d725326f933bf8323e309956a17e52d33fb59d35bb5dda1886f94352938/onnxruntime-1.24.2-cp312-cp312-win_amd64.whl", hash = "sha256:ebcee9276420a65e5fa08b05f18379c2271b5992617e5bdc0d0d6c5ea395c1a1", size = 12506161, upload-time = "2026-02-19T17:14:59.377Z" }, + { url = "https://files.pythonhosted.org/packages/aa/a9/7b06efd5802db881860d961a7cb4efacb058ed694c1c8f096c0c1499d017/onnxruntime-1.24.2-cp312-cp312-win_arm64.whl", hash = "sha256:8d770a934513f6e17937baf3438eaaec5983a23cdaedb81c9fc0dfcf26831c24", size = 12169884, upload-time = "2026-02-19T17:14:49.962Z" }, + { url = "https://files.pythonhosted.org/packages/9c/98/8f5b9ae63f7f6dd5fb2d192454b915ec966a421fdd0effeeef5be7f7221f/onnxruntime-1.24.2-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:038ebcd8363c3835ea83eed66129e1d11d8219438892dfb7dc7656c4d4dfa1f9", size = 17217884, upload-time = "2026-02-19T17:13:36.193Z" }, + { url = "https://files.pythonhosted.org/packages/55/e6/dc4dc59565c93506c45017c0dd3f536f6d1b7bc97047821af13fba2e3def/onnxruntime-1.24.2-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:8235cc11e118ad749c497ba93288c04073eccd8cc6cc508c8a7988ae36ab52d8", size = 15026995, upload-time = "2026-02-19T17:13:25.029Z" }, + { url = "https://files.pythonhosted.org/packages/ac/62/6f2851cf3237a91bc04cdb35434293a623d4f6369f79836929600da574ba/onnxruntime-1.24.2-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e92b46cc6d8be4286436a05382a881c88d85a2ae1ea9cfe5e6fab89f2c3e89cc", size = 17106308, upload-time = "2026-02-19T17:14:09.817Z" }, + { url = "https://files.pythonhosted.org/packages/62/5a/1e2b874daf24f26e98af14281fdbdd6ae1ed548ba471c01ea2a3084c55bb/onnxruntime-1.24.2-cp313-cp313-win_amd64.whl", hash = "sha256:1fd824ee4f6fb811bc47ffec2b25f129f31a087214ca91c8b4f6fda32962b78f", size = 12506095, upload-time = "2026-02-19T17:15:02.434Z" }, + { url = "https://files.pythonhosted.org/packages/2d/6f/8fac5eecb94f861d56a43ede3c2ebcdce60132952d3b72003f3e3d91483c/onnxruntime-1.24.2-cp313-cp313-win_arm64.whl", hash = "sha256:d8cf0acbf90771fff012c33eb2749e8aca2a8b4c66c672f30ee77c140a6fba5b", size = 12168564, upload-time = "2026-02-19T17:14:52.28Z" }, + { url = "https://files.pythonhosted.org/packages/35/e4/7dfed3f445f7289a0abff709d012439c6c901915390704dd918e5f47aad3/onnxruntime-1.24.2-cp313-cp313t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e22fb5d9ac51b61f50cca155ce2927576cc2c42501ede6c0df23a1aeb070bdd5", size = 15036844, upload-time = "2026-02-19T17:13:27.928Z" }, + { url = "https://files.pythonhosted.org/packages/90/45/9d52397e30b0d8c1692afcec5184ca9372ff4d6b0f6039bba9ad479a2563/onnxruntime-1.24.2-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:2956f5220e7be8b09482ae5726caabf78eb549142cdb28523191a38e57fb6119", size = 17117779, upload-time = "2026-02-19T17:14:13.862Z" }, + { url = "https://files.pythonhosted.org/packages/a9/c8/2321cd06ddbb4321326df365ccb8345cdb4e05643f539729f3943c706e97/onnxruntime-1.24.2-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:487e3fdedc24bc93f2acdf47c622de49b3999fb5754e7cfa466e5533a0215051", size = 17219405, upload-time = "2026-02-19T17:13:39.925Z" }, + { url = "https://files.pythonhosted.org/packages/ad/ff/a2cdf95d2647f2a5076eb3fc49ae662e375c4eb5c7b6b675f910f96c8e15/onnxruntime-1.24.2-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9c33398bd6ab1a6b7de9410af7360cd8b6312bc0c4848ddb738456c13dfbec4b", size = 15027713, upload-time = "2026-02-19T17:13:30.693Z" }, + { url = "https://files.pythonhosted.org/packages/0d/74/a1913b3a0fc2f27fe1751e9545745a3f35fd7833e3438a4208b4e215778f/onnxruntime-1.24.2-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:2658b3ce6cb33bdeddfcd74c6da509510310717611220cf2106e6c401febabe5", size = 17106108, upload-time = "2026-02-19T17:14:16.619Z" }, + { url = "https://files.pythonhosted.org/packages/0a/bd/fca80d282bca9848b2c8e101c764432dd61a0e9d2377d1c8b3bab13235d0/onnxruntime-1.24.2-cp314-cp314-win_amd64.whl", hash = "sha256:45b4f68ffec95b2cc0dc96b2b413f69ace9a80a0e5400023c5ac61f73a7a3fdf", size = 12808967, upload-time = "2026-02-19T17:15:05.1Z" }, + { url = "https://files.pythonhosted.org/packages/6d/eb/6b154dd61cac410cacf27a9f53bbf49f4dbfe5b3982f3f5b0247c7bf7b78/onnxruntime-1.24.2-cp314-cp314-win_arm64.whl", hash = "sha256:6c501aaaaa674e689aaac501e26eb96aba908ebc067fe761fbcbed868bd694a6", size = 12491892, upload-time = "2026-02-19T17:14:54.584Z" }, + { url = "https://files.pythonhosted.org/packages/6f/84/14e5e804836476d3ef6ac07afe3ed6bdf01b69f8ef3ce6ae82c6c80b6d62/onnxruntime-1.24.2-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5360d3fd9c08ce17fff757759ce4b152852be14d597130f41174d8271f954630", size = 15036834, upload-time = "2026-02-19T17:13:33.65Z" }, + { url = "https://files.pythonhosted.org/packages/3a/27/ecdd3ae7d49d9f54820ededce2d88ddc3333b9ac9bb5f1d0d6aa3148c686/onnxruntime-1.24.2-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:05a2792b5ef9278a89415a1f39d0a22192a872168257100503a5157165a38e7b", size = 17117770, upload-time = "2026-02-19T17:14:20.048Z" }, +] + [[package]] name = "openai" version = "2.21.0" @@ -3167,6 +3396,13 @@ energy-amd = [ energy-apple = [ { name = "zeus-ml" }, ] +eval-sheets = [ + { name = "google-auth" }, + { name = "gspread" }, +] +eval-wandb = [ + { name = "wandb" }, +] gpu-metrics = [ { name = "pynvml" }, ] @@ -3226,6 +3462,12 @@ server = [ { name = "pydantic" }, { name = "uvicorn" }, ] +speech = [ + { name = "faster-whisper" }, +] +speech-deepgram = [ + { name = "deepgram-sdk" }, +] tools-search = [ { name = "tavily-python" }, ] @@ -3239,10 +3481,14 @@ requires-dist = [ { name = "colbert-ai", marker = "extra == 'memory-colbert'", specifier = ">=0.2" }, { name = "croniter", marker = "extra == 'scheduler'", specifier = ">=2.0" }, { name = "cryptography", marker = "extra == 'security-signing'", specifier = ">=43" }, + { name = "deepgram-sdk", marker = "extra == 'speech-deepgram'", specifier = ">=3.0" }, { name = "discord-py", marker = "extra == 'channel-discord'", specifier = ">=2.3" }, { name = "faiss-cpu", marker = "extra == 'memory-faiss'", specifier = ">=1.7" }, { name = "fastapi", marker = "extra == 'server'", specifier = ">=0.110" }, + { name = "faster-whisper", marker = "extra == 'speech'", specifier = ">=1.0" }, + { name = "google-auth", marker = "extra == 'eval-sheets'", specifier = ">=2.0" }, { name = "google-genai", marker = "extra == 'inference-google'", specifier = ">=1.0" }, + { name = "gspread", marker = "extra == 'eval-sheets'", specifier = ">=6.0" }, { name = "httpx", specifier = ">=0.27" }, { name = "line-bot-sdk", marker = "extra == 'channel-line'", specifier = ">=3.0" }, { name = "litellm", marker = "extra == 'inference-litellm'", specifier = ">=1.40" }, @@ -3286,12 +3532,13 @@ requires-dist = [ { name = "twitchio", marker = "extra == 'channel-twitch'", specifier = ">=2.6" }, { name = "uvicorn", marker = "extra == 'server'", specifier = ">=0.30" }, { name = "viberbot", marker = "extra == 'channel-viber'", specifier = ">=1.0" }, + { name = "wandb", marker = "extra == 'eval-wandb'", specifier = ">=0.17" }, { name = "wasmtime", marker = "extra == 'sandbox-wasm'", specifier = ">=25" }, { name = "zeus-ml", extras = ["apple"], marker = "extra == 'energy-all'" }, { name = "zeus-ml", extras = ["apple"], marker = "extra == 'energy-apple'" }, { name = "zulip", marker = "extra == 'channel-zulip'", specifier = ">=0.9" }, ] -provides-extras = ["dev", "inference-ollama", "inference-vllm", "inference-llamacpp", "inference-mlx", "inference-cloud", "inference-google", "inference-litellm", "tools-search", "memory-faiss", "memory-colbert", "memory-pdf", "memory-bm25", "server", "agents", "openhands", "claude-code", "gpu-metrics", "energy-amd", "energy-apple", "energy-all", "learning", "orchestrator-training", "channel-telegram", "channel-discord", "channel-slack", "channel-webhook", "channel-email", "channel-whatsapp", "channel-signal", "channel-google-chat", "channel-irc", "channel-webchat", "channel-teams", "channel-matrix", "channel-mattermost", "channel-feishu", "channel-bluebubbles", "channel-whatsapp-baileys", "channel-line", "channel-viber", "channel-messenger", "channel-reddit", "channel-mastodon", "channel-xmpp", "channel-rocketchat", "channel-zulip", "channel-twitch", "channel-nostr", "browser", "media", "pdf", "scheduler", "security-signing", "sandbox-wasm", "dashboard", "docs"] +provides-extras = ["dev", "inference-mlx", "inference-cloud", "inference-google", "inference-litellm", "tools-search", "memory-faiss", "memory-colbert", "memory-pdf", "memory-bm25", "server", "openhands", "gpu-metrics", "energy-amd", "energy-apple", "energy-all", "orchestrator-training", "channel-telegram", "channel-discord", "channel-slack", "channel-line", "channel-viber", "channel-messenger", "channel-reddit", "channel-mastodon", "channel-xmpp", "channel-rocketchat", "channel-zulip", "channel-twitch", "channel-nostr", "browser", "media", "pdf", "scheduler", "security-signing", "sandbox-wasm", "dashboard", "speech", "speech-deepgram", "eval-wandb", "eval-sheets", "docs"] [[package]] name = "opentelemetry-api" @@ -3535,7 +3782,8 @@ name = "pandas" version = "2.3.3" source = { registry = "https://pypi.org/simple" } resolution-markers = [ - "python_full_version < '3.11'", + "python_full_version < '3.11' and sys_platform == 'linux'", + "python_full_version < '3.11' and sys_platform != 'linux'", ] dependencies = [ { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, @@ -3601,16 +3849,20 @@ source = { registry = "https://pypi.org/simple" } resolution-markers = [ "python_full_version >= '3.14' and sys_platform == 'win32'", "python_full_version >= '3.14' and sys_platform == 'emscripten'", - "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'", + "python_full_version >= '3.14' and sys_platform == 'linux'", + "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'", "python_full_version == '3.13.*' and sys_platform == 'win32'", "python_full_version == '3.12.*' and sys_platform == 'win32'", "python_full_version == '3.13.*' and sys_platform == 'emscripten'", "python_full_version == '3.12.*' and sys_platform == 'emscripten'", - "python_full_version == '3.13.*' and sys_platform != 'emscripten' and sys_platform != 'win32'", - "python_full_version == '3.12.*' and sys_platform != 'emscripten' and sys_platform != 'win32'", + "python_full_version == '3.13.*' and sys_platform == 'linux'", + "python_full_version == '3.13.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'", + "python_full_version == '3.12.*' and sys_platform == 'linux'", + "python_full_version == '3.12.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'", "python_full_version == '3.11.*' and sys_platform == 'win32'", "python_full_version == '3.11.*' and sys_platform == 'emscripten'", - "python_full_version == '3.11.*' and sys_platform != 'emscripten' and sys_platform != 'win32'", + "python_full_version == '3.11.*' and sys_platform == 'linux'", + "python_full_version == '3.11.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'", ] dependencies = [ { name = "numpy", version = "2.4.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, @@ -4884,6 +5136,19 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/1e/db/4254e3eabe8020b458f1a747140d32277ec7a271daf1d235b70dc0b4e6e3/requests-2.32.5-py3-none-any.whl", hash = "sha256:2462f94637a34fd532264295e186976db0f5d453d1cdd31473c85a6a161affb6", size = 64738, upload-time = "2025-08-18T20:46:00.542Z" }, ] +[[package]] +name = "requests-oauthlib" +version = "2.0.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "oauthlib" }, + { name = "requests" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/42/f2/05f29bc3913aea15eb670be136045bf5c5bbf4b99ecb839da9b422bb2c85/requests-oauthlib-2.0.0.tar.gz", hash = "sha256:b3dffaebd884d8cd778494369603a9e7b58d29111bf6b41bdc2dcd87203af4e9", size = 55650, upload-time = "2024-03-22T20:32:29.939Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/3b/5d/63d4ae3b9daea098d5d6f5da83984853c1bbacd5dc826764b249fe119d24/requests_oauthlib-2.0.0-py2.py3-none-any.whl", hash = "sha256:7dd8a5c40426b779b0868c404bdef9768deccf22749cde15852df527e6269b36", size = 24179, upload-time = "2024-03-22T20:32:28.055Z" }, +] + [[package]] name = "requests-toolbelt" version = "1.0.0" @@ -5137,7 +5402,8 @@ name = "scikit-learn" version = "1.7.2" source = { registry = "https://pypi.org/simple" } resolution-markers = [ - "python_full_version < '3.11'", + "python_full_version < '3.11' and sys_platform == 'linux'", + "python_full_version < '3.11' and sys_platform != 'linux'", ] dependencies = [ { name = "joblib", marker = "python_full_version < '3.11'" }, @@ -5186,16 +5452,20 @@ source = { registry = "https://pypi.org/simple" } resolution-markers = [ "python_full_version >= '3.14' and sys_platform == 'win32'", "python_full_version >= '3.14' and sys_platform == 'emscripten'", - "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'", + "python_full_version >= '3.14' and sys_platform == 'linux'", + "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'", "python_full_version == '3.13.*' and sys_platform == 'win32'", "python_full_version == '3.12.*' and sys_platform == 'win32'", "python_full_version == '3.13.*' and sys_platform == 'emscripten'", "python_full_version == '3.12.*' and sys_platform == 'emscripten'", - "python_full_version == '3.13.*' and sys_platform != 'emscripten' and sys_platform != 'win32'", - "python_full_version == '3.12.*' and sys_platform != 'emscripten' and sys_platform != 'win32'", + "python_full_version == '3.13.*' and sys_platform == 'linux'", + "python_full_version == '3.13.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'", + "python_full_version == '3.12.*' and sys_platform == 'linux'", + "python_full_version == '3.12.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'", "python_full_version == '3.11.*' and sys_platform == 'win32'", "python_full_version == '3.11.*' and sys_platform == 'emscripten'", - "python_full_version == '3.11.*' and sys_platform != 'emscripten' and sys_platform != 'win32'", + "python_full_version == '3.11.*' and sys_platform == 'linux'", + "python_full_version == '3.11.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'", ] dependencies = [ { name = "joblib", marker = "python_full_version >= '3.11'" }, @@ -5248,7 +5518,8 @@ name = "scipy" version = "1.15.3" source = { registry = "https://pypi.org/simple" } resolution-markers = [ - "python_full_version < '3.11'", + "python_full_version < '3.11' and sys_platform == 'linux'", + "python_full_version < '3.11' and sys_platform != 'linux'", ] dependencies = [ { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, @@ -5309,16 +5580,20 @@ source = { registry = "https://pypi.org/simple" } resolution-markers = [ "python_full_version >= '3.14' and sys_platform == 'win32'", "python_full_version >= '3.14' and sys_platform == 'emscripten'", - "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'", + "python_full_version >= '3.14' and sys_platform == 'linux'", + "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'", "python_full_version == '3.13.*' and sys_platform == 'win32'", "python_full_version == '3.12.*' and sys_platform == 'win32'", "python_full_version == '3.13.*' and sys_platform == 'emscripten'", "python_full_version == '3.12.*' and sys_platform == 'emscripten'", - "python_full_version == '3.13.*' and sys_platform != 'emscripten' and sys_platform != 'win32'", - "python_full_version == '3.12.*' and sys_platform != 'emscripten' and sys_platform != 'win32'", + "python_full_version == '3.13.*' and sys_platform == 'linux'", + "python_full_version == '3.13.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'", + "python_full_version == '3.12.*' and sys_platform == 'linux'", + "python_full_version == '3.12.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'", "python_full_version == '3.11.*' and sys_platform == 'win32'", "python_full_version == '3.11.*' and sys_platform == 'emscripten'", - "python_full_version == '3.11.*' and sys_platform != 'emscripten' and sys_platform != 'win32'", + "python_full_version == '3.11.*' and sys_platform == 'linux'", + "python_full_version == '3.11.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'", ] dependencies = [ { name = "numpy", version = "2.4.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, @@ -5392,8 +5667,8 @@ name = "secretstorage" version = "3.5.0" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "cryptography", marker = "python_full_version >= '3.12' and sys_platform != 'emscripten' and sys_platform != 'win32'" }, - { name = "jeepney", marker = "python_full_version >= '3.12' and sys_platform != 'emscripten' and sys_platform != 'win32'" }, + { name = "cryptography", marker = "python_full_version >= '3.12' and sys_platform == 'linux'" }, + { name = "jeepney", marker = "python_full_version >= '3.12' and sys_platform == 'linux'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/1c/03/e834bcd866f2f8a49a85eaff47340affa3bfa391ee9912a952a1faa68c7b/secretstorage-3.5.0.tar.gz", hash = "sha256:f04b8e4689cbce351744d5537bf6b1329c6fc68f91fa666f60a380edddcd11be", size = 19884, upload-time = "2025-11-23T19:02:53.191Z" } wheels = [ @@ -5451,6 +5726,19 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/03/b0/811dae8fb9f2784e138785d481469788f2e0d0c109c5737372454415f55f/sentencepiece-0.2.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:477c81505db072b3ab627e7eab972ea1025331bd3a92bacbf798df2b75ea86ec", size = 1254846, upload-time = "2025-08-12T07:00:40.611Z" }, ] +[[package]] +name = "sentry-sdk" +version = "2.54.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "certifi" }, + { name = "urllib3" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/c8/e9/2e3a46c304e7fa21eaa70612f60354e32699c7102eb961f67448e222ad7c/sentry_sdk-2.54.0.tar.gz", hash = "sha256:2620c2575128d009b11b20f7feb81e4e4e8ae08ec1d36cbc845705060b45cc1b", size = 413813, upload-time = "2026-03-02T15:12:41.355Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/53/39/be412cc86bc6247b8f69e9383d7950711bd86f8d0a4a4b0fe8fad685bc21/sentry_sdk-2.54.0-py2.py3-none-any.whl", hash = "sha256:fd74e0e281dcda63afff095d23ebcd6e97006102cdc8e78a29f19ecdf796a0de", size = 439198, upload-time = "2026-03-02T15:12:39.546Z" }, +] + [[package]] name = "setuptools" version = "82.0.0" @@ -5492,7 +5780,8 @@ name = "slixmpp" version = "1.12.0" source = { registry = "https://pypi.org/simple" } resolution-markers = [ - "python_full_version < '3.11'", + "python_full_version < '3.11' and sys_platform == 'linux'", + "python_full_version < '3.11' and sys_platform != 'linux'", ] dependencies = [ { name = "aiodns", marker = "python_full_version < '3.11'" }, @@ -5530,16 +5819,20 @@ source = { registry = "https://pypi.org/simple" } resolution-markers = [ "python_full_version >= '3.14' and sys_platform == 'win32'", "python_full_version >= '3.14' and sys_platform == 'emscripten'", - "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'", + "python_full_version >= '3.14' and sys_platform == 'linux'", + "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'", "python_full_version == '3.13.*' and sys_platform == 'win32'", "python_full_version == '3.12.*' and sys_platform == 'win32'", "python_full_version == '3.13.*' and sys_platform == 'emscripten'", "python_full_version == '3.12.*' and sys_platform == 'emscripten'", - "python_full_version == '3.13.*' and sys_platform != 'emscripten' and sys_platform != 'win32'", - "python_full_version == '3.12.*' and sys_platform != 'emscripten' and sys_platform != 'win32'", + "python_full_version == '3.13.*' and sys_platform == 'linux'", + "python_full_version == '3.13.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'", + "python_full_version == '3.12.*' and sys_platform == 'linux'", + "python_full_version == '3.12.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'", "python_full_version == '3.11.*' and sys_platform == 'win32'", "python_full_version == '3.11.*' and sys_platform == 'emscripten'", - "python_full_version == '3.11.*' and sys_platform != 'emscripten' and sys_platform != 'win32'", + "python_full_version == '3.11.*' and sys_platform == 'linux'", + "python_full_version == '3.11.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'", ] dependencies = [ { name = "aiodns", marker = "python_full_version >= '3.11'" }, @@ -5959,7 +6252,8 @@ name = "twitchio" version = "2.10.0" source = { registry = "https://pypi.org/simple" } resolution-markers = [ - "python_full_version < '3.11'", + "python_full_version < '3.11' and sys_platform == 'linux'", + "python_full_version < '3.11' and sys_platform != 'linux'", ] dependencies = [ { name = "aiohttp", marker = "python_full_version < '3.11'" }, @@ -5978,16 +6272,20 @@ source = { registry = "https://pypi.org/simple" } resolution-markers = [ "python_full_version >= '3.14' and sys_platform == 'win32'", "python_full_version >= '3.14' and sys_platform == 'emscripten'", - "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'", + "python_full_version >= '3.14' and sys_platform == 'linux'", + "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'", "python_full_version == '3.13.*' and sys_platform == 'win32'", "python_full_version == '3.12.*' and sys_platform == 'win32'", "python_full_version == '3.13.*' and sys_platform == 'emscripten'", "python_full_version == '3.12.*' and sys_platform == 'emscripten'", - "python_full_version == '3.13.*' and sys_platform != 'emscripten' and sys_platform != 'win32'", - "python_full_version == '3.12.*' and sys_platform != 'emscripten' and sys_platform != 'win32'", + "python_full_version == '3.13.*' and sys_platform == 'linux'", + "python_full_version == '3.13.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'", + "python_full_version == '3.12.*' and sys_platform == 'linux'", + "python_full_version == '3.12.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'", "python_full_version == '3.11.*' and sys_platform == 'win32'", "python_full_version == '3.11.*' and sys_platform == 'emscripten'", - "python_full_version == '3.11.*' and sys_platform != 'emscripten' and sys_platform != 'win32'", + "python_full_version == '3.11.*' and sys_platform == 'linux'", + "python_full_version == '3.11.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'", ] dependencies = [ { name = "aiohttp", marker = "python_full_version >= '3.11'" }, @@ -6217,6 +6515,35 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/5d/c1/f01846f10c8383b197a67b4cdfea6eea476aab678ed9f6f5d9ccc8c8dddf/viberbot-1.0.12-py3-none-any.whl", hash = "sha256:ca43fea2945d650c2ef2cbd777f3c546c795bf945278f6620ceda42f4101d801", size = 26164, upload-time = "2021-01-27T13:45:43.726Z" }, ] +[[package]] +name = "wandb" +version = "0.25.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "click" }, + { name = "gitpython" }, + { name = "packaging" }, + { name = "platformdirs" }, + { name = "protobuf" }, + { name = "pydantic" }, + { name = "pyyaml" }, + { name = "requests" }, + { name = "sentry-sdk" }, + { name = "typing-extensions" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/fd/60/d94952549920469524b689479c864c692ca47eca4b8c2fe3389b64a58778/wandb-0.25.0.tar.gz", hash = "sha256:45840495a288e34245d69d07b5a0b449220fbc5b032e6b51c4f92ec9026d2ad1", size = 43951335, upload-time = "2026-02-13T00:17:45.515Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c1/7d/0c131db3ec9deaabbd32263d90863cbfbe07659527e11c35a5c738cecdc5/wandb-0.25.0-py3-none-macosx_12_0_arm64.whl", hash = "sha256:5eecb3c7b5e60d1acfa4b056bfbaa0b79a482566a9db58c9f99724b3862bc8e5", size = 23287536, upload-time = "2026-02-13T00:17:20.265Z" }, + { url = "https://files.pythonhosted.org/packages/c3/95/31bb7f76a966ec87495e5a72ac7570685be162494c41757ac871768dbc4f/wandb-0.25.0-py3-none-macosx_12_0_x86_64.whl", hash = "sha256:daeedaadb183dc466e634fba90ab2bab1d4e93000912be0dee95065a0624a3fd", size = 25196062, upload-time = "2026-02-13T00:17:23.356Z" }, + { url = "https://files.pythonhosted.org/packages/d9/a1/258cdedbf30cebc692198a774cf0ef945b7ed98ee64bdaf62621281c95d8/wandb-0.25.0-py3-none-manylinux_2_28_aarch64.whl", hash = "sha256:5e0127dbcef13eea48f4b84268da7004d34d3120ebc7b2fa9cefb72b49dbb825", size = 22799744, upload-time = "2026-02-13T00:17:26.437Z" }, + { url = "https://files.pythonhosted.org/packages/de/91/ec9465d014cfd199c5b2083d271d31b3c2aedeae66f3d8a0712f7f54bdf3/wandb-0.25.0-py3-none-manylinux_2_28_x86_64.whl", hash = "sha256:6c4c38077836f9b7569a35b0e1dcf1f0c43616fcd936d182f475edbfea063665", size = 25262839, upload-time = "2026-02-13T00:17:28.8Z" }, + { url = "https://files.pythonhosted.org/packages/c7/95/cb2d1c7143f534544147fb53fe87944508b8cb9a058bc5b6f8a94adbee15/wandb-0.25.0-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:6edd8948d305cb73745bf564b807bd73da2ccbd47c548196b8a362f7df40aed8", size = 22853714, upload-time = "2026-02-13T00:17:31.68Z" }, + { url = "https://files.pythonhosted.org/packages/d7/94/68163f70c1669edcf130822aaaea782d8198b5df74443eca0085ec596774/wandb-0.25.0-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:ada6f08629bb014ad6e0a19d5dec478cdaa116431baa3f0a4bf4ab8d9893611f", size = 25358037, upload-time = "2026-02-13T00:17:34.676Z" }, + { url = "https://files.pythonhosted.org/packages/cc/fb/9578eed2c01b2fc6c8b693da110aa9c73a33d7bb556480f5cfc42e48c94e/wandb-0.25.0-py3-none-win32.whl", hash = "sha256:020b42ca4d76e347709d65f59b30d4623a115edc28f462af1c92681cb17eae7c", size = 24604118, upload-time = "2026-02-13T00:17:37.641Z" }, + { url = "https://files.pythonhosted.org/packages/25/97/460f6cb738aaa39b4eb2e6b4c630b2ae4321cdd70a79d5955ea75a878981/wandb-0.25.0-py3-none-win_amd64.whl", hash = "sha256:78307ac0b328f2dc334c8607bec772851215584b62c439eb320c4af4fb077a00", size = 24604122, upload-time = "2026-02-13T00:17:39.991Z" }, + { url = "https://files.pythonhosted.org/packages/27/6c/5847b4dda1dfd52630dac08711d4348c69ed657f0698fc2d949c7f7a6622/wandb-0.25.0-py3-none-win_arm64.whl", hash = "sha256:c6174401fd6fb726295e98d57b4231c100eca96bd17de51bfc64038a57230aaf", size = 21785298, upload-time = "2026-02-13T00:17:42.475Z" }, +] + [[package]] name = "wasmtime" version = "42.0.0"