From 9d28a728a0965bbc75ac42095e3cc57d2f0cb7f3 Mon Sep 17 00:00:00 2001 From: Michael Smith Date: Tue, 2 Jun 2026 19:25:05 -0700 Subject: [PATCH] =?UTF-8?q?feat(claude-code):=20Claude=20Code=20CLI=20prov?= =?UTF-8?q?ider=20=E2=80=94=20Phases=201=E2=80=935=20(#2472)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Steven Enamakel --- PR_DESCRIPTION.md | 148 ------- app/src-tauri/src/claude_code.rs | 71 ++++ app/src-tauri/src/lib.rs | 4 +- .../components/settings/panels/AIPanel.tsx | 68 +++- .../panels/ai/ClaudeCodeStatusCard.tsx | 244 +++++++++++ .../__tests__/ClaudeCodeStatusCard.test.tsx | 161 ++++++++ app/src/services/api/aiSettingsApi.ts | 11 +- app/src/utils/tauriCommands/config.ts | 74 ++++ gitbooks/developing/providers/claude-code.md | 91 +++++ .../inference/provider/claude_code/auth.rs | 53 +++ .../provider/claude_code/auth_status.rs | 238 +++++++++++ .../inference/provider/claude_code/driver.rs | 304 ++++++++++++++ .../provider/claude_code/event_mapper.rs | 379 ++++++++++++++++++ .../provider/claude_code/input_builder.rs | 110 +++++ .../inference/provider/claude_code/mod.rs | 240 +++++++++++ .../provider/claude_code/session_store.rs | 130 ++++++ .../provider/claude_code/stream_parser.rs | 215 ++++++++++ .../inference/provider/claude_code/types.rs | 31 ++ .../provider/claude_code/version_check.rs | 167 ++++++++ src/openhuman/inference/provider/factory.rs | 41 ++ src/openhuman/inference/provider/mod.rs | 1 + src/openhuman/inference/schemas.rs | 52 +++ tests/claude_code_stream_e2e.rs | 103 +++++ 23 files changed, 2775 insertions(+), 161 deletions(-) delete mode 100644 PR_DESCRIPTION.md create mode 100644 app/src-tauri/src/claude_code.rs create mode 100644 app/src/components/settings/panels/ai/ClaudeCodeStatusCard.tsx create mode 100644 app/src/components/settings/panels/ai/__tests__/ClaudeCodeStatusCard.test.tsx create mode 100644 gitbooks/developing/providers/claude-code.md create mode 100644 src/openhuman/inference/provider/claude_code/auth.rs create mode 100644 src/openhuman/inference/provider/claude_code/auth_status.rs create mode 100644 src/openhuman/inference/provider/claude_code/driver.rs create mode 100644 src/openhuman/inference/provider/claude_code/event_mapper.rs create mode 100644 src/openhuman/inference/provider/claude_code/input_builder.rs create mode 100644 src/openhuman/inference/provider/claude_code/mod.rs create mode 100644 src/openhuman/inference/provider/claude_code/session_store.rs create mode 100644 src/openhuman/inference/provider/claude_code/stream_parser.rs create mode 100644 src/openhuman/inference/provider/claude_code/types.rs create mode 100644 src/openhuman/inference/provider/claude_code/version_check.rs create mode 100644 tests/claude_code_stream_e2e.rs diff --git a/PR_DESCRIPTION.md b/PR_DESCRIPTION.md deleted file mode 100644 index 5b4a9b813..000000000 --- a/PR_DESCRIPTION.md +++ /dev/null @@ -1,148 +0,0 @@ -## Summary - -- Adds an iOS client for OpenHuman: device pairing via QR code, mascot chat screen, and push-to-talk voice input. -- No Rust core ships on device; the iOS app connects to the desktop core via LAN HTTP, an E2E-encrypted socket.io tunnel, or cloud HTTP fallback. -- All changes are cfg-gated or platform-guarded; the desktop build is unaffected. -- Adds the `tauri-plugin-ptt` Swift plugin (`packages/tauri-plugin-ptt/`) for AVAudioEngine + SFSpeechRecognizer on iOS. -- Adds CI sanity-check workflow, build scripts, capability catalog entries, and full docs. - -## Problem - -Users with iOS devices had no way to interact with their OpenHuman assistant on the go. The desktop app required a local machine. This PR adds the client-side scaffolding and transport layer needed to bridge iOS to an existing desktop core. - -## Solution - -The iOS app is a subset of the existing React/TypeScript UI, compiled by Tauri v2 into an iOS bundle. A `TransportManager` selects the best transport at runtime. Pairing is secured by an X25519 key agreement; all tunnel traffic uses XChaCha20-Poly1305 encryption. The backend is a blind socket.io forwarder -- it never sees plaintext. - -## Layer-by-layer commits - -| Commit | Layer | Summary | -|--------|-------|---------| -| `a99537f3` | Layer 1 | Rust devices domain -- pairing store, RPC handlers, event bus, crypto (`src/openhuman/devices/`) | -| `4ea14b78` | Layer 2 | TS transport refactor -- `TransportManager`, `LanHttpTransport`, `TunnelTransport`, `CloudHttpTransport`, tunnel crypto (`app/src/services/transport/`, `app/src/lib/tunnel/`) | -| `ba651705` | Layer 3 | Desktop `/settings/devices` UI -- `DevicesPanel`, `PairPhoneModal` with QR generation and 2-second poll | -| `3e0e2a67` | Layer 4 | Tauri shell cfg-gating -- `#[cfg(target_os = "ios")]` guards on CEF-specific code | -| `621fec98` | Layer 5 | iOS app shell -- `PairScreen` (QR scan via `AVCaptureSession`), `MascotScreen` (chat UI) | -| `5ca6cf21` | Layer 6 | `tauri-plugin-ptt` -- Swift PTT plugin (AVAudioEngine, SFSpeechRecognizer, AVSpeechSynthesizer) | -| `41a6a895` | Layer 6 fix | PTT Swift fix -- latest transcript tracking + `@unchecked Sendable` on PTTSpeaker | -| _(this PR)_ | Layers 7+8 | Build scripts, CI, Info.plist, capability catalog, docs, quality pass | - -## Test coverage - -- **Vitest:** 1957 passed, 3 skipped, 1 todo across 218 test files (includes transport, tunnel, devices, iOS, PTT suites). -- **Rust (about_app):** 20 passed -- validates catalog uniqueness, Mobile category, and new capability entries. -- **cargo check (all three Cargo.toml files):** clean (warnings only, pre-existing). - -## What is gated behind the iOS target - -The following only activates on `cfg(target_os = "ios")` or when explicitly called from iOS screens: - -- CEF exclusions in `app/src-tauri/` (accounts webviews, etc.) -- `tauri-plugin-ptt` commands (`start_listening`, `stop_listening`, `speak`, `cancel_speech`, `list_voices`) -- return `NotSupported` on non-iOS targets. -- `packages/tauri-plugin-ptt/ios/` Swift sources -- not compiled for desktop. - -Desktop users see no change. - -## Known TODOs for follow-up PRs - -- **Keychain migration:** iOS symmetric session key is in-memory only; persist to Keychain so the app reconnects after restart without re-pairing. -- **Event-driven pairing detection:** `PairPhoneModal` polls `devices_list` every 2 s. Switch to a socket event subscription when the SSE/socket bridge for `DomainEvent::DevicePaired` lands. -- **Full Xcode CI:** `cargo check --target aarch64-apple-ios` runs with `continue-on-error: true` in the new CI workflow because third-party C deps (cef-dll-sys) may fail without full Xcode on the runner. A follow-up should pin an Xcode-enabled runner and harden this to a hard gate. -- **APNs push notifications:** real-time delivery requires the app to be foregrounded. -- **Multi-region tunnel:** single backend instance only; no failover. -- **Info.plist automation:** developer must manually copy `Info.ios.plist` keys into the generated Xcode project after `tauri ios init`. Should automate via `bundle.iOS.template` once Tauri v2 stabilises the iOS template pipeline. - -## Backend dependency - -**`tinyhumansai/backend#709` must be merged and deployed before end-to-end pairing works.** The `devices_create_pairing` RPC will return a tunnel registration error until the `tunnel:register` / `tunnel:connect` / `tunnel:frame` socket.io contract is live. - -## Manual test plan for iOS reviewer - -_(Requires a physical iPhone or iOS 17+ simulator paired with the desktop app.)_ - -From `packages/tauri-plugin-ptt/README.md`: - -- [ ] Permissions dialog appears on first `startListening` call. -- [ ] Partial transcripts update while speaking; final transcript matches. -- [ ] Hold button to record, release to stop, chat message is sent with transcript. -- [ ] TTS plays through speaker by default when iPhone is held away from ear. -- [ ] BT headset routes audio correctly; disconnecting mid-recording stops gracefully. -- [ ] App backgrounded mid-record produces a final transcript and stops cleanly. -- [ ] Phone call interruption emits `ptt://error` with `code: interrupted`. -- [ ] `cancelSpeech` during TTS emits `tts-ended` with `finished: false`. -- [ ] `listVoices` returns non-empty list of `AVSpeechSynthesisVoice` entries. - -Additional pairing flow checks: - -- [ ] Desktop: Settings > Devices > "Pair iPhone" shows QR code. -- [ ] iOS app: PairScreen scans QR and transitions to MascotScreen after handshake. -- [ ] Desktop: Devices panel lists the paired device with correct label. -- [ ] Desktop: Revoke device removes it from the list; iOS app shows reconnect prompt. -- [ ] QR code expiry: code expires after TTL, "Generate new code" creates a fresh session. - -## Screenshots - -> **PLACEHOLDER:** Before opening the PR, attach screenshots of: -> - Desktop `/settings/devices` panel with a paired device. -> - iOS mascot screen showing a conversation. -> -> These require a device with Xcode signing configured and `tinyhumansai/backend#709` deployed. - -## Submission Checklist - -- [x] Tests added or updated (transport, tunnel, devices, iOS, PTT suites -- see coverage statement above). -- [x] Diff coverage note: new Rust code in `src/openhuman/devices/` was covered in Layer 1 tests; new TS code in `app/src/services/transport/` and `app/src/lib/tunnel/` covered by Vitest suites. PTT Swift layer cannot be unit-tested without iOS toolchain (noted in README). -- [x] Coverage matrix: N/A for this layer (build scripts, CI, docs, catalog). -- [x] No new external network dependencies (all transport calls use existing mock backend or real backend behind feature flag). -- [ ] Manual smoke checklist: iOS path not in `docs/RELEASE-MANUAL-SMOKE.md` yet -- tracked as follow-up. -- [ ] Linked issue: N/A (tracked via Linear). - -## Impact - -- Desktop runtime: no change. -- iOS target: new experimental app bundle (not in release pipeline yet). -- `packages/tauri-plugin-ptt/` is a new crate workspace member; adds to build time only when targeting iOS. -- Capability catalog adds three new `mobile.*` entries and a new `Mobile` category. - -## Related - -- Closes: N/A (new feature) -- Follow-up PR(s): Keychain migration, event-driven pairing, full Xcode CI, APNs. -- Backend: tinyhumansai/backend#709 - ---- - -## AI Authored PR Metadata (required for Codex/Linear PRs) - -### Linear Issue -- Key: N/A -- URL: N/A - -### Commit & Branch -- Branch: `feat/ios-client` -- Commit SHA: _(set after final commit)_ - -### Validation Run -- [x] `pnpm --filter openhuman-app format:check` -- clean -- [x] `pnpm typecheck` -- clean -- [x] Focused tests: Vitest 1957 passed; cargo about_app 20 passed -- [x] Rust fmt/check: `cargo fmt --all` + `cargo check` on all three Cargo.toml -- clean -- [x] Tauri fmt/check: included above - -### Validation Blocked -- command: `cargo check --target aarch64-apple-ios` -- error: May fail on cef-dll-sys C deps without full Xcode; guarded with `continue-on-error: true` in CI. -- impact: Soft gate only; does not block merge. - -### Behavior Changes -- Intended behavior change: Desktop users see new Settings > Devices panel. iOS users can pair and chat. -- User-visible effect: Desktop gains device management UI. iOS app becomes available for sideloading/TestFlight. - -### Parity Contract -- Legacy behavior preserved: All existing desktop flows unaffected. No CEF injection added. No new JS injection in webview accounts. -- Guard/fallback/dispatch parity: PTT commands return `NotSupported` on non-iOS. Transport falls back gracefully. - -### Duplicate / Superseded PR Handling -- Duplicate PR(s): None -- Canonical PR: This PR -- Resolution: N/A diff --git a/app/src-tauri/src/claude_code.rs b/app/src-tauri/src/claude_code.rs new file mode 100644 index 000000000..f3b3a3dee --- /dev/null +++ b/app/src-tauri/src/claude_code.rs @@ -0,0 +1,71 @@ +//! Tauri commands for the Claude Code CLI provider. +//! +//! Provides a cross-platform "open a terminal and run `claude login`" +//! helper. The CLI's OAuth flow is interactive (it prints a URL and +//! waits for the user to paste a code), so we can't host it in-app — we +//! detach into the user's native terminal so they complete login there, +//! then return to OpenHuman and click Recheck in the settings card. + +use std::process::Command; + +/// Open the user's native terminal and run `claude login` inside it. +/// +/// Returns the name of the terminal emulator we launched (for UI +/// confirmation) or an error string if no terminal could be opened. +/// +/// Platform behaviour: +/// - Windows: `cmd /c start "" cmd /k claude login` +/// - macOS: `osascript` → Terminal.app `do script "claude login"` +/// - Linux: try `x-terminal-emulator`, then `gnome-terminal`, +/// `konsole`, `xterm` in that order +#[tauri::command] +pub fn claude_code_login_launch() -> Result { + #[cfg(target_os = "windows")] + { + // `start ""` opens a new console window; the empty quoted title + // prevents cmd from interpreting the first arg as a title. + // `cmd /k` keeps the window open after `claude login` exits so + // the user can read any final output. + Command::new("cmd") + .args(["/c", "start", "", "cmd", "/k", "claude login"]) + .spawn() + .map_err(|e| format!("failed to open cmd: {e}"))?; + return Ok("cmd".into()); + } + + #[cfg(target_os = "macos")] + { + let script = r#"tell application "Terminal" + activate + do script "claude login" +end tell"#; + Command::new("osascript") + .args(["-e", script]) + .spawn() + .map_err(|e| format!("failed to open Terminal.app: {e}"))?; + return Ok("Terminal.app".into()); + } + + #[cfg(target_os = "linux")] + { + let terminals: &[(&str, &[&str])] = &[ + ("x-terminal-emulator", &["-e", "claude", "login"]), + ("gnome-terminal", &["--", "claude", "login"]), + ("konsole", &["-e", "claude", "login"]), + ("xfce4-terminal", &["-e", "claude login"]), + ("xterm", &["-e", "claude", "login"]), + ]; + for (term, args) in terminals { + match Command::new(term).args(*args).spawn() { + Ok(_) => return Ok(term.to_string()), + Err(_) => continue, + } + } + return Err("no terminal emulator found (tried x-terminal-emulator, gnome-terminal, konsole, xfce4-terminal, xterm). Run `claude login` manually.".into()); + } + + #[cfg(not(any(target_os = "windows", target_os = "macos", target_os = "linux")))] + { + Err("claude_code_login_launch is not supported on this platform".into()) + } +} diff --git a/app/src-tauri/src/lib.rs b/app/src-tauri/src/lib.rs index 7ddf9c36c..41c01934a 100644 --- a/app/src-tauri/src/lib.rs +++ b/app/src-tauri/src/lib.rs @@ -18,6 +18,7 @@ mod cef_profile; // logic is unit-tested on any host; the Win32 glue is windows-only. #[cfg(any(target_os = "windows", test))] mod cef_singleton_wait; +mod claude_code; mod companion_commands; mod core_process; mod core_rpc; @@ -3166,7 +3167,8 @@ pub fn run() { mcp_commands::mcp_resolve_binary_path, mcp_commands::mcp_open_client_config, loopback_oauth::start_loopback_oauth_listener, - loopback_oauth::stop_loopback_oauth_listener + loopback_oauth::stop_loopback_oauth_listener, + claude_code::claude_code_login_launch ]) .build(tauri::generate_context!()) .expect("error while building tauri application") diff --git a/app/src/components/settings/panels/AIPanel.tsx b/app/src/components/settings/panels/AIPanel.tsx index 8afe1bf3a..49ec9e120 100644 --- a/app/src/components/settings/panels/AIPanel.tsx +++ b/app/src/components/settings/panels/AIPanel.tsx @@ -53,6 +53,7 @@ import { import { ConfirmationModal } from '../../intelligence/ConfirmationModal'; import SettingsHeader from '../components/SettingsHeader'; import { useSettingsNavigation } from '../hooks/useSettingsNavigation'; +import { ClaudeCodeStatusCard } from './ai/ClaudeCodeStatusCard'; import { routingWithProviderRemoved } from './aiRouting'; import { authStyleForBuiltinCloudProvider, @@ -97,7 +98,8 @@ export type ProviderRef = | { kind: 'openhuman' } | { kind: 'default' } | { kind: 'cloud'; providerSlug: string; model: string; temperature?: number | null } - | { kind: 'local'; model: string; temperature?: number | null }; + | { kind: 'local'; model: string; temperature?: number | null } + | { kind: 'claude-code'; model: string; temperature?: number | null }; type Workload = { id: WorkloadId; group: WorkloadGroup; label: string; description: string }; @@ -859,6 +861,7 @@ function describeProvider(ref: ProviderRef, providers: BackgroundLoopProviderVie if (ref.kind === 'openhuman') return 'Managed · OpenHuman'; if (ref.kind === 'default') return 'Default route'; if (ref.kind === 'local') return `Local ${ref.model}`; + if (ref.kind === 'claude-code') return `Claude Code CLI ${ref.model || 'default model'}`; const provider = providers.find(p => p.slug === ref.providerSlug); return `${provider?.label ?? ref.providerSlug} ${ref.model || 'custom model'}`; } @@ -1727,7 +1730,15 @@ interface CustomRoutingDialogProps { onSubmit: (next: ProviderRef) => void; } -type CustomDialogSource = { kind: 'cloud'; providerSlug: string } | { kind: 'local' }; +type CustomDialogSource = + | { kind: 'cloud'; providerSlug: string } + | { kind: 'local' } + | { kind: 'claude-code' }; + +/** Default model identifier presented when the user first picks the + * Claude Code CLI source. The CLI accepts any model id the underlying + * Claude account can run, so this is just a sensible starting point. */ +const CLAUDE_CODE_DEFAULT_MODEL = 'sonnet-4-5'; function providerRefSignature(ref: ProviderRef): string { switch (ref.kind) { @@ -1739,6 +1750,8 @@ function providerRefSignature(ref: ProviderRef): string { return `cloud:${ref.providerSlug}:${ref.model}:${ref.temperature ?? ''}`; case 'local': return `local:${ref.model}:${ref.temperature ?? ''}`; + case 'claude-code': + return `claude-code:${ref.model}:${ref.temperature ?? ''}`; } } @@ -1814,19 +1827,23 @@ const CustomRoutingDialog = ({ ? { kind: 'cloud', providerSlug: initial.providerSlug } : initial.kind === 'local' ? { kind: 'local' } - : customCloud[0] - ? { kind: 'cloud', providerSlug: customCloud[0].slug } - : localAvailable - ? { kind: 'local' } - : null; + : initial.kind === 'claude-code' + ? { kind: 'claude-code' } + : customCloud[0] + ? { kind: 'cloud', providerSlug: customCloud[0].slug } + : localAvailable + ? { kind: 'local' } + : null; const [source, setSource] = useState(initialSource); const [model, setModel] = useState(() => { - if (initial.kind === 'cloud' || initial.kind === 'local') return initial.model; + if (initial.kind === 'cloud' || initial.kind === 'local' || initial.kind === 'claude-code') + return initial.model; if (initialSource?.kind === 'cloud') { const p = customCloud.find(c => c.slug === initialSource.providerSlug); return p ? '' : ''; } + if (initialSource?.kind === 'claude-code') return CLAUDE_CODE_DEFAULT_MODEL; return localModels[0]?.id ?? ''; }); const [cloudModels, setCloudModels] = useState([]); @@ -1841,7 +1858,9 @@ const CustomRoutingDialog = ({ // Optional temperature override for this workload. `null` = use provider/global default; // a finite number means "send `temperature: X` upstream for this workload only". const [temperature, setTemperature] = useState( - initial.kind === 'cloud' || initial.kind === 'local' ? (initial.temperature ?? null) : null + initial.kind === 'cloud' || initial.kind === 'local' || initial.kind === 'claude-code' + ? (initial.temperature ?? null) + : null ); const selectedCloud = @@ -1923,6 +1942,8 @@ const CustomRoutingDialog = ({ model: model.trim(), temperature: temp, }); + } else if (source.kind === 'claude-code') { + onSubmit({ kind: 'claude-code', model: model.trim(), temperature: temp }); } else { onSubmit({ kind: 'local', model: model.trim(), temperature: temp }); } @@ -1950,7 +1971,9 @@ const CustomRoutingDialog = ({ } }; - const noProviders = customCloud.length === 0 && !localAvailable; + // Claude Code CLI is always available as a source — never show the + // empty state when it's the only option. + const noProviders = false; return (
@@ -2021,6 +2047,7 @@ const CustomRoutingDialog = ({ ))} {localAvailable && } +
@@ -2042,6 +2069,20 @@ const CustomRoutingDialog = ({ ))} + ) : source?.kind === 'claude-code' ? ( +
+ setModel(e.target.value)} + placeholder="sonnet-4-5" + className="w-full rounded-lg border border-stone-300 dark:border-neutral-700 bg-white dark:bg-neutral-900 px-3 py-2 text-sm font-mono text-stone-900 dark:text-neutral-100 placeholder-stone-400 dark:placeholder-neutral-500 focus:border-primary-500 focus:outline-none focus:ring-1 focus:ring-primary-500" + /> +

+ Any model id your Claude account can run (e.g. sonnet-4-5,{' '} + opus-4-7). Passed verbatim to claude --model. +

+
) : cloudModelsLoading ? (