mirror of
https://github.com/tinyhumansai/openhuman.git
synced 2026-07-29 14:02:19 +00:00
refactor(core): move CLI adapters out of src/core/ into per-domain cli modules (#758)
* refactor(voice): move standalone CLI adapter into voice domain
Move the blocking `openhuman voice` / `openhuman dictate` dictation-server
subcommand out of `src/core/cli.rs` and into a new `src/openhuman/voice/cli.rs`
owned by the voice domain. The core CLI dispatcher now routes to
`voice::cli::run_standalone_subcommand` and no longer imports voice internals.
Why this shape: the standalone server blocks forever on the hotkey listener,
which doesn't fit the controller registry's request/response contract. A
domain-owned CLI adapter is the right home for long-lived operational
commands; the controller registry stays for RPC-style capabilities.
Establishes the exposure rule documented in PLAN.md for the rest of the
backend overhaul.
- src/openhuman/voice/cli.rs: new 118-line adapter (flag parsing, tokio
runtime, run_standalone call)
- src/openhuman/voice/mod.rs: declare `pub mod cli`
- src/core/cli.rs: delete 92-line run_voice_server_command, dispatch only
- PLAN.md: new scope doc + exposure rule
* refactor(text_input): move CLI adapter into domain
Move src/core/text_input_cli.rs to src/openhuman/text_input/cli.rs and
drop the stale `pub mod text_input_cli` from src/core/mod.rs. Core CLI
dispatcher now routes `text-input` to the domain-owned adapter.
The moved file has a mixed shape that validates the exposure rule from
the previous voice commit:
- `run` starts a long-lived HTTP JSON-RPC server (domain cli.rs shape)
- `read / insert / ghost / dismiss` are short-lived UX wrappers around
`text_input::rpc::*` (pretty-print, flag parsing — CLI affordance, not
transport duplication)
Both shapes legitimately belong in the domain. The controller registry
stays for transport-agnostic capabilities; CLI UX wrappers live next to
their domain RPC.
- git mv src/core/text_input_cli.rs src/openhuman/text_input/cli.rs
- src/openhuman/text_input/mod.rs: declare `pub mod cli`
- src/core/mod.rs: drop `pub mod text_input_cli`
- src/core/cli.rs: update dispatch path
* refactor(tree_summarizer): move CLI adapter into domain
git mv src/core/tree_summarizer_cli.rs src/openhuman/tree_summarizer/cli.rs
and drop the stale `pub mod tree_summarizer_cli` from src/core/mod.rs.
Core CLI dispatcher now routes `tree-summarizer` to the domain-owned
adapter.
All five subcommands (ingest / run / query / status / rebuild) are short
-lived UX wrappers over registry handlers that already exist in
tree_summarizer/schemas.rs — no long-running server, no business logic
duplicated. Straight relocation keeps transport generic.
- git mv src/core/tree_summarizer_cli.rs src/openhuman/tree_summarizer/cli.rs
- src/openhuman/tree_summarizer/mod.rs: declare `pub mod cli`
- src/core/mod.rs: drop `pub mod tree_summarizer_cli`
- src/core/cli.rs: update dispatch path
* refactor(screen_intelligence): move + split CLI adapter into domain
Move src/core/screen_intelligence_cli.rs (699 lines, 7 subcommands) into
a domain-owned cli/ submodule, split by responsibility so no single file
exceeds the project's 500-line soft limit.
Layout:
screen_intelligence/cli/mod.rs (204) dispatch + shared helpers
(CliOpts, parse_opts,
bootstrap_engine, logging,
print_help)
screen_intelligence/cli/server.rs (79) run_server (long-running
capture+vision loop)
screen_intelligence/cli/session.rs (150) status / start / stop
screen_intelligence/cli/capture.rs (173) capture / vision (inspect pair)
screen_intelligence/cli/doctor.rs (107) readiness diagnostics
Classification before moving:
- no business-logic duplication vs schemas.rs handlers
- heavy fns (doctor 103, capture 97, start 89) were 80%+ pretty-printing
and CLI-side orchestration (polling loop, flag-driven save) — legit
CLI UX, not extracted domain logic
- shared helpers stayed pub(super) inside mod.rs; no new domain APIs
introduced
Core transport changes:
- delete src/core/screen_intelligence_cli.rs
- drop `pub mod screen_intelligence_cli` from src/core/mod.rs
- src/core/cli.rs dispatch routes to domain cli::
After this commit, src/core/cli.rs contains only dispatch lines for all
four migrated domains (voice, text_input, tree_summarizer,
screen_intelligence) — no embedded domain logic.
* refactor(screen_intelligence): address external review — extract capture save + tighten cli visibility
Follow-ups from codex + gemini review of the branch:
1. Extract CaptureFrame construction + disk save from the CLI.
`capture.rs --keep` was building a CaptureFrame inline (stamping
timestamps, copying context fields) and calling
`AccessibilityEngine::save_screenshot_to_disk` directly. Added
`AccessibilityEngine::save_capture_test_result(workspace_dir,
&CaptureTestResult, reason) -> Option<Result<PathBuf, String>>`
so the CLI only passes the result through and picks a reason
label. Domain owns the frame shape, not the CLI.
2. Tighten CLI module visibility across all four migrated domains.
`pub mod cli` -> `pub(crate) mod cli` and
`pub fn run_*_command` -> `pub(crate) fn` for voice, text_input,
tree_summarizer, screen_intelligence. Only src/core/cli.rs needs
to see these; they're transport plumbing, not domain public API.
No behavior change.
* style: cargo fmt
* address review comments from coderabbitai
- screen_intelligence/cli/server: use effective keep_screenshots (opts.keep
|| config flag) for both SiServerConfig and status output, so the server
behaves consistently with what the CLI reports.
- voice/cli: surface config load errors with a warning instead of silently
falling back to Config::default(); reject unknown --mode values instead
of silently coercing them to Push.
- PLAN.md: narrow the Phase 4 enforcement grep so it forbids domain
imports/non-dispatcher references in the transport layer while still
allowing crate::openhuman::<domain>::cli::run_*(...) dispatch calls,
which this PR legitimately introduces.
* style: cargo fmt voice cli
* style: cargo fmt + bump tauri-cef vendor
* fix(screen_intelligence): narrow save_capture_test_result to pub(crate) (addresses @coderabbitai nitpick on engine.rs:486)
AccessibilityEngine::save_capture_test_result is only called from the
domain's own cli/capture.rs (crate-internal). Making it pub leaks it
into the public API surface unnecessarily — tighten to pub(crate) to
match the visibility-tightening done across the rest of this PR.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* update
* chore: update .gitignore to include scheduled_tasks.lock in app/.claude
* ran formatter
---------
Co-authored-by: Jwalin Shah <jshah1331@gmail.com>
Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Jwalin Shah
co-authored by
Claude Sonnet 4.6
Jwalin Shah
Steven Enamakel
Claude Sonnet 4.6
Jwalin Shah
Steven Enamakel
parent
8212438112
commit
325f019853
+4
-97
@@ -61,12 +61,12 @@ pub fn run_from_cli_args(args: &[String]) -> Result<()> {
|
||||
"call" => run_call_command(&args[1..]),
|
||||
// Domain-specific CLI adapters that don't follow the generic namespace pattern.
|
||||
"screen-intelligence" => {
|
||||
crate::core::screen_intelligence_cli::run_screen_intelligence_command(&args[1..])
|
||||
crate::openhuman::screen_intelligence::cli::run_screen_intelligence_command(&args[1..])
|
||||
}
|
||||
"voice" | "dictate" => run_voice_server_command(&args[1..]),
|
||||
"text-input" => crate::core::text_input_cli::run_text_input_command(&args[1..]),
|
||||
"voice" | "dictate" => crate::openhuman::voice::cli::run_standalone_subcommand(&args[1..]),
|
||||
"text-input" => crate::openhuman::text_input::cli::run_text_input_command(&args[1..]),
|
||||
"tree-summarizer" => {
|
||||
crate::core::tree_summarizer_cli::run_tree_summarizer_command(&args[1..])
|
||||
crate::openhuman::tree_summarizer::cli::run_tree_summarizer_command(&args[1..])
|
||||
}
|
||||
"memory" => crate::core::memory_cli::run_memory_command(&args[1..]),
|
||||
"agent" => {
|
||||
@@ -334,99 +334,6 @@ fn run_call_command(args: &[String]) -> Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Handles the `voice` subcommand to run the standalone voice dictation server.
|
||||
///
|
||||
/// Listens for a hotkey, records audio, transcribes via whisper, and inserts
|
||||
/// the result into the active text field.
|
||||
fn run_voice_server_command(args: &[String]) -> Result<()> {
|
||||
use crate::openhuman::voice::hotkey::ActivationMode;
|
||||
use crate::openhuman::voice::server::{run_standalone, VoiceServerConfig};
|
||||
|
||||
let mut hotkey: Option<String> = None;
|
||||
let mut mode: Option<String> = None;
|
||||
let mut skip_cleanup = false;
|
||||
let mut verbose = false;
|
||||
let mut i = 0usize;
|
||||
|
||||
while i < args.len() {
|
||||
match args[i].as_str() {
|
||||
"--hotkey" => {
|
||||
hotkey = Some(
|
||||
args.get(i + 1)
|
||||
.ok_or_else(|| anyhow::anyhow!("missing value for --hotkey"))?
|
||||
.clone(),
|
||||
);
|
||||
i += 2;
|
||||
}
|
||||
"--mode" => {
|
||||
mode = Some(
|
||||
args.get(i + 1)
|
||||
.ok_or_else(|| anyhow::anyhow!("missing value for --mode"))?
|
||||
.clone(),
|
||||
);
|
||||
i += 2;
|
||||
}
|
||||
"--skip-cleanup" => {
|
||||
skip_cleanup = true;
|
||||
i += 1;
|
||||
}
|
||||
"-v" | "--verbose" => {
|
||||
verbose = true;
|
||||
i += 1;
|
||||
}
|
||||
"-h" | "--help" => {
|
||||
println!("Usage: openhuman voice [--hotkey <combo>] [--mode <tap|push>] [--skip-cleanup] [-v]");
|
||||
println!();
|
||||
println!(" --hotkey <combo> Key combination (default: fn)");
|
||||
println!(
|
||||
" --mode <tap|push> Activation: tap to toggle, push to hold (default: push)"
|
||||
);
|
||||
println!(" --skip-cleanup Skip LLM post-processing on transcriptions");
|
||||
println!(" -v, --verbose Enable debug logging");
|
||||
println!();
|
||||
println!("Standalone voice dictation server. Press the hotkey to dictate,");
|
||||
println!("transcribed text is inserted into the active text field.");
|
||||
return Ok(());
|
||||
}
|
||||
other => return Err(anyhow::anyhow!("unknown voice arg: {other}")),
|
||||
}
|
||||
}
|
||||
|
||||
crate::core::logging::init_for_cli_run(verbose, CliLogDefault::Global);
|
||||
|
||||
let rt = tokio::runtime::Builder::new_multi_thread()
|
||||
.enable_all()
|
||||
.build()?;
|
||||
|
||||
rt.block_on(async {
|
||||
let mut config = crate::openhuman::config::Config::load_or_init()
|
||||
.await
|
||||
.unwrap_or_default();
|
||||
config.apply_env_overrides();
|
||||
|
||||
let activation_mode = match mode.as_deref() {
|
||||
Some("tap") => ActivationMode::Tap,
|
||||
_ => ActivationMode::Push,
|
||||
};
|
||||
|
||||
let server_config = VoiceServerConfig {
|
||||
hotkey: hotkey.unwrap_or_else(|| config.voice_server.hotkey.clone()),
|
||||
activation_mode,
|
||||
skip_cleanup,
|
||||
context: None,
|
||||
min_duration_secs: config.voice_server.min_duration_secs,
|
||||
silence_threshold: config.voice_server.silence_threshold,
|
||||
custom_dictionary: config.voice_server.custom_dictionary.clone(),
|
||||
};
|
||||
|
||||
run_standalone(config, server_config)
|
||||
.await
|
||||
.map_err(anyhow::Error::msg)
|
||||
})?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Dispatches commands that fall under a specific namespace (e.g., `openhuman <namespace> <function>`).
|
||||
///
|
||||
/// It looks up the function schema for validation and executes the request.
|
||||
|
||||
@@ -16,11 +16,8 @@ pub mod jsonrpc;
|
||||
pub mod logging;
|
||||
pub mod memory_cli;
|
||||
pub mod rpc_log;
|
||||
pub mod screen_intelligence_cli;
|
||||
pub mod shutdown;
|
||||
pub mod socketio;
|
||||
pub mod text_input_cli;
|
||||
pub mod tree_summarizer_cli;
|
||||
pub mod types;
|
||||
|
||||
/// Canonical function contract for domain controllers.
|
||||
|
||||
@@ -1,699 +0,0 @@
|
||||
//! `openhuman screen-intelligence` — standalone CLI for the screen intelligence loop.
|
||||
//!
|
||||
//! Boots **only** the screen intelligence engine (accessibility capture + local-AI
|
||||
//! vision) without the full desktop app, Socket.IO, or skills runtime. Useful for
|
||||
//! testing the capture → save → vision-analysis pipeline from a terminal.
|
||||
//!
|
||||
//! Usage:
|
||||
//! openhuman screen-intelligence run [--ttl <secs>] [--keep] [-v]
|
||||
//! openhuman screen-intelligence status [-v]
|
||||
//! openhuman screen-intelligence capture [--keep] [-v]
|
||||
//! openhuman screen-intelligence start [--ttl <secs>] [-v]
|
||||
//! openhuman screen-intelligence stop [-v]
|
||||
//! openhuman screen-intelligence doctor [-v]
|
||||
//! openhuman screen-intelligence vision [--limit <n>] [-v]
|
||||
|
||||
use anyhow::Result;
|
||||
use std::sync::Arc;
|
||||
|
||||
/// Entry point for `openhuman screen-intelligence <subcommand>`.
|
||||
pub fn run_screen_intelligence_command(args: &[String]) -> Result<()> {
|
||||
if args.is_empty() || is_help(&args[0]) {
|
||||
print_help();
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
match args[0].as_str() {
|
||||
"run" => run_server(&args[1..]),
|
||||
"status" => run_status(&args[1..]),
|
||||
"capture" => run_capture(&args[1..]),
|
||||
"start" => run_start_session(&args[1..]),
|
||||
"stop" => run_stop_session(&args[1..]),
|
||||
"doctor" => run_doctor(&args[1..]),
|
||||
"vision" => run_vision(&args[1..]),
|
||||
other => Err(anyhow::anyhow!(
|
||||
"unknown screen-intelligence subcommand '{other}'. Run `openhuman screen-intelligence --help`."
|
||||
)),
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Shared helpers
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
struct CliOpts {
|
||||
verbose: bool,
|
||||
ttl_secs: u64,
|
||||
keep: bool,
|
||||
limit: usize,
|
||||
no_vision_model: bool,
|
||||
}
|
||||
|
||||
fn parse_opts(args: &[String]) -> Result<(CliOpts, Vec<String>)> {
|
||||
let mut verbose = false;
|
||||
let mut ttl_secs: u64 = 300;
|
||||
let mut keep = false;
|
||||
let mut limit: usize = 10;
|
||||
let mut no_vision_model = false;
|
||||
let mut rest = Vec::new();
|
||||
let mut i = 0;
|
||||
|
||||
while i < args.len() {
|
||||
match args[i].as_str() {
|
||||
"--no-vision-model" | "--ocr-only" => {
|
||||
no_vision_model = true;
|
||||
i += 1;
|
||||
}
|
||||
"--ttl" => {
|
||||
let val = args
|
||||
.get(i + 1)
|
||||
.ok_or_else(|| anyhow::anyhow!("missing value for --ttl"))?;
|
||||
ttl_secs = val
|
||||
.parse()
|
||||
.map_err(|e| anyhow::anyhow!("invalid --ttl: {e}"))?;
|
||||
i += 2;
|
||||
}
|
||||
"--limit" => {
|
||||
let val = args
|
||||
.get(i + 1)
|
||||
.ok_or_else(|| anyhow::anyhow!("missing value for --limit"))?;
|
||||
limit = val
|
||||
.parse()
|
||||
.map_err(|e| anyhow::anyhow!("invalid --limit: {e}"))?;
|
||||
i += 2;
|
||||
}
|
||||
"--keep" => {
|
||||
keep = true;
|
||||
i += 1;
|
||||
}
|
||||
"-v" | "--verbose" => {
|
||||
verbose = true;
|
||||
i += 1;
|
||||
}
|
||||
"-h" | "--help" => {
|
||||
rest.push(args[i].clone());
|
||||
i += 1;
|
||||
}
|
||||
_ => {
|
||||
rest.push(args[i].clone());
|
||||
i += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Ok((
|
||||
CliOpts {
|
||||
verbose,
|
||||
ttl_secs,
|
||||
keep,
|
||||
limit,
|
||||
no_vision_model,
|
||||
},
|
||||
rest,
|
||||
))
|
||||
}
|
||||
|
||||
/// Bootstrap the screen intelligence engine with config.
|
||||
async fn bootstrap_engine(
|
||||
verbose: bool,
|
||||
) -> Result<Arc<crate::openhuman::screen_intelligence::AccessibilityEngine>> {
|
||||
bootstrap_engine_with_opts(verbose, false).await
|
||||
}
|
||||
|
||||
/// Bootstrap with CLI overrides.
|
||||
async fn bootstrap_engine_with_opts(
|
||||
verbose: bool,
|
||||
no_vision_model: bool,
|
||||
) -> Result<Arc<crate::openhuman::screen_intelligence::AccessibilityEngine>> {
|
||||
use crate::openhuman::config::Config;
|
||||
use crate::openhuman::screen_intelligence::global_engine;
|
||||
|
||||
let mut config = Config::load_or_init()
|
||||
.await
|
||||
.map_err(|e| anyhow::anyhow!("config load failed: {e}"))?;
|
||||
|
||||
if no_vision_model {
|
||||
config.screen_intelligence.use_vision_model = false;
|
||||
}
|
||||
|
||||
let engine = global_engine();
|
||||
let _ = engine
|
||||
.apply_config(config.screen_intelligence.clone())
|
||||
.await;
|
||||
|
||||
if verbose {
|
||||
log::info!(
|
||||
"[screen-intelligence-cli] engine initialized, enabled={}, vision={}, use_vision_model={}, keep_screenshots={}, workspace={}",
|
||||
config.screen_intelligence.enabled,
|
||||
config.screen_intelligence.vision_enabled,
|
||||
config.screen_intelligence.use_vision_model,
|
||||
config.screen_intelligence.keep_screenshots,
|
||||
config.workspace_dir.display(),
|
||||
);
|
||||
}
|
||||
|
||||
Ok(engine)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Subcommands
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// `openhuman screen-intelligence run` — start the standalone capture + vision loop.
|
||||
///
|
||||
/// Delegates to [`crate::openhuman::screen_intelligence::server::run_standalone`],
|
||||
/// which boots the engine, starts a capture session, and blocks in a
|
||||
/// monitoring loop logging captures and vision summaries until TTL or Ctrl+C.
|
||||
fn run_server(args: &[String]) -> Result<()> {
|
||||
let (opts, rest) = parse_opts(args)?;
|
||||
|
||||
if rest.iter().any(|a| is_help(a)) {
|
||||
println!("Usage: openhuman screen-intelligence run [--ttl <secs>] [--keep] [--no-vision-model] [-v]");
|
||||
println!();
|
||||
println!("Start the screen intelligence capture + vision loop.");
|
||||
println!("Captures screenshots at baseline FPS, runs OCR and vision analysis,");
|
||||
println!("and logs summaries. Blocks until TTL expires or Ctrl+C.");
|
||||
println!();
|
||||
println!(" --ttl <secs> Session duration (default: 300)");
|
||||
println!(" --keep Keep screenshots on disk after vision processing");
|
||||
println!(" --no-vision-model Skip the vision LLM — use OCR + text LLM only");
|
||||
println!(" --ocr-only Alias for --no-vision-model");
|
||||
println!(" -v, --verbose Enable debug logging");
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
crate::core::logging::init_for_cli_run(
|
||||
opts.verbose,
|
||||
crate::core::logging::CliLogDefault::Global,
|
||||
);
|
||||
|
||||
let rt = tokio::runtime::Builder::new_multi_thread()
|
||||
.enable_all()
|
||||
.build()?;
|
||||
|
||||
rt.block_on(async {
|
||||
let mut config = crate::openhuman::config::Config::load_or_init()
|
||||
.await
|
||||
.map_err(|e| anyhow::anyhow!("config load failed: {e}"))?;
|
||||
|
||||
if opts.no_vision_model {
|
||||
config.screen_intelligence.use_vision_model = false;
|
||||
}
|
||||
|
||||
let server_config = crate::openhuman::screen_intelligence::server::SiServerConfig {
|
||||
ttl_secs: opts.ttl_secs,
|
||||
log_interval_secs: 5,
|
||||
keep_screenshots: opts.keep,
|
||||
};
|
||||
|
||||
let mode_label = if config.screen_intelligence.use_vision_model {
|
||||
format!("vision LLM ({})", config.local_ai.vision_model_id)
|
||||
} else {
|
||||
"OCR + text LLM (no vision model)".to_string()
|
||||
};
|
||||
|
||||
eprintln!();
|
||||
eprintln!(" Screen Intelligence");
|
||||
eprintln!(" ───────────────────");
|
||||
eprintln!(" TTL: {}s", opts.ttl_secs);
|
||||
eprintln!(" Mode: {}", mode_label);
|
||||
eprintln!(
|
||||
" FPS: {}",
|
||||
config.screen_intelligence.baseline_fps
|
||||
);
|
||||
eprintln!(
|
||||
" Keep screenshots: {}",
|
||||
opts.keep || config.screen_intelligence.keep_screenshots
|
||||
);
|
||||
eprintln!();
|
||||
eprintln!(" Capturing → Vision → Log. Press Ctrl+C to stop.");
|
||||
eprintln!();
|
||||
|
||||
crate::openhuman::screen_intelligence::server::run_standalone(config, server_config)
|
||||
.await
|
||||
.map_err(|e| anyhow::anyhow!("{e}"))
|
||||
})
|
||||
}
|
||||
|
||||
/// `openhuman screen-intelligence status` — print current engine status as JSON.
|
||||
fn run_status(args: &[String]) -> Result<()> {
|
||||
if args.iter().any(|a| is_help(a)) {
|
||||
println!("Usage: openhuman screen-intelligence status [-v]");
|
||||
println!();
|
||||
println!("Print current screen intelligence engine status (permissions, session, config).");
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
let (opts, _) = parse_opts(args)?;
|
||||
init_quiet_logging(opts.verbose);
|
||||
|
||||
let rt = tokio::runtime::Builder::new_multi_thread()
|
||||
.enable_all()
|
||||
.build()?;
|
||||
|
||||
rt.block_on(async {
|
||||
let engine = bootstrap_engine(opts.verbose).await?;
|
||||
let status = engine.status().await;
|
||||
println!(
|
||||
"{}",
|
||||
serde_json::to_string_pretty(&status).unwrap_or_else(|_| format!("{:?}", status))
|
||||
);
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
/// `openhuman screen-intelligence capture` — take a single screenshot and print info.
|
||||
fn run_capture(args: &[String]) -> Result<()> {
|
||||
if args.iter().any(|a| is_help(a)) {
|
||||
println!("Usage: openhuman screen-intelligence capture [--keep] [-v]");
|
||||
println!();
|
||||
println!("Take a single screenshot, optionally save to workspace, and print diagnostics.");
|
||||
println!();
|
||||
println!(" --keep Save the screenshot to {{workspace}}/screenshots/");
|
||||
println!(" -v, --verbose Enable debug logging");
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
let (opts, _) = parse_opts(args)?;
|
||||
init_quiet_logging(opts.verbose);
|
||||
|
||||
let rt = tokio::runtime::Builder::new_multi_thread()
|
||||
.enable_all()
|
||||
.build()?;
|
||||
|
||||
rt.block_on(async {
|
||||
let engine = bootstrap_engine(opts.verbose).await?;
|
||||
let result = engine.capture_test().await;
|
||||
|
||||
if result.ok {
|
||||
eprintln!(" Capture: OK");
|
||||
eprintln!(" Mode: {}", result.capture_mode);
|
||||
eprintln!(" Timing: {}ms", result.timing_ms);
|
||||
if let Some(bytes) = result.bytes_estimate {
|
||||
eprintln!(" Size: {} bytes", bytes);
|
||||
}
|
||||
if let Some(ctx) = &result.context {
|
||||
eprintln!(
|
||||
" App: {}",
|
||||
ctx.app_name.as_deref().unwrap_or("unknown")
|
||||
);
|
||||
eprintln!(
|
||||
" Window: {}",
|
||||
ctx.window_title.as_deref().unwrap_or("unknown")
|
||||
);
|
||||
}
|
||||
|
||||
// Save to disk if --keep
|
||||
if opts.keep {
|
||||
if let Some(image_ref) = &result.image_ref {
|
||||
let config = crate::openhuman::config::Config::load_or_init()
|
||||
.await
|
||||
.map_err(|e| anyhow::anyhow!("config load failed: {e}"))?;
|
||||
|
||||
let frame = crate::openhuman::screen_intelligence::CaptureFrame {
|
||||
captured_at_ms: chrono::Utc::now().timestamp_millis(),
|
||||
reason: "cli_capture".to_string(),
|
||||
app_name: result
|
||||
.context
|
||||
.as_ref()
|
||||
.and_then(|c| c.app_name.clone()),
|
||||
window_title: result
|
||||
.context
|
||||
.as_ref()
|
||||
.and_then(|c| c.window_title.clone()),
|
||||
image_ref: Some(image_ref.clone()),
|
||||
};
|
||||
|
||||
match crate::openhuman::screen_intelligence::AccessibilityEngine::save_screenshot_to_disk(
|
||||
&config.workspace_dir,
|
||||
&frame,
|
||||
) {
|
||||
Ok(path) => {
|
||||
eprintln!(" Saved: {}", path.display());
|
||||
}
|
||||
Err(e) => {
|
||||
eprintln!(" Save failed: {e}");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
} else {
|
||||
eprintln!(" Capture: FAILED");
|
||||
if let Some(err) = &result.error {
|
||||
eprintln!(" Error: {err}");
|
||||
}
|
||||
std::process::exit(1);
|
||||
}
|
||||
|
||||
// Also print as JSON for machine-readable output.
|
||||
let mut json_result = serde_json::to_value(&result).unwrap_or_default();
|
||||
// Strip image_ref from JSON output (too large for terminal).
|
||||
if let Some(obj) = json_result.as_object_mut() {
|
||||
obj.remove("image_ref");
|
||||
}
|
||||
println!(
|
||||
"{}",
|
||||
serde_json::to_string_pretty(&json_result).unwrap_or_default()
|
||||
);
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
/// `openhuman screen-intelligence start` — start a capture + vision session.
|
||||
fn run_start_session(args: &[String]) -> Result<()> {
|
||||
if args.iter().any(|a| is_help(a)) {
|
||||
println!(
|
||||
"Usage: openhuman screen-intelligence start [--ttl <secs>] [--no-vision-model] [-v]"
|
||||
);
|
||||
println!();
|
||||
println!("Start a screen intelligence capture session with vision analysis.");
|
||||
println!("The session runs until TTL expires or Ctrl+C is pressed.");
|
||||
println!();
|
||||
println!(" --ttl <secs> Session duration (default: 300, max: 3600)");
|
||||
println!(" --no-vision-model Skip the vision LLM — use OCR + text LLM only");
|
||||
println!(" --ocr-only Alias for --no-vision-model");
|
||||
println!(" -v, --verbose Enable debug logging");
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
let (opts, _) = parse_opts(args)?;
|
||||
crate::core::logging::init_for_cli_run(
|
||||
opts.verbose,
|
||||
crate::core::logging::CliLogDefault::Global,
|
||||
);
|
||||
|
||||
let rt = tokio::runtime::Builder::new_multi_thread()
|
||||
.enable_all()
|
||||
.build()?;
|
||||
|
||||
rt.block_on(async {
|
||||
let engine = bootstrap_engine_with_opts(opts.verbose, opts.no_vision_model).await?;
|
||||
|
||||
let params = crate::openhuman::screen_intelligence::StartSessionParams {
|
||||
consent: true,
|
||||
ttl_secs: Some(opts.ttl_secs),
|
||||
screen_monitoring: Some(true),
|
||||
};
|
||||
|
||||
match engine.start_session(params).await {
|
||||
Ok(session) => {
|
||||
eprintln!(" Session started!");
|
||||
eprintln!(" TTL: {}s", session.ttl_secs);
|
||||
eprintln!(" Vision: {}", session.vision_enabled);
|
||||
eprintln!(" Panic hotkey: {}", session.panic_hotkey);
|
||||
eprintln!();
|
||||
eprintln!(" Capturing screenshots and running vision analysis...");
|
||||
eprintln!(" Press Ctrl+C to stop.");
|
||||
eprintln!();
|
||||
|
||||
// Print periodic status updates until the session ends.
|
||||
let mut tick = tokio::time::interval(std::time::Duration::from_secs(5));
|
||||
loop {
|
||||
tick.tick().await;
|
||||
let status = engine.status().await;
|
||||
if !status.session.active {
|
||||
eprintln!(
|
||||
"\n Session ended: {}",
|
||||
status
|
||||
.session
|
||||
.stop_reason
|
||||
.unwrap_or_else(|| "unknown".into())
|
||||
);
|
||||
break;
|
||||
}
|
||||
eprintln!(
|
||||
" [{}] captures={} vision={} queue={} last_app={:?}",
|
||||
chrono::Utc::now().format("%H:%M:%S"),
|
||||
status.session.capture_count,
|
||||
status.session.vision_state,
|
||||
status.session.vision_queue_depth,
|
||||
status.session.last_context.as_deref().unwrap_or("-"),
|
||||
);
|
||||
if let Some(summary) = &status.session.last_vision_summary {
|
||||
let truncated = if summary.chars().count() > 100 {
|
||||
format!("{}…", summary.chars().take(100).collect::<String>())
|
||||
} else {
|
||||
summary.clone()
|
||||
};
|
||||
eprintln!(" notes: {truncated}");
|
||||
}
|
||||
}
|
||||
}
|
||||
Err(e) => {
|
||||
eprintln!(" Failed to start session: {e}");
|
||||
std::process::exit(1);
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
/// `openhuman screen-intelligence stop` — stop an active session.
|
||||
fn run_stop_session(args: &[String]) -> Result<()> {
|
||||
if args.iter().any(|a| is_help(a)) {
|
||||
println!("Usage: openhuman screen-intelligence stop [-v]");
|
||||
println!();
|
||||
println!("Stop the active screen intelligence session.");
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
let (opts, _) = parse_opts(args)?;
|
||||
init_quiet_logging(opts.verbose);
|
||||
|
||||
let rt = tokio::runtime::Builder::new_multi_thread()
|
||||
.enable_all()
|
||||
.build()?;
|
||||
|
||||
rt.block_on(async {
|
||||
let engine = bootstrap_engine(opts.verbose).await?;
|
||||
let session = engine.stop_session(Some("cli_stop".to_string())).await;
|
||||
eprintln!(
|
||||
" Session stopped: {}",
|
||||
session
|
||||
.stop_reason
|
||||
.unwrap_or_else(|| "no active session".into())
|
||||
);
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
/// `openhuman screen-intelligence doctor` — diagnostic readiness check.
|
||||
fn run_doctor(args: &[String]) -> Result<()> {
|
||||
if args.iter().any(|a| is_help(a)) {
|
||||
println!("Usage: openhuman screen-intelligence doctor [-v]");
|
||||
println!();
|
||||
println!("Check system readiness: permissions, platform support, vision config.");
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
let (opts, _) = parse_opts(args)?;
|
||||
init_quiet_logging(opts.verbose);
|
||||
|
||||
let rt = tokio::runtime::Builder::new_multi_thread()
|
||||
.enable_all()
|
||||
.build()?;
|
||||
|
||||
rt.block_on(async {
|
||||
let _engine = bootstrap_engine(opts.verbose).await?;
|
||||
|
||||
let doctor_json =
|
||||
crate::openhuman::screen_intelligence::rpc::accessibility_doctor_cli_json()
|
||||
.await
|
||||
.map_err(|e| anyhow::anyhow!("doctor check failed: {e}"))?;
|
||||
|
||||
let summary = &doctor_json["result"]["summary"];
|
||||
let recommendations = &doctor_json["result"]["recommendations"];
|
||||
let permissions = &doctor_json["result"]["permissions"];
|
||||
|
||||
eprintln!(" Screen Intelligence Doctor");
|
||||
eprintln!(" ──────────────────────────");
|
||||
eprintln!();
|
||||
|
||||
let check = |ok: bool| if ok { "✓" } else { "✗" };
|
||||
|
||||
let platform_ok = summary["platform_supported"].as_bool().unwrap_or(false);
|
||||
let screen_ok = summary["screen_capture_ready"].as_bool().unwrap_or(false);
|
||||
let control_ok = summary["accessibility_ready"].as_bool().unwrap_or(false);
|
||||
let input_ok = summary["input_monitoring_ready"].as_bool().unwrap_or(false);
|
||||
let overall_ok = summary["overall_ready"].as_bool().unwrap_or(false);
|
||||
|
||||
eprintln!(" {} Platform supported", check(platform_ok));
|
||||
eprintln!(" {} Screen recording", check(screen_ok));
|
||||
eprintln!(" {} Accessibility automation", check(control_ok));
|
||||
eprintln!(" {} Input monitoring", check(input_ok));
|
||||
eprintln!();
|
||||
|
||||
// Vision config check.
|
||||
let config = crate::openhuman::config::Config::load_or_init().await.ok();
|
||||
if let Some(ref cfg) = config {
|
||||
let si = &cfg.screen_intelligence;
|
||||
let la = &cfg.local_ai;
|
||||
eprintln!(" Config:");
|
||||
eprintln!(" enabled: {}", si.enabled);
|
||||
eprintln!(" vision_enabled: {}", si.vision_enabled);
|
||||
eprintln!(" use_vision_model: {}", si.use_vision_model);
|
||||
eprintln!(" baseline_fps: {}", si.baseline_fps);
|
||||
eprintln!(" keep_screenshots: {}", si.keep_screenshots);
|
||||
eprintln!(" local_ai.enabled: {}", la.enabled);
|
||||
eprintln!(" local_ai.provider: {}", la.provider);
|
||||
if si.vision_enabled && !la.enabled {
|
||||
eprintln!(" ⚠ Vision is enabled but local_ai.enabled=false — vision analysis will fail");
|
||||
}
|
||||
}
|
||||
|
||||
eprintln!();
|
||||
if overall_ok {
|
||||
eprintln!(" ✓ Overall: READY");
|
||||
} else {
|
||||
eprintln!(" ✗ Overall: NOT READY");
|
||||
eprintln!();
|
||||
eprintln!(" Recommendations:");
|
||||
if let Some(recs) = recommendations.as_array() {
|
||||
for rec in recs {
|
||||
if let Some(s) = rec.as_str() {
|
||||
eprintln!(" • {s}");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
eprintln!();
|
||||
|
||||
// Machine-readable JSON output.
|
||||
println!(
|
||||
"{}",
|
||||
serde_json::to_string_pretty(&serde_json::json!({
|
||||
"summary": summary,
|
||||
"permissions": permissions,
|
||||
"config": config.as_ref().map(|c| serde_json::json!({
|
||||
"enabled": c.screen_intelligence.enabled,
|
||||
"vision_enabled": c.screen_intelligence.vision_enabled,
|
||||
"use_vision_model": c.screen_intelligence.use_vision_model,
|
||||
"baseline_fps": c.screen_intelligence.baseline_fps,
|
||||
"keep_screenshots": c.screen_intelligence.keep_screenshots,
|
||||
"local_ai_enabled": c.local_ai.enabled,
|
||||
"local_ai_provider": c.local_ai.provider,
|
||||
})),
|
||||
}))
|
||||
.unwrap_or_default()
|
||||
);
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
/// `openhuman screen-intelligence vision` — inspect recent vision summaries.
|
||||
fn run_vision(args: &[String]) -> Result<()> {
|
||||
if args.iter().any(|a| is_help(a)) {
|
||||
println!("Usage: openhuman screen-intelligence vision [--limit <n>] [-v]");
|
||||
println!();
|
||||
println!("Print recent vision summaries from the active session.");
|
||||
println!();
|
||||
println!(" --limit <n> Maximum summaries to show (default: 10)");
|
||||
println!(" -v, --verbose Enable debug logging");
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
let (opts, _) = parse_opts(args)?;
|
||||
init_quiet_logging(opts.verbose);
|
||||
|
||||
let rt = tokio::runtime::Builder::new_multi_thread()
|
||||
.enable_all()
|
||||
.build()?;
|
||||
|
||||
rt.block_on(async {
|
||||
let engine = bootstrap_engine(opts.verbose).await?;
|
||||
let result = engine.vision_recent(Some(opts.limit)).await;
|
||||
|
||||
if result.summaries.is_empty() {
|
||||
eprintln!(" No vision summaries available.");
|
||||
eprintln!(" Start a session first: openhuman screen-intelligence start");
|
||||
} else {
|
||||
eprintln!(" {} vision summary(ies):\n", result.summaries.len());
|
||||
for (i, s) in result.summaries.iter().enumerate() {
|
||||
let ts = chrono::DateTime::from_timestamp_millis(s.captured_at_ms)
|
||||
.map(|dt| dt.format("%H:%M:%S").to_string())
|
||||
.unwrap_or_else(|| "?".to_string());
|
||||
eprintln!(
|
||||
" [{}] {} — {} (confidence: {:.0}%)",
|
||||
i + 1,
|
||||
ts,
|
||||
s.app_name.as_deref().unwrap_or("unknown"),
|
||||
s.confidence * 100.0,
|
||||
);
|
||||
if !s.ui_state.is_empty() {
|
||||
let truncated = if s.ui_state.chars().count() > 120 {
|
||||
format!("{}…", s.ui_state.chars().take(120).collect::<String>())
|
||||
} else {
|
||||
s.ui_state.clone()
|
||||
};
|
||||
eprintln!(" ui: {truncated}");
|
||||
}
|
||||
if !s.actionable_notes.is_empty() {
|
||||
let truncated = if s.actionable_notes.chars().count() > 120 {
|
||||
format!(
|
||||
"{}…",
|
||||
s.actionable_notes.chars().take(120).collect::<String>()
|
||||
)
|
||||
} else {
|
||||
s.actionable_notes.clone()
|
||||
};
|
||||
eprintln!(" notes: {truncated}");
|
||||
}
|
||||
eprintln!();
|
||||
}
|
||||
}
|
||||
|
||||
// Machine-readable output.
|
||||
println!(
|
||||
"{}",
|
||||
serde_json::to_string_pretty(&result).unwrap_or_default()
|
||||
);
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Helpers
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Quiet logging: only `warn` unless verbose (used for non-server subcommands).
|
||||
fn init_quiet_logging(verbose: bool) {
|
||||
if !verbose && std::env::var_os("RUST_LOG").is_none() {
|
||||
std::env::set_var("RUST_LOG", "warn");
|
||||
}
|
||||
crate::core::logging::init_for_cli_run(verbose, crate::core::logging::CliLogDefault::Global);
|
||||
}
|
||||
|
||||
fn is_help(value: &str) -> bool {
|
||||
matches!(value, "-h" | "--help" | "help")
|
||||
}
|
||||
|
||||
fn print_help() {
|
||||
println!("openhuman screen-intelligence — standalone screen intelligence runtime\n");
|
||||
println!("Boots only the screen intelligence engine (accessibility capture + local-AI");
|
||||
println!("vision) without the full desktop app, Socket.IO, or skills runtime.\n");
|
||||
println!("Usage:");
|
||||
println!(" openhuman screen-intelligence run [--ttl <secs>] [--no-vision-model] [-v]");
|
||||
println!(" openhuman screen-intelligence status [-v]");
|
||||
println!(" openhuman screen-intelligence capture [--keep] [-v]");
|
||||
println!(" openhuman screen-intelligence start [--ttl <secs>] [--no-vision-model] [-v]");
|
||||
println!(" openhuman screen-intelligence stop [-v]");
|
||||
println!(" openhuman screen-intelligence doctor [-v]");
|
||||
println!(" openhuman screen-intelligence vision [--limit <n>] [-v]");
|
||||
println!();
|
||||
println!("Subcommands:");
|
||||
println!(" run Start the capture → vision → log loop (blocks until TTL/Ctrl+C)");
|
||||
println!(" status Print current engine status (permissions, session, config)");
|
||||
println!(" capture Take a single screenshot and print diagnostics");
|
||||
println!(" start Start a capture + vision session (runs until TTL or Ctrl+C)");
|
||||
println!(" stop Stop the active session");
|
||||
println!(" doctor Check system readiness (permissions, vision config, platform)");
|
||||
println!(" vision Print recent vision summaries from the active session");
|
||||
println!();
|
||||
println!("Common options:");
|
||||
println!(" --ttl <secs> Session TTL (default: 300)");
|
||||
println!(" --limit <n> Max vision summaries for 'vision' (default: 10)");
|
||||
println!(" --keep Save screenshot to disk (for 'capture')");
|
||||
println!(" --no-vision-model Skip vision LLM — use OCR + text LLM only");
|
||||
println!(" --ocr-only Alias for --no-vision-model");
|
||||
println!(" -v, --verbose Enable debug logging");
|
||||
}
|
||||
@@ -0,0 +1,155 @@
|
||||
//! Capture + vision inspection subcommands: `capture`, `vision`.
|
||||
|
||||
use anyhow::Result;
|
||||
|
||||
use super::{bootstrap_engine, init_quiet_logging, is_help, parse_opts};
|
||||
|
||||
/// `openhuman screen-intelligence capture` — take a single screenshot and print info.
|
||||
pub(super) fn run_capture(args: &[String]) -> Result<()> {
|
||||
if args.iter().any(|a| is_help(a)) {
|
||||
println!("Usage: openhuman screen-intelligence capture [--keep] [-v]");
|
||||
println!();
|
||||
println!("Take a single screenshot, optionally save to workspace, and print diagnostics.");
|
||||
println!();
|
||||
println!(" --keep Save the screenshot to {{workspace}}/screenshots/");
|
||||
println!(" -v, --verbose Enable debug logging");
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
let (opts, _) = parse_opts(args)?;
|
||||
init_quiet_logging(opts.verbose);
|
||||
|
||||
let rt = tokio::runtime::Builder::new_multi_thread()
|
||||
.enable_all()
|
||||
.build()?;
|
||||
|
||||
rt.block_on(async {
|
||||
let engine = bootstrap_engine(opts.verbose).await?;
|
||||
let result = engine.capture_test().await;
|
||||
|
||||
if result.ok {
|
||||
eprintln!(" Capture: OK");
|
||||
eprintln!(" Mode: {}", result.capture_mode);
|
||||
eprintln!(" Timing: {}ms", result.timing_ms);
|
||||
if let Some(bytes) = result.bytes_estimate {
|
||||
eprintln!(" Size: {} bytes", bytes);
|
||||
}
|
||||
if let Some(ctx) = &result.context {
|
||||
eprintln!(
|
||||
" App: {}",
|
||||
ctx.app_name.as_deref().unwrap_or("unknown")
|
||||
);
|
||||
eprintln!(
|
||||
" Window: {}",
|
||||
ctx.window_title.as_deref().unwrap_or("unknown")
|
||||
);
|
||||
}
|
||||
|
||||
// Save to disk if --keep
|
||||
if opts.keep {
|
||||
let config = crate::openhuman::config::Config::load_or_init()
|
||||
.await
|
||||
.map_err(|e| anyhow::anyhow!("config load failed: {e}"))?;
|
||||
|
||||
match crate::openhuman::screen_intelligence::AccessibilityEngine::save_capture_test_result(
|
||||
&config.workspace_dir,
|
||||
&result,
|
||||
"cli_capture",
|
||||
) {
|
||||
Some(Ok(path)) => eprintln!(" Saved: {}", path.display()),
|
||||
Some(Err(e)) => eprintln!(" Save failed: {e}"),
|
||||
None => {}
|
||||
}
|
||||
}
|
||||
} else {
|
||||
eprintln!(" Capture: FAILED");
|
||||
if let Some(err) = &result.error {
|
||||
eprintln!(" Error: {err}");
|
||||
}
|
||||
std::process::exit(1);
|
||||
}
|
||||
|
||||
// Also print as JSON for machine-readable output.
|
||||
let mut json_result = serde_json::to_value(&result).unwrap_or_default();
|
||||
// Strip image_ref from JSON output (too large for terminal).
|
||||
if let Some(obj) = json_result.as_object_mut() {
|
||||
obj.remove("image_ref");
|
||||
}
|
||||
println!(
|
||||
"{}",
|
||||
serde_json::to_string_pretty(&json_result).unwrap_or_default()
|
||||
);
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
/// `openhuman screen-intelligence vision` — inspect recent vision summaries.
|
||||
pub(super) fn run_vision(args: &[String]) -> Result<()> {
|
||||
if args.iter().any(|a| is_help(a)) {
|
||||
println!("Usage: openhuman screen-intelligence vision [--limit <n>] [-v]");
|
||||
println!();
|
||||
println!("Print recent vision summaries from the active session.");
|
||||
println!();
|
||||
println!(" --limit <n> Maximum summaries to show (default: 10)");
|
||||
println!(" -v, --verbose Enable debug logging");
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
let (opts, _) = parse_opts(args)?;
|
||||
init_quiet_logging(opts.verbose);
|
||||
|
||||
let rt = tokio::runtime::Builder::new_multi_thread()
|
||||
.enable_all()
|
||||
.build()?;
|
||||
|
||||
rt.block_on(async {
|
||||
let engine = bootstrap_engine(opts.verbose).await?;
|
||||
let result = engine.vision_recent(Some(opts.limit)).await;
|
||||
|
||||
if result.summaries.is_empty() {
|
||||
eprintln!(" No vision summaries available.");
|
||||
eprintln!(" Start a session first: openhuman screen-intelligence start");
|
||||
} else {
|
||||
eprintln!(" {} vision summary(ies):\n", result.summaries.len());
|
||||
for (i, s) in result.summaries.iter().enumerate() {
|
||||
let ts = chrono::DateTime::from_timestamp_millis(s.captured_at_ms)
|
||||
.map(|dt| dt.format("%H:%M:%S").to_string())
|
||||
.unwrap_or_else(|| "?".to_string());
|
||||
eprintln!(
|
||||
" [{}] {} — {} (confidence: {:.0}%)",
|
||||
i + 1,
|
||||
ts,
|
||||
s.app_name.as_deref().unwrap_or("unknown"),
|
||||
s.confidence * 100.0,
|
||||
);
|
||||
if !s.ui_state.is_empty() {
|
||||
let truncated = if s.ui_state.chars().count() > 120 {
|
||||
format!("{}…", s.ui_state.chars().take(120).collect::<String>())
|
||||
} else {
|
||||
s.ui_state.clone()
|
||||
};
|
||||
eprintln!(" ui: {truncated}");
|
||||
}
|
||||
if !s.actionable_notes.is_empty() {
|
||||
let truncated = if s.actionable_notes.chars().count() > 120 {
|
||||
format!(
|
||||
"{}…",
|
||||
s.actionable_notes.chars().take(120).collect::<String>()
|
||||
)
|
||||
} else {
|
||||
s.actionable_notes.clone()
|
||||
};
|
||||
eprintln!(" notes: {truncated}");
|
||||
}
|
||||
eprintln!();
|
||||
}
|
||||
}
|
||||
|
||||
// Machine-readable output.
|
||||
println!(
|
||||
"{}",
|
||||
serde_json::to_string_pretty(&result).unwrap_or_default()
|
||||
);
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,107 @@
|
||||
//! `openhuman screen-intelligence doctor` — diagnostic readiness check.
|
||||
|
||||
use anyhow::Result;
|
||||
|
||||
use super::{bootstrap_engine, init_quiet_logging, is_help, parse_opts};
|
||||
|
||||
pub(super) fn run_doctor(args: &[String]) -> Result<()> {
|
||||
if args.iter().any(|a| is_help(a)) {
|
||||
println!("Usage: openhuman screen-intelligence doctor [-v]");
|
||||
println!();
|
||||
println!("Check system readiness: permissions, platform support, vision config.");
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
let (opts, _) = parse_opts(args)?;
|
||||
init_quiet_logging(opts.verbose);
|
||||
|
||||
let rt = tokio::runtime::Builder::new_multi_thread()
|
||||
.enable_all()
|
||||
.build()?;
|
||||
|
||||
rt.block_on(async {
|
||||
let _engine = bootstrap_engine(opts.verbose).await?;
|
||||
|
||||
let doctor_json =
|
||||
crate::openhuman::screen_intelligence::rpc::accessibility_doctor_cli_json()
|
||||
.await
|
||||
.map_err(|e| anyhow::anyhow!("doctor check failed: {e}"))?;
|
||||
|
||||
let summary = &doctor_json["result"]["summary"];
|
||||
let recommendations = &doctor_json["result"]["recommendations"];
|
||||
let permissions = &doctor_json["result"]["permissions"];
|
||||
|
||||
eprintln!(" Screen Intelligence Doctor");
|
||||
eprintln!(" ──────────────────────────");
|
||||
eprintln!();
|
||||
|
||||
let check = |ok: bool| if ok { "✓" } else { "✗" };
|
||||
|
||||
let platform_ok = summary["platform_supported"].as_bool().unwrap_or(false);
|
||||
let screen_ok = summary["screen_capture_ready"].as_bool().unwrap_or(false);
|
||||
let control_ok = summary["accessibility_ready"].as_bool().unwrap_or(false);
|
||||
let input_ok = summary["input_monitoring_ready"].as_bool().unwrap_or(false);
|
||||
let overall_ok = summary["overall_ready"].as_bool().unwrap_or(false);
|
||||
|
||||
eprintln!(" {} Platform supported", check(platform_ok));
|
||||
eprintln!(" {} Screen recording", check(screen_ok));
|
||||
eprintln!(" {} Accessibility automation", check(control_ok));
|
||||
eprintln!(" {} Input monitoring", check(input_ok));
|
||||
eprintln!();
|
||||
|
||||
// Vision config check.
|
||||
let config = crate::openhuman::config::Config::load_or_init().await.ok();
|
||||
if let Some(ref cfg) = config {
|
||||
let si = &cfg.screen_intelligence;
|
||||
let la = &cfg.local_ai;
|
||||
eprintln!(" Config:");
|
||||
eprintln!(" enabled: {}", si.enabled);
|
||||
eprintln!(" vision_enabled: {}", si.vision_enabled);
|
||||
eprintln!(" use_vision_model: {}", si.use_vision_model);
|
||||
eprintln!(" baseline_fps: {}", si.baseline_fps);
|
||||
eprintln!(" keep_screenshots: {}", si.keep_screenshots);
|
||||
eprintln!(" local_ai.enabled: {}", la.enabled);
|
||||
eprintln!(" local_ai.provider: {}", la.provider);
|
||||
if si.vision_enabled && !la.enabled {
|
||||
eprintln!(" ⚠ Vision is enabled but local_ai.enabled=false — vision analysis will fail");
|
||||
}
|
||||
}
|
||||
|
||||
eprintln!();
|
||||
if overall_ok {
|
||||
eprintln!(" ✓ Overall: READY");
|
||||
} else {
|
||||
eprintln!(" ✗ Overall: NOT READY");
|
||||
eprintln!();
|
||||
eprintln!(" Recommendations:");
|
||||
if let Some(recs) = recommendations.as_array() {
|
||||
for rec in recs {
|
||||
if let Some(s) = rec.as_str() {
|
||||
eprintln!(" • {s}");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
eprintln!();
|
||||
|
||||
// Machine-readable JSON output.
|
||||
println!(
|
||||
"{}",
|
||||
serde_json::to_string_pretty(&serde_json::json!({
|
||||
"summary": summary,
|
||||
"permissions": permissions,
|
||||
"config": config.as_ref().map(|c| serde_json::json!({
|
||||
"enabled": c.screen_intelligence.enabled,
|
||||
"vision_enabled": c.screen_intelligence.vision_enabled,
|
||||
"use_vision_model": c.screen_intelligence.use_vision_model,
|
||||
"baseline_fps": c.screen_intelligence.baseline_fps,
|
||||
"keep_screenshots": c.screen_intelligence.keep_screenshots,
|
||||
"local_ai_enabled": c.local_ai.enabled,
|
||||
"local_ai_provider": c.local_ai.provider,
|
||||
})),
|
||||
}))
|
||||
.unwrap_or_default()
|
||||
);
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,204 @@
|
||||
//! `openhuman screen-intelligence` — standalone CLI for the screen intelligence loop.
|
||||
//!
|
||||
//! Boots **only** the screen intelligence engine (accessibility capture + local-AI
|
||||
//! vision) without the full desktop app, Socket.IO, or skills runtime. Useful for
|
||||
//! testing the capture → save → vision-analysis pipeline from a terminal.
|
||||
//!
|
||||
//! Usage:
|
||||
//! openhuman screen-intelligence run [--ttl <secs>] [--keep] [-v]
|
||||
//! openhuman screen-intelligence status [-v]
|
||||
//! openhuman screen-intelligence capture [--keep] [-v]
|
||||
//! openhuman screen-intelligence start [--ttl <secs>] [-v]
|
||||
//! openhuman screen-intelligence stop [-v]
|
||||
//! openhuman screen-intelligence doctor [-v]
|
||||
//! openhuman screen-intelligence vision [--limit <n>] [-v]
|
||||
|
||||
use anyhow::Result;
|
||||
use std::sync::Arc;
|
||||
|
||||
mod capture;
|
||||
mod doctor;
|
||||
mod server;
|
||||
mod session;
|
||||
|
||||
/// Entry point for `openhuman screen-intelligence <subcommand>`.
|
||||
pub(crate) fn run_screen_intelligence_command(args: &[String]) -> Result<()> {
|
||||
if args.is_empty() || is_help(&args[0]) {
|
||||
print_help();
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
match args[0].as_str() {
|
||||
"run" => server::run_server(&args[1..]),
|
||||
"status" => session::run_status(&args[1..]),
|
||||
"capture" => capture::run_capture(&args[1..]),
|
||||
"start" => session::run_start_session(&args[1..]),
|
||||
"stop" => session::run_stop_session(&args[1..]),
|
||||
"doctor" => doctor::run_doctor(&args[1..]),
|
||||
"vision" => capture::run_vision(&args[1..]),
|
||||
other => Err(anyhow::anyhow!(
|
||||
"unknown screen-intelligence subcommand '{other}'. Run `openhuman screen-intelligence --help`."
|
||||
)),
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Shared helpers (visible to sibling subcommand modules)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
pub(super) struct CliOpts {
|
||||
pub verbose: bool,
|
||||
pub ttl_secs: u64,
|
||||
pub keep: bool,
|
||||
pub limit: usize,
|
||||
pub no_vision_model: bool,
|
||||
}
|
||||
|
||||
pub(super) fn parse_opts(args: &[String]) -> Result<(CliOpts, Vec<String>)> {
|
||||
let mut verbose = false;
|
||||
let mut ttl_secs: u64 = 300;
|
||||
let mut keep = false;
|
||||
let mut limit: usize = 10;
|
||||
let mut no_vision_model = false;
|
||||
let mut rest = Vec::new();
|
||||
let mut i = 0;
|
||||
|
||||
while i < args.len() {
|
||||
match args[i].as_str() {
|
||||
"--no-vision-model" | "--ocr-only" => {
|
||||
no_vision_model = true;
|
||||
i += 1;
|
||||
}
|
||||
"--ttl" => {
|
||||
let val = args
|
||||
.get(i + 1)
|
||||
.ok_or_else(|| anyhow::anyhow!("missing value for --ttl"))?;
|
||||
ttl_secs = val
|
||||
.parse()
|
||||
.map_err(|e| anyhow::anyhow!("invalid --ttl: {e}"))?;
|
||||
i += 2;
|
||||
}
|
||||
"--limit" => {
|
||||
let val = args
|
||||
.get(i + 1)
|
||||
.ok_or_else(|| anyhow::anyhow!("missing value for --limit"))?;
|
||||
limit = val
|
||||
.parse()
|
||||
.map_err(|e| anyhow::anyhow!("invalid --limit: {e}"))?;
|
||||
i += 2;
|
||||
}
|
||||
"--keep" => {
|
||||
keep = true;
|
||||
i += 1;
|
||||
}
|
||||
"-v" | "--verbose" => {
|
||||
verbose = true;
|
||||
i += 1;
|
||||
}
|
||||
"-h" | "--help" => {
|
||||
rest.push(args[i].clone());
|
||||
i += 1;
|
||||
}
|
||||
_ => {
|
||||
rest.push(args[i].clone());
|
||||
i += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Ok((
|
||||
CliOpts {
|
||||
verbose,
|
||||
ttl_secs,
|
||||
keep,
|
||||
limit,
|
||||
no_vision_model,
|
||||
},
|
||||
rest,
|
||||
))
|
||||
}
|
||||
|
||||
/// Bootstrap the screen intelligence engine with config.
|
||||
pub(super) async fn bootstrap_engine(
|
||||
verbose: bool,
|
||||
) -> Result<Arc<crate::openhuman::screen_intelligence::AccessibilityEngine>> {
|
||||
bootstrap_engine_with_opts(verbose, false).await
|
||||
}
|
||||
|
||||
/// Bootstrap with CLI overrides.
|
||||
pub(super) async fn bootstrap_engine_with_opts(
|
||||
verbose: bool,
|
||||
no_vision_model: bool,
|
||||
) -> Result<Arc<crate::openhuman::screen_intelligence::AccessibilityEngine>> {
|
||||
use crate::openhuman::config::Config;
|
||||
use crate::openhuman::screen_intelligence::global_engine;
|
||||
|
||||
let mut config = Config::load_or_init()
|
||||
.await
|
||||
.map_err(|e| anyhow::anyhow!("config load failed: {e}"))?;
|
||||
|
||||
if no_vision_model {
|
||||
config.screen_intelligence.use_vision_model = false;
|
||||
}
|
||||
|
||||
let engine = global_engine();
|
||||
let _ = engine
|
||||
.apply_config(config.screen_intelligence.clone())
|
||||
.await;
|
||||
|
||||
if verbose {
|
||||
log::info!(
|
||||
"[screen-intelligence-cli] engine initialized, enabled={}, vision={}, use_vision_model={}, keep_screenshots={}, workspace={}",
|
||||
config.screen_intelligence.enabled,
|
||||
config.screen_intelligence.vision_enabled,
|
||||
config.screen_intelligence.use_vision_model,
|
||||
config.screen_intelligence.keep_screenshots,
|
||||
config.workspace_dir.display(),
|
||||
);
|
||||
}
|
||||
|
||||
Ok(engine)
|
||||
}
|
||||
|
||||
/// Quiet logging: only `warn` unless verbose (used for non-server subcommands).
|
||||
pub(super) fn init_quiet_logging(verbose: bool) {
|
||||
if !verbose && std::env::var_os("RUST_LOG").is_none() {
|
||||
std::env::set_var("RUST_LOG", "warn");
|
||||
}
|
||||
crate::core::logging::init_for_cli_run(verbose, crate::core::logging::CliLogDefault::Global);
|
||||
}
|
||||
|
||||
pub(super) fn is_help(value: &str) -> bool {
|
||||
matches!(value, "-h" | "--help" | "help")
|
||||
}
|
||||
|
||||
fn print_help() {
|
||||
println!("openhuman screen-intelligence — standalone screen intelligence runtime\n");
|
||||
println!("Boots only the screen intelligence engine (accessibility capture + local-AI");
|
||||
println!("vision) without the full desktop app, Socket.IO, or skills runtime.\n");
|
||||
println!("Usage:");
|
||||
println!(" openhuman screen-intelligence run [--ttl <secs>] [--no-vision-model] [-v]");
|
||||
println!(" openhuman screen-intelligence status [-v]");
|
||||
println!(" openhuman screen-intelligence capture [--keep] [-v]");
|
||||
println!(" openhuman screen-intelligence start [--ttl <secs>] [--no-vision-model] [-v]");
|
||||
println!(" openhuman screen-intelligence stop [-v]");
|
||||
println!(" openhuman screen-intelligence doctor [-v]");
|
||||
println!(" openhuman screen-intelligence vision [--limit <n>] [-v]");
|
||||
println!();
|
||||
println!("Subcommands:");
|
||||
println!(" run Start the capture → vision → log loop (blocks until TTL/Ctrl+C)");
|
||||
println!(" status Print current engine status (permissions, session, config)");
|
||||
println!(" capture Take a single screenshot and print diagnostics");
|
||||
println!(" start Start a capture + vision session (runs until TTL or Ctrl+C)");
|
||||
println!(" stop Stop the active session");
|
||||
println!(" doctor Check system readiness (permissions, vision config, platform)");
|
||||
println!(" vision Print recent vision summaries from the active session");
|
||||
println!();
|
||||
println!("Common options:");
|
||||
println!(" --ttl <secs> Session TTL (default: 300)");
|
||||
println!(" --limit <n> Max vision summaries for 'vision' (default: 10)");
|
||||
println!(" --keep Save screenshot to disk (for 'capture')");
|
||||
println!(" --no-vision-model Skip vision LLM — use OCR + text LLM only");
|
||||
println!(" --ocr-only Alias for --no-vision-model");
|
||||
println!(" -v, --verbose Enable debug logging");
|
||||
}
|
||||
@@ -0,0 +1,78 @@
|
||||
//! `openhuman screen-intelligence run` — start the standalone capture + vision loop.
|
||||
|
||||
use anyhow::Result;
|
||||
|
||||
use super::{is_help, parse_opts};
|
||||
|
||||
/// Delegates to [`crate::openhuman::screen_intelligence::server::run_standalone`],
|
||||
/// which boots the engine, starts a capture session, and blocks in a
|
||||
/// monitoring loop logging captures and vision summaries until TTL or Ctrl+C.
|
||||
pub(super) fn run_server(args: &[String]) -> Result<()> {
|
||||
let (opts, rest) = parse_opts(args)?;
|
||||
|
||||
if rest.iter().any(|a| is_help(a)) {
|
||||
println!("Usage: openhuman screen-intelligence run [--ttl <secs>] [--keep] [--no-vision-model] [-v]");
|
||||
println!();
|
||||
println!("Start the screen intelligence capture + vision loop.");
|
||||
println!("Captures screenshots at baseline FPS, runs OCR and vision analysis,");
|
||||
println!("and logs summaries. Blocks until TTL expires or Ctrl+C.");
|
||||
println!();
|
||||
println!(" --ttl <secs> Session duration (default: 300)");
|
||||
println!(" --keep Keep screenshots on disk after vision processing");
|
||||
println!(" --no-vision-model Skip the vision LLM — use OCR + text LLM only");
|
||||
println!(" --ocr-only Alias for --no-vision-model");
|
||||
println!(" -v, --verbose Enable debug logging");
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
crate::core::logging::init_for_cli_run(
|
||||
opts.verbose,
|
||||
crate::core::logging::CliLogDefault::Global,
|
||||
);
|
||||
|
||||
let rt = tokio::runtime::Builder::new_multi_thread()
|
||||
.enable_all()
|
||||
.build()?;
|
||||
|
||||
rt.block_on(async {
|
||||
let mut config = crate::openhuman::config::Config::load_or_init()
|
||||
.await
|
||||
.map_err(|e| anyhow::anyhow!("config load failed: {e}"))?;
|
||||
|
||||
if opts.no_vision_model {
|
||||
config.screen_intelligence.use_vision_model = false;
|
||||
}
|
||||
|
||||
let keep_screenshots = opts.keep || config.screen_intelligence.keep_screenshots;
|
||||
|
||||
let server_config = crate::openhuman::screen_intelligence::server::SiServerConfig {
|
||||
ttl_secs: opts.ttl_secs,
|
||||
log_interval_secs: 5,
|
||||
keep_screenshots,
|
||||
};
|
||||
|
||||
let mode_label = if config.screen_intelligence.use_vision_model {
|
||||
format!("vision LLM ({})", config.local_ai.vision_model_id)
|
||||
} else {
|
||||
"OCR + text LLM (no vision model)".to_string()
|
||||
};
|
||||
|
||||
eprintln!();
|
||||
eprintln!(" Screen Intelligence");
|
||||
eprintln!(" ───────────────────");
|
||||
eprintln!(" TTL: {}s", opts.ttl_secs);
|
||||
eprintln!(" Mode: {}", mode_label);
|
||||
eprintln!(
|
||||
" FPS: {}",
|
||||
config.screen_intelligence.baseline_fps
|
||||
);
|
||||
eprintln!(" Keep screenshots: {}", keep_screenshots);
|
||||
eprintln!();
|
||||
eprintln!(" Capturing → Vision → Log. Press Ctrl+C to stop.");
|
||||
eprintln!();
|
||||
|
||||
crate::openhuman::screen_intelligence::server::run_standalone(config, server_config)
|
||||
.await
|
||||
.map_err(|e| anyhow::anyhow!("{e}"))
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,152 @@
|
||||
//! Session lifecycle subcommands: `start`, `stop`, `status`.
|
||||
|
||||
use anyhow::Result;
|
||||
|
||||
use super::{
|
||||
bootstrap_engine, bootstrap_engine_with_opts, init_quiet_logging, is_help, parse_opts,
|
||||
};
|
||||
|
||||
/// `openhuman screen-intelligence status` — print current engine status as JSON.
|
||||
pub(super) fn run_status(args: &[String]) -> Result<()> {
|
||||
if args.iter().any(|a| is_help(a)) {
|
||||
println!("Usage: openhuman screen-intelligence status [-v]");
|
||||
println!();
|
||||
println!("Print current screen intelligence engine status (permissions, session, config).");
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
let (opts, _) = parse_opts(args)?;
|
||||
init_quiet_logging(opts.verbose);
|
||||
|
||||
let rt = tokio::runtime::Builder::new_multi_thread()
|
||||
.enable_all()
|
||||
.build()?;
|
||||
|
||||
rt.block_on(async {
|
||||
let engine = bootstrap_engine(opts.verbose).await?;
|
||||
let status = engine.status().await;
|
||||
println!(
|
||||
"{}",
|
||||
serde_json::to_string_pretty(&status).unwrap_or_else(|_| format!("{:?}", status))
|
||||
);
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
/// `openhuman screen-intelligence start` — start a capture + vision session.
|
||||
pub(super) fn run_start_session(args: &[String]) -> Result<()> {
|
||||
if args.iter().any(|a| is_help(a)) {
|
||||
println!(
|
||||
"Usage: openhuman screen-intelligence start [--ttl <secs>] [--no-vision-model] [-v]"
|
||||
);
|
||||
println!();
|
||||
println!("Start a screen intelligence capture session with vision analysis.");
|
||||
println!("The session runs until TTL expires or Ctrl+C is pressed.");
|
||||
println!();
|
||||
println!(" --ttl <secs> Session duration (default: 300, max: 3600)");
|
||||
println!(" --no-vision-model Skip the vision LLM — use OCR + text LLM only");
|
||||
println!(" --ocr-only Alias for --no-vision-model");
|
||||
println!(" -v, --verbose Enable debug logging");
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
let (opts, _) = parse_opts(args)?;
|
||||
crate::core::logging::init_for_cli_run(
|
||||
opts.verbose,
|
||||
crate::core::logging::CliLogDefault::Global,
|
||||
);
|
||||
|
||||
let rt = tokio::runtime::Builder::new_multi_thread()
|
||||
.enable_all()
|
||||
.build()?;
|
||||
|
||||
rt.block_on(async {
|
||||
let engine = bootstrap_engine_with_opts(opts.verbose, opts.no_vision_model).await?;
|
||||
|
||||
let params = crate::openhuman::screen_intelligence::StartSessionParams {
|
||||
consent: true,
|
||||
ttl_secs: Some(opts.ttl_secs),
|
||||
screen_monitoring: Some(true),
|
||||
};
|
||||
|
||||
match engine.start_session(params).await {
|
||||
Ok(session) => {
|
||||
eprintln!(" Session started!");
|
||||
eprintln!(" TTL: {}s", session.ttl_secs);
|
||||
eprintln!(" Vision: {}", session.vision_enabled);
|
||||
eprintln!(" Panic hotkey: {}", session.panic_hotkey);
|
||||
eprintln!();
|
||||
eprintln!(" Capturing screenshots and running vision analysis...");
|
||||
eprintln!(" Press Ctrl+C to stop.");
|
||||
eprintln!();
|
||||
|
||||
// Print periodic status updates until the session ends.
|
||||
let mut tick = tokio::time::interval(std::time::Duration::from_secs(5));
|
||||
loop {
|
||||
tick.tick().await;
|
||||
let status = engine.status().await;
|
||||
if !status.session.active {
|
||||
eprintln!(
|
||||
"\n Session ended: {}",
|
||||
status
|
||||
.session
|
||||
.stop_reason
|
||||
.unwrap_or_else(|| "unknown".into())
|
||||
);
|
||||
break;
|
||||
}
|
||||
eprintln!(
|
||||
" [{}] captures={} vision={} queue={} last_app={:?}",
|
||||
chrono::Utc::now().format("%H:%M:%S"),
|
||||
status.session.capture_count,
|
||||
status.session.vision_state,
|
||||
status.session.vision_queue_depth,
|
||||
status.session.last_context.as_deref().unwrap_or("-"),
|
||||
);
|
||||
if let Some(summary) = &status.session.last_vision_summary {
|
||||
let truncated = if summary.chars().count() > 100 {
|
||||
format!("{}…", summary.chars().take(100).collect::<String>())
|
||||
} else {
|
||||
summary.clone()
|
||||
};
|
||||
eprintln!(" notes: {truncated}");
|
||||
}
|
||||
}
|
||||
}
|
||||
Err(e) => {
|
||||
eprintln!(" Failed to start session: {e}");
|
||||
std::process::exit(1);
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
/// `openhuman screen-intelligence stop` — stop an active session.
|
||||
pub(super) fn run_stop_session(args: &[String]) -> Result<()> {
|
||||
if args.iter().any(|a| is_help(a)) {
|
||||
println!("Usage: openhuman screen-intelligence stop [-v]");
|
||||
println!();
|
||||
println!("Stop the active screen intelligence session.");
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
let (opts, _) = parse_opts(args)?;
|
||||
init_quiet_logging(opts.verbose);
|
||||
|
||||
let rt = tokio::runtime::Builder::new_multi_thread()
|
||||
.enable_all()
|
||||
.build()?;
|
||||
|
||||
rt.block_on(async {
|
||||
let engine = bootstrap_engine(opts.verbose).await?;
|
||||
let session = engine.stop_session(Some("cli_stop".to_string())).await;
|
||||
eprintln!(
|
||||
" Session stopped: {}",
|
||||
session
|
||||
.stop_reason
|
||||
.unwrap_or_else(|| "no active session".into())
|
||||
);
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
@@ -478,6 +478,27 @@ impl AccessibilityEngine {
|
||||
}
|
||||
}
|
||||
|
||||
/// Save the image payload from a [`CaptureTestResult`] to disk, reconstructing
|
||||
/// the minimum [`CaptureFrame`] needed by [`save_screenshot_to_disk`].
|
||||
///
|
||||
/// Returns `None` when the result carries no `image_ref` (nothing to save).
|
||||
/// Callers supply a `reason` string to label the frame (e.g. `"cli_capture"`).
|
||||
pub(crate) fn save_capture_test_result(
|
||||
workspace_dir: &std::path::Path,
|
||||
result: &CaptureTestResult,
|
||||
reason: &str,
|
||||
) -> Option<Result<PathBuf, String>> {
|
||||
let image_ref = result.image_ref.as_ref()?.clone();
|
||||
let frame = CaptureFrame {
|
||||
captured_at_ms: chrono::Utc::now().timestamp_millis(),
|
||||
reason: reason.to_string(),
|
||||
app_name: result.context.as_ref().and_then(|c| c.app_name.clone()),
|
||||
window_title: result.context.as_ref().and_then(|c| c.window_title.clone()),
|
||||
image_ref: Some(image_ref),
|
||||
};
|
||||
Some(Self::save_screenshot_to_disk(workspace_dir, &frame))
|
||||
}
|
||||
|
||||
/// Save a screenshot PNG to `{workspace_dir}/screenshots/{timestamp}_{app}.png`.
|
||||
pub fn save_screenshot_to_disk(
|
||||
workspace_dir: &std::path::Path,
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
//! Screen capture, accessibility automation, and vision summaries (macOS-focused).
|
||||
|
||||
pub(crate) mod cli;
|
||||
pub mod ops;
|
||||
mod schemas;
|
||||
pub mod server;
|
||||
|
||||
@@ -14,7 +14,7 @@
|
||||
use anyhow::Result;
|
||||
|
||||
/// Entry point for `openhuman text-input <subcommand>`.
|
||||
pub fn run_text_input_command(args: &[String]) -> Result<()> {
|
||||
pub(crate) fn run_text_input_command(args: &[String]) -> Result<()> {
|
||||
if args.is_empty() || is_help(&args[0]) {
|
||||
print_help();
|
||||
return Ok(());
|
||||
@@ -4,6 +4,7 @@
|
||||
//! Thin orchestration layer consumed by autocomplete, voice control, and other
|
||||
//! text-aware features. All platform work delegates to `accessibility::*`.
|
||||
|
||||
pub(crate) mod cli;
|
||||
pub mod ops;
|
||||
mod schemas;
|
||||
mod types;
|
||||
|
||||
@@ -13,7 +13,7 @@
|
||||
use anyhow::Result;
|
||||
|
||||
/// Entry point for `openhuman tree-summarizer <subcommand>`.
|
||||
pub fn run_tree_summarizer_command(args: &[String]) -> Result<()> {
|
||||
pub(crate) fn run_tree_summarizer_command(args: &[String]) -> Result<()> {
|
||||
if args.is_empty() || is_help(&args[0]) {
|
||||
print_help();
|
||||
return Ok(());
|
||||
@@ -6,6 +6,7 @@
|
||||
//! Stored as markdown files in `memory/namespaces/{ns}/tree/`.
|
||||
|
||||
pub mod bus;
|
||||
pub(crate) mod cli;
|
||||
pub mod engine;
|
||||
pub mod ops;
|
||||
pub mod store;
|
||||
|
||||
@@ -0,0 +1,122 @@
|
||||
//! Voice CLI adapter — domain-owned.
|
||||
//!
|
||||
//! Handles the `openhuman voice` / `openhuman dictate` subcommand which runs a
|
||||
//! long-lived, blocking standalone dictation server (hotkey → record →
|
||||
//! transcribe → insert). This flow doesn't fit the request/response controller
|
||||
//! registry pattern because it blocks forever on the hotkey listener, so the
|
||||
//! adapter lives here inside the voice domain rather than in `src/core/cli.rs`.
|
||||
|
||||
use anyhow::{anyhow, Result};
|
||||
|
||||
use crate::core::logging::{init_for_cli_run, CliLogDefault};
|
||||
use crate::openhuman::voice::hotkey::ActivationMode;
|
||||
use crate::openhuman::voice::server::{run_standalone, VoiceServerConfig};
|
||||
|
||||
/// Parse and execute the `openhuman voice` / `openhuman dictate` subcommand.
|
||||
///
|
||||
/// Supported flags:
|
||||
/// --hotkey <combo> Key combination (default from config, usually `fn`)
|
||||
/// --mode <tap|push> Activation mode (default push)
|
||||
/// --skip-cleanup Skip LLM post-processing on transcriptions
|
||||
/// -v / --verbose Enable debug logging
|
||||
/// -h / --help Print usage
|
||||
pub(crate) fn run_standalone_subcommand(args: &[String]) -> Result<()> {
|
||||
let mut hotkey: Option<String> = None;
|
||||
let mut mode: Option<String> = None;
|
||||
let mut skip_cleanup = false;
|
||||
let mut verbose = false;
|
||||
let mut i = 0usize;
|
||||
|
||||
while i < args.len() {
|
||||
match args[i].as_str() {
|
||||
"--hotkey" => {
|
||||
hotkey = Some(
|
||||
args.get(i + 1)
|
||||
.ok_or_else(|| anyhow!("missing value for --hotkey"))?
|
||||
.clone(),
|
||||
);
|
||||
i += 2;
|
||||
}
|
||||
"--mode" => {
|
||||
mode = Some(
|
||||
args.get(i + 1)
|
||||
.ok_or_else(|| anyhow!("missing value for --mode"))?
|
||||
.clone(),
|
||||
);
|
||||
i += 2;
|
||||
}
|
||||
"--skip-cleanup" => {
|
||||
skip_cleanup = true;
|
||||
i += 1;
|
||||
}
|
||||
"-v" | "--verbose" => {
|
||||
verbose = true;
|
||||
i += 1;
|
||||
}
|
||||
"-h" | "--help" => {
|
||||
print_help();
|
||||
return Ok(());
|
||||
}
|
||||
other => return Err(anyhow!("unknown voice arg: {other}")),
|
||||
}
|
||||
}
|
||||
|
||||
log::debug!(
|
||||
"[voice-cli] starting standalone server hotkey={:?} mode={:?} skip_cleanup={} verbose={}",
|
||||
hotkey,
|
||||
mode,
|
||||
skip_cleanup,
|
||||
verbose
|
||||
);
|
||||
|
||||
init_for_cli_run(verbose, CliLogDefault::Global);
|
||||
|
||||
let rt = tokio::runtime::Builder::new_multi_thread()
|
||||
.enable_all()
|
||||
.build()?;
|
||||
|
||||
rt.block_on(async {
|
||||
let mut config = match crate::openhuman::config::Config::load_or_init().await {
|
||||
Ok(cfg) => cfg,
|
||||
Err(e) => {
|
||||
log::warn!("[voice-cli] config load failed, using defaults: {e}");
|
||||
crate::openhuman::config::Config::default()
|
||||
}
|
||||
};
|
||||
config.apply_env_overrides();
|
||||
|
||||
let activation_mode = match mode.as_deref() {
|
||||
Some("tap") => ActivationMode::Tap,
|
||||
Some("push") | None => ActivationMode::Push,
|
||||
Some(other) => return Err(anyhow!("invalid --mode '{other}', expected tap|push")),
|
||||
};
|
||||
|
||||
let server_config = VoiceServerConfig {
|
||||
hotkey: hotkey.unwrap_or_else(|| config.voice_server.hotkey.clone()),
|
||||
activation_mode,
|
||||
skip_cleanup,
|
||||
context: None,
|
||||
min_duration_secs: config.voice_server.min_duration_secs,
|
||||
silence_threshold: config.voice_server.silence_threshold,
|
||||
custom_dictionary: config.voice_server.custom_dictionary.clone(),
|
||||
};
|
||||
|
||||
run_standalone(config, server_config)
|
||||
.await
|
||||
.map_err(anyhow::Error::msg)
|
||||
})?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn print_help() {
|
||||
println!("Usage: openhuman voice [--hotkey <combo>] [--mode <tap|push>] [--skip-cleanup] [-v]");
|
||||
println!();
|
||||
println!(" --hotkey <combo> Key combination (default: fn)");
|
||||
println!(" --mode <tap|push> Activation: tap to toggle, push to hold (default: push)");
|
||||
println!(" --skip-cleanup Skip LLM post-processing on transcriptions");
|
||||
println!(" -v, --verbose Enable debug logging");
|
||||
println!();
|
||||
println!("Standalone voice dictation server. Press the hotkey to dictate,");
|
||||
println!("transcribed text is inserted into the active text field.");
|
||||
}
|
||||
@@ -5,6 +5,7 @@
|
||||
//! standalone voice dictation server (hotkey → record → transcribe → insert).
|
||||
|
||||
pub mod audio_capture;
|
||||
pub(crate) mod cli;
|
||||
pub mod dictation_listener;
|
||||
pub mod hallucination;
|
||||
pub mod hotkey;
|
||||
|
||||
Reference in New Issue
Block a user