mirror of
https://github.com/tinyhumansai/openhuman.git
synced 2026-07-28 13:32:23 +00:00
df800dcdb87da744762f842e03bf6991cb062fcf
67
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 CodeGhost21andGitHub
|
cd7a56581c | fix(sentry): Rust source context + per-release deploy marker (#405) (#1067) | ||
|
Steven EnamakelandGitHub
|
9dd878cbfb | feat(scheduler-gate): throttle background AI on battery / busy CPU (#1062) | ||
|
|
800fafb080 |
feat(morning_briefing): inject ambient runtime + user + datetime into system prompt (#959)
Co-authored-by: Steven Enamakel <31011319+senamakel@users.noreply.github.com> |
||
|
|
fd0398d5fa |
feat(memory): MD-on-disk content + participant bucketing + async pipeline foundation (#1008)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai> |
||
|
|
4f9da9eb8d |
test(foundation): coverage matrix + strategy + gap-fill batch (#773) (#980)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
|
e3c46d1c3d |
fix(whatsapp): upgrade WhatsApp Web channel to upstream whatsapp-rust 0.5 (#916)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Steven EnamakelandGitHub
|
7867496d86 | feat(home): banners, welcome typewriter, conversation gating (#936) | ||
|
|
e0e7e1bd77 |
refactor: extract inline tests to sibling _tests.rs files (130 god-files) (#835)
Co-authored-by: Jwalin Shah <jshah1331@gmail.com> Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai> |
||
|
Steven EnamakelandGitHub
|
13242cc9d6 | feat(integrations): stock-price tools and expanded Parallel surface (#911) | ||
|
Steven EnamakelandGitHub
|
c191a1f0aa | fix(core-update): support versioned tarball/zip release assets (#908) | ||
|
CodeGhost21andGitHub
|
b3b451fd28 |
fix(composio): keep chat integrations cache in sync on Windows (#749)
On Windows the `ComposioConnectionCreated` → `wait_for_connection_active` invalidation path often misses: the 60 s readiness poll can expire before the user finishes the hosted OAuth flow (Defender SmartScreen, slower browser launch, extra consent dialogs), so `invalidate_connected_integrations_cache` never fires and the chat runtime stays frozen on the pre-connect snapshot while Settings correctly shows "Connected" via its 5 s `listConnections` poll. Layer two cross-platform defenses on top of the existing event-bus path: 1. `composio_list_connections` (the RPC the UI polls every 5 s) now diffs the backend's active-toolkit set against every cached entry and invalidates on divergence, so chat converges to truth within one poll interval — regardless of how the OAuth round-trip completed. 2. Add a 60 s TTL on the `INTEGRATIONS_CACHE` so stale entries can't live forever even if nothing polls. 3. Grep-friendly `[composio][integrations]` logs at every decision point (cache hit / miss / expiry / divergence diff) for quick diagnosis from user-supplied debug dumps. Also covers the regression with six new unit tests (activate, disconnect, steady state, non-active rows, CONNECTED vs ACTIVE alias, TTL expiry) serialized via a shared test mutex so they don't race the existing mock-backend tests over the process-wide cache. |
||
|
|
5fff5568d3 |
feat(memory): Phase 2 memory tree - preprocessing, scoring, admission gate (#708) (#733)
* feat(memory): phase 1 memory tree - multi-source ingestion & canonical chunks (#707) Adds an isolated memory tree layer under src/openhuman/memory/tree/ implementing Phase 1 of the new memory architecture (umbrella #711). Zero edits to existing memory/*.rs files - the new layer coexists with the legacy TinyHumans-backed client. - Source adapters: chat / email / document -> canonical Markdown - Token-bounded chunker with deterministic SHA-256 chunk IDs - SQLite persistence at <workspace>/memory_tree/chunks.db with full provenance metadata (source_kind, source_id, owner, timestamps, tags, time_range) and back-pointer to raw source - Unified JSON-RPC ingest (dispatches on source_kind + JSON payload): openhuman.memory_tree_ingest, _list_chunks, _get_chunk - DataSource enum covering the 8 providers from m.excalidraw step 1 (Discord/Telegram/Whatsapp/Gmail/OtherEmail/Notion/MeetingNotes/DriveDocs) - ~40 unit tests (chunk ID stability, UTF-8-safe splitting, canonicalisation idempotence, store round-trip, filter behavior) Additive only: new tables in a new DB file, new JSON-RPC namespace, no existing behavior changes. Feeds #708 (scoring), #709 (summary trees), #710 (query tools). Closes #707. Parent: #711. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(memory): phase 2 memory tree - preprocessing, scoring, admission gate (#708) Adds the scoring / admission layer between Phase 1's chunker and store. Stacked on feat/707-memory-ingestion (PR #732) - depends on Phase 1's chunk substrate. - Pluggable EntityExtractor trait + CompositeExtractor chain - RegexEntityExtractor: mechanical entities (emails, URLs, @handles, #hashtags) - Always on, deterministic, zero deps, UTF-8-safe char spans - Five weighted signals: token count, unique-word ratio, metadata weight, source weight (per-DataSource), interaction (reply/sent/mention/dm tags), entity density - Exact-match entity canonicalisation (email lowercased, @ and # stripped) - Admission gate drops chunks below configurable threshold (default 0.3) - Score rationale persists for EVERY chunk (kept or dropped) for debugging - Entities indexed for KEPT chunks only - Two new SQLite tables added to the memory_tree DB: - mem_tree_score: per-chunk score rationale with all signal values - mem_tree_entity_index: inverted index entity_id -> node_id - Idempotent ALTER TABLE migration adds embedding BLOB column to mem_tree_chunks (used in Phase 3 retrieval, wired but not populated here) - Ingest pipeline converted to async to accommodate the extractor trait; blocking SQLite work isolated on spawn_blocking; JSON-RPC surface unchanged (same memory_tree_ingest / list / get methods) - Phase 2 deliberately ships without GLiNER/semantic NER - per-chunk semantic entities land later behind a cargo feature flag; the composite extractor interface keeps that drop-in trivial Additive only: new tables, new columns, new module. Existing Phase 1 behavior unchanged except that low-signal chunks are now dropped before reaching mem_tree_chunks. Raise score_drop_threshold to 0 to disable the gate and restore Phase-1-identical behavior. Closes #708. Parent: #711. Depends on: #707 (#732). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Fix memory tree scoring persistence issues * Fix memory tree scoring robustness issues from PR review - ingest: fail fast if scorer returns fewer/more results than chunks (silent zip truncation would drop chunks or their score rationale) - score::persist_score{,_tx}: clear stale entity-index rows before re-indexing a re-scored chunk, since INSERT OR REPLACE never deletes rows whose entity_id is no longer in the new extraction - score::store::lookup_entity: clamp limit to i64::MAX before casting to prevent a large usize wrapping into a negative LIMIT Adds clear_entity_index_drops_stale_rows regression test. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai> |
||
|
|
3da80d852e |
feat(skills,node): integrate SKILL.md + managed Node runtime (#681) (#723)
* build(skills): add serde_yaml for SKILL.md frontmatter (#681) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(skills): parse SKILL.md frontmatter and add multi-path discovery (#681) Extends the skills module with agentskills.io-style discovery: * Parses YAML frontmatter in SKILL.md (name, description, version, author, tags, allowed-tools, license) via serde_yaml, with a catch-all extras map for forward-compatible keys. * Adds SkillScope (User / Project / Legacy) and discover_skills(), which scans ~/.openhuman/skills, ~/.agents/skills, <ws>/.openhuman/ skills, <ws>/.agents/skills and the legacy <ws>/skills directory. * Gates project-scope loading behind an explicit <ws>/.openhuman/trust marker (is_workspace_trusted helper). * Resolves name collisions with project > user > legacy precedence and surfaces shadowing as per-skill warnings. * Inventories bundled resources under scripts/, references/, assets/. * Keeps the existing load_skills(workspace_dir) API as a backwards-compatible wrapper for existing callers. * Preserves legacy skill.json fallback (marked legacy = true). * Lenient validation: missing / mismatched / oversized name or description produce warnings instead of errors. * Adds 12 unit tests covering frontmatter parsing, trust gating, collision shadowing, lenient fallbacks, legacy JSON and resource inventory. One existing prompt test updated to use ..Default::default() for the expanded struct. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(skills): align frontmatter with agentskills.io spec (#681) Per the agentskills.io SKILL.md spec, only name, description, license, compatibility, metadata, and allowed-tools are valid top-level keys. Our SkillFrontmatter had version, author, and tags as top-level fields, which drifted from the spec and would silently swallow mis-shaped data for skills authored against the canonical schema. Demote version/author/tags into the metadata map. Non-spec top-level keys still parse via #[serde(flatten)] into extra, and when present there we emit a migration warning and still populate the derived Skill fields so existing skills keep working. Add compatibility as an optional string alongside license. New tests cover the spec-compliant metadata shape, the deprecated top-level fallback path, and verify that the full spec frontmatter parses without leaking into extras. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * build(runtime): add tar+xz2+zip for node runtime extraction (#681) Node.js distributions ship as .tar.xz on Unix and .zip on Windows. The upcoming Node runtime bootstrap extracts these in pure Rust; xz2 is built with the `static` feature so liblzma is bundled and not a system dependency. zip is pinned to default-features = false + deflate to avoid pulling in bzip2/zstd/aes which we don't need for Node archives. Deps only — no behavior change. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(config): add NodeConfig with env overrides (#681) Introduce managed Node.js runtime configuration so skills that require `node`/`npm` (agentskills.io packages with build steps) can be wired in behind a single config switch. Fields: - `node.enabled` — master switch (default true) - `node.version` — pinned release (default `v22.11.0` LTS) - `node.cache_dir` — absolute path for managed distributions; empty = use workspace default - `node.prefer_system` — reuse matching system `node` when found (default true); set false for reproducible CI / airgapped deploys Env overrides land in load.rs alongside the existing LOCAL_AI_TIER block: - OPENHUMAN_NODE_ENABLED - OPENHUMAN_NODE_VERSION - OPENHUMAN_NODE_CACHE_DIR - OPENHUMAN_NODE_PREFER_SYSTEM No resolver / downloader yet — subsequent commits in the #681 series consume this config to bootstrap the runtime. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(node_runtime): detect compatible system node on PATH (#681) Introduce the `openhuman::node_runtime` module and its first piece: a synchronous resolver that walks `PATH`, probes `node --version`, and returns a `SystemNode` when the host toolchain major-version matches the configured target. Why resolve first: a successful probe lets the bootstrap skip a ~60 MB download per managed install. Matching is intentionally loose on the patch level (Node LTS lines are ABI-stable and skills pin their own deps via package-lock.json); operators needing strict pinning can set `node.prefer_system = false`. Tracing is verbose by design — resolver decisions gate the download path, so operators need a clear breadcrumb trail at `debug`/`info`. Includes unit tests for `parse_node_version` covering `v` prefix, whitespace, major-only, and malformed inputs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(node_runtime): SHASUMS256-verified distribution downloader (#681) Add the second piece of the managed Node.js runtime: streaming download of OS/arch-specific prebuilt archives from nodejs.org, gated by a SHA-256 match against the release's signed `SHASUMS256.txt`. Key contracts: - `NodeDistribution::for_host(version)` picks the right archive for the current OS/arch tuple. Supported matrix: * darwin-{arm64,x64}.tar.xz * linux-{arm64,x64,armv7l}.tar.xz * win-{arm64,x64}.zip Unsupported hosts surface a clear error so the caller can flip `node.enabled = false` or point at a pre-installed toolchain. - `fetch_shasums(client, version)` parses `SHASUMS256.txt` into a filename -> hex digest map. Tolerant of trailing / signature blocks. - `download_distribution(...)` streams chunks into the target path, computes SHA-256 on the fly, and wipes the partial file if the digest does not match. Integrity check is mandatory — skills will execute untrusted code inside the resolved runtime. Verbose tracing at `info` and `debug` follows the repo debug-logging rule; operators can grep `[node_runtime::downloader]` to trace every GET, chunk boundary (via total-bytes log), and hash decision. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(node_runtime): archive extraction + atomic install (#681) Extract downloaded Node.js distributions and move them into the final cache path without leaving the reader observing a half-populated directory. - `extract_distribution(archive, extract_root, is_zip)` — wraps the synchronous `tar`/`xz2`/`zip` crates in `spawn_blocking` and returns the single top-level folder produced by the archive (`node-vX.Y.Z- <os>-<arch>/`). `set_preserve_permissions(true)` + `set_overwrite( true)` on the tar side keeps the `node` binary's `+x` bit. The zip path restores Unix mode bits via `unix_mode()` so cross-OS builds still land correct permissions. - `atomic_install(staged, final_dest)` — renames a staged directory into place via a single `rename(2)`, moving any pre-existing install to a `.old-<pid>` sibling first and cleaning it up on success. This gives concurrent readers a "before" or "after" view, never a partial one. - Unsafe zip paths (`enclosed_name()` rejection) are logged and skipped rather than traversed — defence against malicious archives, even though the digest check in the downloader already rules out tampered official releases. No new tests here: the logic leans on upstream tar/zip crates, and meaningful end-to-end coverage requires a real archive on disk — that comes in the integration-test commit later in this series. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(node_runtime): bootstrap mutex + cache + bin path helper (#681) Stitch resolver, downloader, and extractor into a single idempotent entry point callers use at startup (or lazily before the first `node_exec`/`npm_exec` call). `NodeBootstrap::resolve()` contract: 1. Return cached `ResolvedNode` if a previous call already succeeded. 2. Bail when `node.enabled = false`. 3. If `node.prefer_system`, probe the host PATH. Matching major version wins — return a `System`-sourced `ResolvedNode`. 4. Otherwise compute the install path (`{cache_dir}/node-v{version}-{os}-{arch}/`). If it already contains valid bins, reuse it. This makes the bootstrap ~free across restarts once a managed install lands. 5. Else fetch `SHASUMS256.txt`, locate the expected digest for our archive, stream the download, extract into a `.stage-<pid>` scratch dir, `atomic_install` the top-level folder into the final path, and remove scratch + archive to reclaim disk. Concurrency is handled by a `tokio::sync::Mutex<Option<ResolvedNode>>` — parallel callers queue behind the first one; the winner memoises, the losers pick up the cached result. No race can produce two concurrent downloads of the same archive. Platform-specific bin layout lives in `managed_bin_dir` / `build_ resolved`: - Unix: `<install>/bin/{node,npm}` - Windows: `<install>/{node.exe,npm.cmd}` This closes the resolver/downloader/cache layer promised in the #681 checkpoint. Tools (`node_exec`, `npm_exec`) and shell-PATH injection layer on top in the next batch of commits. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(tools): node_exec runs JS via managed node runtime (#681) Executes `inline_code` (via `node -e`) or a workspace-relative `script_path` through the NodeBootstrap-resolved `node` binary. POSIX single-quote quoting keeps user input inert; `env_clear` + allow-list mirrors the shell tool so secrets never leak. 300s default timeout (capped at 1800s), 1MB stdout/stderr caps. PATH is prepended with the resolved bin dir so child processes see managed `node`/`npm`. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(tools): npm_exec runs npm via managed node runtime (#681) Thin npm CLI wrapper paired with node_exec. Subcommands go through `is_sane_subcommand` (alphanumerics + `._-:` only) and a deny-list (publish/adduser/login/token/…) blocks registry-mutation and auth flows. `cwd` is resolved under the workspace — absolute paths and `..` components are rejected. 600s default timeout (1800s ceiling), 1MB stdout/stderr caps. PATH prepended with managed bin dir so npm's own node/corepack lookups hit the managed toolchain. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(tools): shell prepends managed node bin to PATH when cached (#681) Adds a non-blocking `NodeBootstrap::try_cached()` primitive that peeks the memoised `ResolvedNode` without holding the async lock or triggering a download. ShellTool gains an optional `node_bootstrap` field wired through a new `with_node_bootstrap` constructor; when set, each shell invocation consults `try_cached()` and, on a hit, prepends the managed bin dir to the child PATH using the platform separator. Unrelated shell commands stay byte-identical — no download is ever forced from the shell path. Consequence: once `node_exec`/`npm_exec` have resolved the toolchain once, skills that shell out to `node`/`npm`/`npx`/`corepack` transparently pick up the managed install without any per-command coordination. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(tools): register node_exec + npm_exec behind node.enabled (#681) Wires the skills-oriented Node.js toolchain into the tool registry: * Constructs one session-scoped `NodeBootstrap` in `all_tools_with_runtime` when `root_config.node.enabled` is true — all three consumers (ShellTool, NodeExecTool, NpmExecTool) share the same `Arc<NodeBootstrap>` so the download/extract/install pipeline runs at most once per session and the memoised `ResolvedNode` is reused across every shelling-out call. * Swaps ShellTool to `with_node_bootstrap(...)` when the bootstrap exists so shell PATH injection fires automatically once any node/npm tool has resolved the runtime. * Registers `node_exec` and `npm_exec` only when the flag is on — flipping `node.enabled = false` cleanly removes both tools and falls back to the legacy ShellTool construction. * Adds two regression tests (`all_tools_registers_node_exec_when_node_enabled` and `all_tools_excludes_node_exec_when_node_disabled`) so future refactors can't silently drop the wiring. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(agents): grant node_exec + npm_exec to code_executor + tool_maker (#681) code_executor is the sandboxed developer sub-agent — it writes, runs, and debugs code — and tool_maker is the narrow self-healing agent that polyfills missing commands. Both now see the managed Node.js tools so skills and polyfills can call into the resolved runtime directly rather than relying on whatever `node`/`npm` happens to be on the host PATH. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * style(core): apply cargo fmt auto-fixes (#681) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * chore(cargo): sync Cargo.lock to OpenHuman v0.52.26 after rebase (#681) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(node_runtime): strip leading v/V defensively in resolve_from_system (#681) CodeRabbit R1 flagged that build_resolved receives a raw version string from SystemNode. detect_system_node already trims the prefix, but trim defensively at the resolve_from_system boundary too so any future code path constructing SystemNode with an un-normalised version won't emit "vvX.Y.Z". Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(node_runtime): propagate flush errors and clean up partial archives (#681) CodeRabbit R2 flagged that download_distribution silently swallowed flush errors via `.ok()` and left partial files on disk when any chunk/write/flush step errored. Wrap the streaming loop in an async block returning Result<()>, propagate flush errors, and on any failure delete the partial archive so a retry starts clean and callers never see a half-written file. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(node_runtime): restore previous install on staged rename failure (#681) CodeRabbit R3 flagged that atomic_install left the user with no Node runtime on disk if the staged->final rename failed after a backup had been taken. Wrap the rename in `if let Err`, restore the backup if present, log restore failures separately (warning), and always return the original error so the caller sees the true failure cause. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(node_runtime): enforce real 5s timeout on node --version probe (#681) CodeRabbit R4 flagged that probe_node_version used Command::output() with no real timeout — a broken shim or FUSE-backed binary could hang the bootstrap forever. Add wait-timeout crate dep and rewrite the probe to spawn with piped stdio, wait_timeout(5s), kill on timeout, and read stdout/stderr from the piped handles after exit. Logs a warning when a probe times out so the install can be diagnosed. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(tools): reject script_path escapes in node_exec (#681) CodeRabbit R5 flagged that node_exec joined user-supplied script_path onto the workspace without rejecting `..` segments or absolute paths, letting a prompt-injected `../../../etc/passwd` read arbitrary files via node. Add a resolve_script_path helper mirroring npm_exec::resolve_cwd — reject empty, absolute, parent-dir, and Windows-prefix components. Extra positional args stay opaque (shell-quoted, not path-checked) and get an explanatory comment at the call site. Unit tests cover each rejection case plus the relative- subdir happy path. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(skills): drop dead SKILL.md branch in legacy manifest loader (#681) CodeRabbit N1 flagged dead code: load_from_legacy_manifest only runs when load_skill_dir already determined SKILL.md is absent, so the fallback that re-checked for SKILL.md and its helper read_skill_md_description could never execute. Simplify to a direct description/location decision and delete the dead helper. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(config): extract parse_env_bool helper with warn on unknown values (#681) CodeRabbit N2 flagged two near-identical boolean env-override match blocks for OPENHUMAN_NODE_ENABLED and OPENHUMAN_NODE_PREFER_SYSTEM. Extract a module-level parse_env_bool helper that accepts 1/true/yes/on and 0/false/no/off (case-insensitive) and emits a tracing::warn when the value is unrecognised so silent mis-spellings don't invisibly leave the config unchanged. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(node_runtime): detect corrupted cache missing npm (#723) `probe_managed_install` now verifies the npm launcher exists before reusing an extracted install. A download interrupted after `node` was extracted but before `npm` would otherwise be cached forever and `npm_exec` could never self-heal — now the corrupted cache forces a fresh download via the normal resolve path. Addresses CodeRabbit review feedback on PR #723. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(node_runtime): skip non-executable PATH shims when probing node (#723) `which_node` previously returned the first matching filename on `PATH` regardless of its execute bit. A non-executable `node` placeholder earlier in `PATH` (e.g. an unprivileged shim left by a failed install) would mask a valid later install and force the managed runtime download. Now checks `file && (mode & 0o111 != 0)` on Unix to mirror shell `which` behaviour. Windows remains unchanged — `.exe` suffix already encodes executability for the loader. Addresses CodeRabbit review feedback on PR #723. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(skills): deterministic entry order in scan_root (#723) `read_dir` order is unspecified by the OS. When two sibling skill directories declare the same logical `frontmatter.name` (which can differ from the folder name), cross-scope/same-scope deduplication downstream would pick a non-deterministic winner across runs. Sort entries by on-disk directory name for a stable, reproducible order so collision resolution is deterministic. Addresses CodeRabbit review feedback on PR #723. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(skills): surface YAML parse errors as skill warnings (#723) `parse_skill_md` previously swallowed the `serde_yaml` error via `log::warn!` and fell back to an empty `SkillFrontmatter`, then the catalog reported a generic "could not parse — exposing directory as placeholder". Skill authors had no way to see the real cause without scraping logs. Return parse-level diagnostics as a third tuple element. `load_from_skill_md` merges them into the skill's user-visible `warnings`, so the catalog now surfaces the actual YAML error (e.g. "frontmatter parse error: mapping values are not allowed here at line 3 column 12"). Addresses CodeRabbit review feedback on PR #723. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(skills): skip symlinks when walking skill resources (#723) `walk_files` used `is_dir()` / `is_file()` which transparently follow symlinks. Two failure modes: 1. **Unbounded recursion** — a skill resource symlink that points back at an ancestor (e.g. `resources/self -> resources/`) would cause infinite traversal, eventually blowing the stack. 2. **Silent out-of-tree leakage** — a symlink pointing at `/`, `/etc`, or another skill's directory would enumerate its contents into the current skill's resource listing. Switch to `entry.file_type()` and skip symlinks before descending. Directories and regular files behave as before. Addresses CodeRabbit review feedback on PR #723. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(node_runtime): default managed cache to user-owned OS cache (#723) Default `cache_root()` to `dirs::cache_dir()/openhuman/node-runtime/` so a repo cannot ship a checked-in `./node-runtime/` and have the bootstrap reuse it as a trusted managed install. Explicit `config.cache_dir` still wins; workspace-local falls back only when `dirs::cache_dir()` is unavailable, and we emit a warning on that fallback. As a second line of defence, `probe_managed_install()` now canonicalises both the install dir and the cache root and refuses any install that escapes the cache tree. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(node_runtime): require npm when accepting system node (#723) On distros that package `node` and `npm` separately (Debian/Ubuntu, Alpine `nodejs-current`, some NixOS setups) the host `node` can be present without `npm`. `npm_exec` then fails every call because the resolved `SystemNode` has no usable npm launcher. Before returning `Some(SystemNode)`, locate `npm` on `PATH` and probe `npm --version` through the same `wait_timeout` path as the node probe. Either missing gate falls back to the managed download flow. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(skills): surface user-scope skills via load_skills (#723) Existing production callers (`agent::harness::session::builder`, `channels::runtime::startup`) reach the skill catalog via `load_skills(workspace_dir)`. Previously this shim passed `None` for the home directory, so skills installed under `~/.openhuman/skills/` and `~/.agents/skills/` were silently dropped even after the multi-scope discovery landed in `discover_skills`. Delegate to `discover_skills_inner` with `dirs::home_dir()` so user-scope skills reach the runtime; project-scope still wins on name collision. Tests stay hermetic via a `load_skills_ws` helper that preserves the old workspace-only semantics. A new `load_skills_surfaces_user_scope` test drives a tempdir-as-home through `discover_skills` and asserts the user-scope skill is returned with `SkillScope::User`. Addresses CodeRabbit review comment 3116332244. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(skills): skip symlinked skill dirs during scan (#723) `scan_root` previously accepted any entry where `path.is_dir()` returned true. `is_dir()` dereferences symlinks, so a link from `<skills-root>/foo -> /some/external/tree` would load as a legitimate skill even though `walk_files` already rejects symlinks deeper in the resource walker. Attacker-authored symlinks in a skills root would therefore have one remaining escape hatch. Switch to `entry.file_type()` (non-dereferencing) and reject both symlinked and non-directory entries at the top level. Treat a failed `file_type()` probe as "not safe to traverse" and skip. A new `symlinked_skill_dirs_are_skipped` test (Unix-only, since symlinks are the platform guarantee being tested) creates an external skill tempdir, links it into the workspace skills root, and asserts `load_skills_ws` returns empty. Addresses CodeRabbit review comment 3116332252. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(skills): reject symlinked resource roots (#723) `inventory_resources` gated each resource sub-root (`scripts/`, `references/`, `assets/`) on `root.is_dir()`. `is_dir()` follows symlinks, so a `scripts -> /etc` link inside a skill directory would pass the check and `walk_files` would inventory the external tree. Deeper symlinks inside the walk were already rejected by `walk_files`, but the root-level check was the missing layer. Use `std::fs::symlink_metadata` for a non-dereferencing probe and reject roots whose own `file_type().is_symlink()` returns true. Treat any `symlink_metadata` error as a non-existent root and skip. A new `symlinked_resource_roots_are_rejected` test (Unix-only) links a skill's `assets/` sub-root to an external tempdir with a file inside and asserts `inventory_resources` returns empty. Addresses CodeRabbit review comment 3116332260. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Mega MindandGitHub
|
1792135aea |
fix(settings): tool capability toggles now persist and are enforced at runtime (#720)
* chore: update OpenHuman version to 0.52.26 and refine tool preference handling - Bumped the OpenHuman package version to 0.52.26 in Cargo.lock files. - Enhanced the ToolsPanel component to include user feedback on save status. - Implemented filtering of tools based on user preferences in the Rust backend, allowing for more customizable tool availability. - Added new utility functions to map UI tool IDs to Rust tool names for better integration. These changes improve user experience and maintain compatibility with the latest features. * style: apply cargo fmt to user_filter.rs |
||
|
Steven Enamakel
|
8bdecf6100 |
chore: update OpenHuman version to 0.52.24 and enhance CSP in tauri.conf.json
- Bumped the OpenHuman package version to 0.52.24 in both Cargo.lock files. - Updated the Content Security Policy (CSP) in tauri.conf.json to allow connections from localhost and 127.0.0.1, improving development flexibility. These changes ensure compatibility with the latest OpenHuman features and enhance security settings for local development. |
||
|
Steven EnamakelandGitHub
|
e5f571cec7 |
feat(tokenjuice): Rust port of terminal-output compaction engine (#644)
* feat(tokenjuice): implement core functionality for terminal-output compaction engine - Introduced the `tokenjuice` module, which includes the classification and reduction of tool outputs based on JSON-configured rules. - Added new dependencies for Unicode handling: `unicode-segmentation` and `unicode-width`. - Implemented the `classify` module to match tool execution inputs against predefined rules, enhancing the ability to process and summarize terminal outputs. - Created a comprehensive set of types and utilities for managing tool execution inputs and classification results. - Established a built-in rule set for common tools, improving the initial setup and usability of the `tokenjuice` engine. - Enhanced testing framework with integration tests to ensure the accuracy of output compaction and classification. These changes lay the groundwork for a robust terminal-output management system, facilitating better interaction with various tools and improving overall user experience. * feat(tokenjuice): implement tokenjuice module for terminal output compaction - Introduced the `tokenjuice` module, which includes functionality for classifying and reducing terminal output based on JSON-configured rules. - Added new dependencies: `unicode-segmentation` and `unicode-width` to support text processing. - Created a new `classify.rs` file for rule classification logic, including matching helpers and scoring functions. - Implemented a `reduce.rs` file to handle the main reduction pipeline and text normalization. - Established a structured approach for loading and compiling rules from multiple sources, including built-in and user-defined rules. - Added integration tests to ensure the correctness of the output reduction process. These changes enhance the application's ability to manage and compact verbose tool outputs, improving overall efficiency and user experience. * test(tokenjuice): enhance test coverage for classification and reduction logic - Added a series of unit tests to `classify.rs` to validate the behavior of tool name filters and argument matching, ensuring correct classification of tool executions. - Introduced tests for edge cases in `reduce.rs`, including command tokenization and normalization of execution inputs, to improve robustness against various input formats. - Expanded tests in `builtin.rs` to cover duplicate ID reporting and compile issues, enhancing error handling and reporting mechanisms. - Implemented additional tests in `compiler.rs` to verify regex handling in rule definitions, ensuring invalid patterns are correctly ignored. These enhancements improve the overall test coverage and reliability of the tokenjuice module, facilitating better maintenance and future development. * test(tokenjuice): add edge-case tests for gh table and reduction pipeline * style(tokenjuice): apply cargo fmt * feat(tokenjuice): wire into agent tool loop output compaction Add `tokenjuice::compact_tool_output` helper and call it in the agent tool loop after credential scrubbing (and on error paths with exit=1), before any optional payload_summarizer. Derives argv/command heuristically from JSON tool arguments (command / args / argv / cmd shapes) so shell-wrapping tools still match upstream family rules (git/*, package/*, tests/*, etc.). Pass-through safe: outputs under 512 bytes or where compaction saves <5% are returned untouched. * fix(tokenjuice): address coderabbit review comments - classify: derive command from argv join when input.command is unset, so commandIncludes* rules still match argv-only callers - rules/loader: log read_dir / file_type / read_to_string failures at debug level so permission or filesystem issues are observable rather than silently skipped - text/ansi: add trace log at strip_ansi entry/exit with lengths (no text content) per the project debug-logging rules - tests: remove orphan src/openhuman/tokenjuice/tests/integration.rs which was never wired into any module declaration; the real fixture- parity runner lives at tests/tokenjuice_integration.rs and asserts hard when the fixtures directory is missing Vendored-rule issues (docker-ps / kubectl-describe / git/branch / grep casing / counter-pattern overbreadth / etc.) come from upstream and are left as-is; this module is a straight port of the upstream rule set and should not fork from it in v1. |
||
|
Steven EnamakelandGitHub
|
8f4696bdfb |
feat(composio): per-toolkit tool curation, user scopes, and Gmail HTML→markdown (#643)
* feat(composio): add user scope management for toolkits - Introduced `get_user_scopes` and `set_user_scopes` functions to manage per-toolkit user scope preferences, allowing for read, write, and admin classifications. - Updated `all_controller_schemas` and `all_registered_controllers` to include new schemas for user scope management. - Implemented `evaluate_tool_visibility` to determine tool visibility based on user-defined scopes, enhancing security and control over tool actions. - Added `UserScopePref` struct to store user preferences and integrated it with memory storage for persistence. - Enhanced existing tools to respect user scope preferences during execution, ensuring actions align with user-defined permissions. These changes improve the flexibility and security of toolkit interactions, allowing users to customize their access levels for different actions. * feat(gmail): expand GMAIL_CURATED tools for enhanced email management - Updated the GMAIL_CURATED constant to include additional tools for reading and writing emails, such as GMAIL_LIST_MESSAGES, GMAIL_LIST_THREADS, GMAIL_GET_ATTACHMENT, and GMAIL_FORWARD_MESSAGE. - Improved organization of tools by categorizing them under distinct sections for reading messages, managing drafts, and handling labels. - Enhanced admin tools with new functionalities like GMAIL_BATCH_DELETE_MESSAGES and GMAIL_UNTRASH_THREAD, improving overall email management capabilities. These changes provide a more comprehensive toolkit for interacting with Gmail, enhancing user experience and functionality. * refactor(tool_scope, gmail): improve code formatting and organization - Reformatted the `ADMIN` and `GMAIL_CURATED` constants for better readability by aligning the entries vertically. - Enhanced the clarity of the `classify_unknown` function by improving the structure of the constants, making it easier to maintain and understand. - Updated test assertions in `toolkit_from_slug` for consistency in formatting, ensuring clearer test outputs. These changes improve the overall readability and maintainability of the codebase, aligning with ongoing refactoring efforts. * feat(github): add GitHub toolkit and curated tools for enhanced integration - Introduced a new GitHub provider module, including a curated catalog of GitHub actions tailored for common tasks such as repository management, issue tracking, and pull request handling. - Implemented the `catalog_for_toolkit` function to allow fallback to a static curated list for toolkits without a native provider, ensuring consistent tool visibility and access. - Updated the `evaluate_tool_visibility` function to prioritize curated tools from registered providers, enhancing the overall user experience and security by enforcing whitelist checks. These changes expand the capabilities of the Composio toolkit, providing users with a comprehensive set of tools for interacting with GitHub, while maintaining a focus on user-defined permissions and visibility. * refactor(tools): improve formatting and organization of curated tools - Reformatted the `GITHUB_CURATED`, `NOTION_CURATED`, and other tool constants for better readability by aligning entries vertically. - Enhanced the clarity of the code structure, making it easier to maintain and understand. - Updated related documentation to reflect the changes in formatting and organization. These changes improve the overall readability and maintainability of the codebase, aligning with ongoing refactoring efforts. * feat(catalogs): introduce curated catalogs for various toolkits - Added a new module `catalogs.rs` containing curated tool lists for Slack, Discord, Google Calendar, Google Drive, Google Docs, and more, enhancing the Composio toolkit's integration capabilities. - Updated the `mod.rs` file to include the new `catalogs` module and modified the `catalog_for_toolkit` function to support these curated lists, allowing for better organization and access to toolkit actions. - This addition improves the overall functionality and user experience by providing a comprehensive set of tools for interacting with popular platforms. * feat(post-process): implement HTML to markdown conversion for Gmail responses - Introduced a new `post_process` module to handle per-toolkit response modifications, specifically for converting HTML content to markdown format. - Enhanced the `ComposioExecuteTool` to apply post-processing on successful responses, improving the clarity and usability of data returned from Gmail. - Added tests to ensure the correct functionality of HTML detection and conversion, validating the integrity of the post-processing logic. These changes enhance the user experience by streamlining the handling of HTML content in responses, making it more suitable for further processing and display. * refactor(post_process): reorganize HTML detection constants for improved readability - Moved HTML detection markers in the `looks_like_html` function to a more structured format, enhancing clarity and maintainability. - This change aligns with ongoing efforts to improve code organization and readability within the post-processing module. * refactor(catalogs): enhance formatting and organization of curated tool constants - Reformatted the `SLACK_CURATED`, `DISCORD_CURATED`, and `GOOGLECALENDAR_CURATED` constants for improved readability by aligning entries vertically. - This change enhances the clarity and maintainability of the code, aligning with ongoing refactoring efforts to improve code organization. * feat(connect-modal): wire read/write/admin scope toggles to composio prefs Adds the three scope toggles to the connected-state of the ComposioConnectModal so users can gate which Composio actions the agent may invoke per integration. Loads the stored pref via `composio_get_user_scopes` once the modal lands in the connected phase and persists changes through `composio_set_user_scopes` with optimistic updates and rollback on error. - Adds `getUserScopes` / `setUserScopes` to composioApi. - Adds `ComposioUserScopePref` type mirror. - Renders accessible role="switch" toggles with hint text per row. * refactor(ComposioConnectModal): simplify SCOPE_ROWS definition - Streamlined the definition of SCOPE_ROWS in ComposioConnectModal by removing unnecessary line breaks, enhancing code readability and maintainability. - Updated the agent.toml configuration to include "composio_execute" in the tools list, expanding the capabilities of the integrations agent. * fix(composio): apply curated whitelist + scope pref to integrations_agent prompt The agent prompt for integrations_agent was rendering every action returned by the backend's `composio_list_tools` for each connected toolkit, bypassing the curation/scope filter that the meta-tool layer applies. Concretely the GitHub integrations_agent prompt was showing ~500 actions including non-curated entries like GITHUB_ADD_OR_UPDATE_TEAM_REPOSITORY_PERMISSIONS. Adds `is_action_visible_with_pref(slug, pref)` — a sync helper that mirrors the meta-tool layer's decision logic — and applies it in: - `fetch_connected_integrations_uncached` (bulk session-cached path) - `fetch_toolkit_actions` (per-toolkit spawn-time path) One pref load per toolkit (not per action) keeps the cost minimal. * refactor(fetch_connected_integrations): streamline action visibility filter Simplified the action visibility filter in `fetch_connected_integrations_uncached` by consolidating the filter logic into a single line. This change enhances code readability while maintaining the existing functionality of applying user preferences to the displayed actions. * fix(prompt): drop duplicate `### Available Tools` listing in text-mode preamble Text-mode subagent prompts were rendering the tool catalog twice: once in the prompt template's `## Tools` section (with the richer `Call as: NAME[arg|arg]` signatures from `prompts::ToolsSection::build`) and once in `### Available Tools` under `## Tool Use Protocol` (`Parameters: name:type, ...` format). For an integrations_agent toolkit spawn (~50 actions) this doubled the tool listing bytes for no informational gain. Keep only the protocol preamble (essential for text mode); the catalog stays in `## Tools`. Removes `summarise_parameters` and `first_line_truncated` which were the sole consumers, plus the now-unused `std::fmt::Write` import. * review: address PR feedback (UTF-8 boundary, structured logs, doc, debug) Real fixes: - post_process: walk back to UTF-8 char boundary before truncating to 4096 bytes; previous `&s[..4096]` could panic mid-codepoint. Adds regression test that places a 3-byte char straddling the cutoff. - composioApi: move misplaced `execute` docstring back above `execute` (it had drifted above the new `getUserScopes`). - schemas: include `get_user_scopes` / `set_user_scopes` in the `every_known_schema_key_resolves` test. Diagnosability: - schemas: structured `[composio:scopes]` debug/error logs at entry, exit, and every early-return in `handle_get_user_scopes` / `handle_set_user_scopes` (method + toolkit + pref fields). The memory-not-ready branch now logs an error before returning. - composioApi + ConnectModal: grep-friendly `[composio][scopes]` debug logs around getUserScopes / setUserScopes RPC round-trips and the toggle handler (old → new state, persisted result, errors). - user_scopes: load_or_default now logs the same normalized `key` that `load()` does, so traces correlate across both code paths. Cleanups: - tools: replace external `idx` + `Vec::retain` in `filter_list_tools_response` with a drain + zip + filter_map pattern. - tool_scope: document the no-underscore-in-toolkit-name assumption of `toolkit_from_slug`, and call out the `microsoft_teams` alias. - Cargo.toml: comment on the html2md vs htmd choice. Pushback (no change): - Reviewer asked to split the type-only import in ConnectModal into a separate `import type` line; ESLint's `no-duplicate-imports` rejects that, so the inline `type` form (functionally identical) stays. |
||
|
Steven EnamakelandGitHub
|
99d61f93a7 |
feat(webui-messaging): multi-provider webview accounts, scanners, and chat runtime (#629)
* feat(accounts): implement accounts management with webview integration - Added a new Accounts page for managing user accounts, including the ability to add and remove accounts. - Introduced AddAccountModal for selecting account providers and initiating account setup. - Implemented WebviewHost to display third-party web applications (e.g., WhatsApp Web) within the app. - Enhanced routing to include a protected route for the Accounts page. - Updated the BottomTabBar to include an Accounts tab for easy navigation. - Integrated Redux for state management of accounts, messages, and logs, ensuring a seamless user experience. - Updated dependencies in Cargo.lock to version 0.52.9 for compatibility. This commit significantly enhances the application's functionality by allowing users to manage accounts directly within the app, improving overall user engagement and experience. * refactor(accounts): clean up Accounts component layout and improve readability - Removed unnecessary comments and simplified the structure of the Accounts component for better clarity. - Adjusted the rendering logic to enhance the layout of the active account section, improving user experience. - Reformatted text in the no accounts message for better readability. - Streamlined the import statements by consolidating related imports, enhancing code organization. * feat(accounts): enhance account management with new providers and routing updates - Introduced support for additional account providers: Telegram, LinkedIn, Gmail, and Slack, expanding user options for account management. - Updated routing to replace the old /conversations path with /chat, streamlining navigation and improving user experience. - Refactored the App component to include an AppShell for better layout management, ensuring the bottom tab bar visibility aligns with the selected account. - Enhanced the BottomTabBar component to reflect the new routing and account options, improving accessibility and usability. - Implemented fullscreen logic for accounts, allowing for a more immersive experience when interacting with selected accounts. - Added utility functions for managing fullscreen states and account provider icons, enhancing code organization and maintainability. This commit significantly improves the application's account management capabilities, providing users with a more flexible and engaging experience. * feat(cef): integrate Chromium Embedded Framework support and enhance webview functionality - Added support for the Chromium Embedded Framework (CEF) as an alternative runtime, allowing for improved webview capabilities. - Updated Cargo.toml to include new dependencies and features for CEF integration, ensuring compatibility with existing Tauri plugins. - Enhanced the WhatsApp recipe to include ghost-text autocomplete functionality, improving user experience during message composition. - Implemented WebSocket observation in the WhatsApp recipe to capture and forward relevant message frames, enhancing real-time interaction. - Introduced user agent spoofing for specific providers to bypass fingerprinting checks, ensuring better compatibility with services like Slack and LinkedIn. - Refactored various components to accommodate the new runtime and improve overall code organization and maintainability. This commit significantly enhances the application's webview capabilities and user interaction with messaging services, providing a more robust and flexible experience. * feat(cef): add development command for CEF and update configuration - Introduced a new development command `dev:cef` in both package.json files to streamline the development process for the Chromium Embedded Framework (CEF). - Updated Cargo.toml to include the `tauri/devtools` feature alongside `tauri/cef`, enhancing debugging capabilities. - Modified tauri.conf.json to adjust visibility settings for the application window and refined the Content Security Policy (CSP) for improved security. - Enhanced resource paths in tauri.conf.json to support recursive file inclusion for better resource management. - Updated the Rust code to bypass macOS Keychain prompts when using CEF, improving user experience during development. This commit enhances the development workflow for CEF integration, providing better tools and configurations for developers. * fix(cef): update development command for CEF to include signing identity - Modified the `dev:cef` command in package.json to include the `APPLE_SIGNING_IDENTITY` environment variable, enhancing the development process for CEF on macOS. - This change improves the build process by ensuring proper code signing during development, streamlining the workflow for developers working with CEF integration. * feat(cef): enhance development command for CEF with safe storage setup - Updated the `dev:cef` command in package.json to include a call to a new script, `setup-chromium-safe-storage.sh`, which pre-seeds the "Chromium Safe Storage" keychain entry with a permissive ACL. - Added the `setup-chromium-safe-storage.sh` script to ensure that CEF/Chromium can read the keychain entry without prompting, improving the development experience on macOS. - This change streamlines the setup process for developers working with CEF integration, ensuring a smoother workflow. * feat(whatsapp): enhance message ingestion and IndexedDB integration - Introduced a new `IngestMessage` interface to standardize message structure for WhatsApp. - Updated `IngestPayload` to include additional fields for better message handling, including `provider`, `chatId`, and `day`. - Implemented a new function `persistWhatsappChatDay` to handle the ingestion of chat messages by day, improving data organization and retrieval. - Enhanced the WhatsApp recipe to utilize IndexedDB for direct data access, eliminating the need for DOM scraping and improving performance. - Updated the Tauri configuration to enable development tools for easier debugging of webview accounts. This commit significantly improves the application's ability to manage and ingest WhatsApp messages, providing a more robust and efficient user experience. * feat(cdp): integrate IndexedDB scanner for WhatsApp via Chrome DevTools Protocol - Added a new module for scanning IndexedDB using the Chrome DevTools Protocol (CDP), enabling direct access to WhatsApp data without DOM scraping. - Implemented a scanner that communicates with the embedded CEF instance to read and decrypt messages stored in IndexedDB. - Updated the Tauri application to manage the new scanner, ensuring it operates seamlessly with existing webview accounts. - Enhanced the Cargo.toml and Cargo.lock files to include necessary dependencies such as `tokio-tungstenite` and `futures-util` for asynchronous operations. - Refactored the WhatsApp recipe to utilize the new scanning capabilities, improving performance and data handling. This commit significantly enhances the application's ability to interact with WhatsApp's IndexedDB, providing a more efficient and robust user experience. * feat(cdp): enhance message diagnostics in IndexedDB scanner - Updated the ScanSnapshot struct to include new fields for message diagnostics: `messageKeyUnion`, `messageTypeBreakdown`, and `sampleByType`, providing a comprehensive overview of message structures and types. - Modified the scanner logic to capture and log detailed information about message types and their shapes, improving debugging capabilities. - Refactored the JavaScript scanner to aggregate message key signatures and counts, enhancing the analysis of message records. This commit significantly improves the application's ability to analyze and log message data from WhatsApp's IndexedDB, facilitating better debugging and data handling. * feat(cdp): implement fast DOM scraping and crypto key extraction for WhatsApp - Introduced a new fast-tick DOM scraping mechanism to extract rendered WhatsApp message bodies, enabling near real-time message updates without relying on IndexedDB. - Added scripts for capturing and logging CryptoKey operations within WhatsApp's workers, allowing for better analysis of key derivations and decryptions. - Enhanced the CDP scanner to interleave fast DOM scans with full IndexedDB scans, optimizing data retrieval and reducing UI spamming during idle periods. - Updated the ScanSnapshot struct to include new fields for DOM-scraped messages and crypto operation statistics, improving the overall diagnostic capabilities of the application. This commit significantly enhances the application's ability to interact with WhatsApp's messaging system, providing a more efficient and responsive user experience. * feat(whatsapp): replace cdp_indexeddb with whatsapp_scanner for enhanced message handling - Replaced the `cdp_indexeddb` module with `whatsapp_scanner` to streamline the scanning process for WhatsApp messages. - Updated the application to manage the new `ScannerRegistry` for WhatsApp, improving the integration with the Chrome DevTools Protocol. - Introduced new scripts for fast DOM scraping and full IndexedDB scanning, optimizing data retrieval and enhancing real-time message updates. - Added a new `dom_scan.js` for efficient extraction of rendered message bodies directly from the DOM, reducing reliance on IndexedDB. - Enhanced the `ScanSnapshot` struct to accommodate new fields for DOM-scraped messages, improving diagnostic capabilities. This commit significantly improves the application's ability to interact with WhatsApp, providing a more efficient and responsive user experience. * refactor(whatsapp): replace JavaScript DOM scanning with Rust-based DOM snapshot - Removed the `dom_scan.js` script and replaced it with a new Rust module `dom_snapshot.rs` that captures DOM snapshots directly via the Chrome DevTools Protocol, enhancing performance and reliability. - Introduced a new `idb.rs` module for scanning WhatsApp's IndexedDB, streamlining data retrieval and improving integration with the Rust backend. - Updated the `ScanSnapshot` struct to accommodate changes in data handling, ensuring compatibility with the new scanning methods. - Enhanced overall message handling capabilities, providing a more efficient and responsive user experience. This commit significantly improves the application's ability to interact with WhatsApp, leveraging Rust for better performance and reducing reliance on JavaScript for DOM operations. * feat(docs): add webview integration playbook for third-party messaging - Introduced a comprehensive playbook detailing the process for integrating third-party webviews (e.g., Instagram, Messenger) into the application. - Documented architecture, workflow, and best practices for building and debugging new integrations, leveraging Rust and Chrome DevTools Protocol. - Included step-by-step instructions for setting up scanners, monitoring logs, and optimizing message handling, ensuring a streamlined development experience for future integrations. This addition enhances the documentation, providing developers with a clear guide to implement and maintain webview integrations effectively. * docs(webview): improve table formatting for clarity - Enhanced the formatting of tables in the webview integration playbook to improve readability and consistency. - Adjusted column headers and alignment for better presentation of job intervals and costs, ensuring clearer communication of scanning processes and common pitfalls. This update aims to provide a more user-friendly documentation experience for developers integrating third-party webviews. * feat(slack): integrate Slack scanner for message extraction and management - Updated the OpenHuman package version to 0.52.15 in both Cargo.lock files. - Introduced a new Slack scanner module to extract messages, users, and channels from Slack's IndexedDB using the Chrome DevTools Protocol. - Added functionality to manage Slack accounts within the application, allowing for automatic opening of Slack webviews based on environment variables. - Enhanced the existing webview account management to support Slack integration, ensuring seamless interaction with the Slack API. This commit significantly improves the application's ability to interact with Slack, providing a robust framework for message handling and account management. * fix(slack): one-doc-per-channel + omit CDP indexName param CEF 146's IndexedDB.requestData rejects `indexName: ""` with "Could not get index"; the CDP spec says empty string means the primary-key index but this backend only accepts the field unset. Omit it entirely so the Slack Redux-persist dump actually comes back. Also switch memory grouping from (channel, day) → channel. Each Slack channel is now one long-running memory doc keyed by channel name (e.g. `general`, `team-product`, `elvin516`), falling back to channel id for non-slug names. Every transcript line carries its own `YYYY-MM-DD HH:MM` stamp and the header records the full date range. `infer_team_id` updated to Slack's real DB naming pattern `objectStore-<TEAM>-<USER>` (not `ReduxPersistIDB:` as initially assumed). * fix(onboarding): update navigation and test descriptions in OnboardingOverlay tests - Changed the navigation path from '/conversations' to '/chat' in the OnboardingOverlay tests to reflect the updated routing logic. - Updated test descriptions for clarity, ensuring they accurately describe the functionality being tested. These changes enhance the accuracy and readability of the onboarding tests, aligning them with the current application flow. * fix(conversations): add return statement to Conversations component - Introduced a return statement in the Conversations component to ensure proper rendering of the sidebar or page variant. - This change enhances the component's functionality by ensuring it returns the expected JSX structure. These modifications improve the overall structure and behavior of the Conversations component. * feat(accounts): add Discord integration and enhance AddAccountModal - Introduced Discord as a new account provider, including its icon and service details. - Updated the AddAccountModal to filter out already connected providers, improving user experience. - Enhanced the UI to display a message when all providers are connected, ensuring clarity for users. - Implemented context menu functionality for account management, allowing users to log out directly from the accounts list. These changes expand the application's capabilities by integrating Discord and refining account management features. * feat(discord): integrate Discord scanner for HTTP and WebSocket monitoring - Added a new `discord_scanner` module to capture Discord API calls and WebSocket frames using the Chrome DevTools Protocol (CDP). - Updated the `lib.rs` to manage the new Discord scanner alongside existing WhatsApp and Slack scanners. - Enhanced the `webview_accounts` module to support Discord account management, including scanner registration and cleanup. These changes expand the application's capabilities by enabling real-time monitoring of Discord interactions, enhancing user experience and functionality. * feat(google-meet): integrate Google Meet as a new account provider - Added Google Meet as a supported account provider, including its icon and service details. - Updated the account management logic to handle Google Meet interactions, including recipe integration for call monitoring and notifications. - Enhanced the UI to accommodate the new provider, ensuring a seamless user experience when managing accounts. These changes expand the application's capabilities by integrating Google Meet, allowing users to join calls and receive notifications directly within the app. * refactor(runtime): remove notification handling and composer autocomplete - Eliminated the notification interception logic and associated functions, streamlining the runtime code. - Removed composer autocomplete features, transferring responsibility for ghost-text overlays to the UI host. - Updated comments to reflect the changes and clarify the remaining functionality. These modifications simplify the runtime script, focusing on core features while delegating UI responsibilities. * feat(google-meet): enhance Google Meet integration with lifecycle event handling - Implemented lifecycle event handling for Google Meet, including events for call start, captions, and call end. - Introduced in-memory storage for caption snapshots during meetings, allowing for the generation of markdown transcripts upon call completion. - Added interfaces for payload structures related to Google Meet events, improving type safety and clarity in the codebase. - Updated the webview account service to manage active meetings and flush transcripts to memory, ensuring a seamless user experience. These changes significantly enhance the Google Meet integration, enabling real-time caption handling and transcript generation, thereby improving the overall functionality of the application. * feat(cef): integrate CEF-based notification handling and update dependencies - Added support for native OS notifications through the `tauri-runtime-cef` crate, enabling interception of browser notifications in embedded webviews. - Introduced a new submodule for `tauri-cef` to manage CEF dependencies and facilitate notification handling. - Updated the `.gitignore` to exclude CEF-related build artifacts and lock files. - Removed the deprecated `notification_scanner` module, streamlining the codebase and focusing on the new CEF integration. - Enhanced the `webview_accounts` module to register and manage CEF browser notifications, improving user experience with real-time alerts. These changes significantly enhance the application's notification capabilities, leveraging CEF for a more integrated and responsive user experience. * feat(cef): update CEF integration and dependency management - Added `cef` and `tauri-runtime-cef` as dependencies to enhance CEF support. - Updated `Cargo.toml` to reference `tauri-runtime-cef` from a local path, ensuring proper integration with the vendored CEF submodule. - Removed direct Git references for Tauri packages, streamlining dependency management by using local paths. These changes improve the application's CEF capabilities and simplify the dependency structure, facilitating better integration and maintenance. * feat(browserscan): add BrowserScan as a new account provider with associated resources - Introduced BrowserScan as a development-only account provider, including its icon and service details. - Updated the account provider types and management logic to accommodate BrowserScan, enhancing the application's capabilities. - Added a new recipe and manifest for BrowserScan, ensuring it integrates seamlessly into the existing webview account lifecycle. - Enhanced the UI to display BrowserScan, providing users with a bot-detection sandbox for testing purposes. These changes expand the application's functionality by integrating BrowserScan, allowing for improved testing and development workflows. * feat(google-meet): enhance Google Meet integration with new transcript handling and session recovery - Introduced a new service to handle Google Meet transcripts, enabling structured note extraction and proactive follow-up actions. - Implemented session recovery logic to manage in-progress meetings when navigating away from the call. - Updated the webview account service to log call events and captions, improving monitoring and debugging capabilities. - Enhanced the Google Meet recipe to persist meeting state across navigations, ensuring seamless user experience. These changes significantly improve the Google Meet integration, allowing for better management of meeting transcripts and user interactions. * refactor(App): reorganize imports and clean up code structure - Removed unused imports from App.tsx to streamline the code. - Adjusted the import order for better readability and consistency. - Enhanced the BottomTabBar component by simplifying the button rendering logic. - Cleaned up the AddAccountModal component by consolidating prop destructuring. - Improved formatting in various components for better code clarity. These changes enhance code maintainability and readability across the application. * update tauri * feat(google-meet): improve caption handling and speaker identification logic - Enhanced the logic for extracting speaker names from Google Meet rows, adding checks to filter out icon ligatures and irrelevant text. - Updated the caption processing to better identify and score caption regions, ensuring more accurate transcript generation. - Introduced new utility functions to differentiate between real captions and icon names, improving the overall reliability of the captioning feature. These changes significantly enhance the accuracy and usability of the Google Meet integration, providing users with clearer and more relevant caption data. * chore(dependencies): update Cargo.lock with new package versions and remove obsolete entries - Bump OpenHuman version to 0.52.20. - Add new dependencies: cef, futures-util, tauri-plugin-notification, tauri-runtime-cef, tokio-tungstenite, bzip2, clap, console, cookie_store, data-encoding, dioxus-debug-cell, document-features, download-cef, encode_unicode, filetime. - Remove obsolete packages: alloc-no-stdlib, alloc-stdlib, brotli, brotli-decompressor, gdkx11, gdkx11-sys. - Update existing dependencies to their latest versions where applicable. * chore(dependencies): update submodule URLs and pin plugin versions - Changed the tauri-cef submodule URL from SSH to HTTPS for consistency. - Updated Cargo.toml to pin tauri plugins to a specific commit for reproducibility. - Adjusted Cargo.lock to reflect the new plugin source format with pinned revisions. - Modified the builder configuration in lib.rs to conditionally enable devtools in debug builds only, enhancing security in release builds. - Updated webview_accounts to restrict devtools access based on build type, ensuring better control over webview inspection. * feat(ui): enhance BottomTabBar and AddAccountModal interactions - Added focus and blur event handlers to BottomTabBar for improved visibility management. - Implemented keyboard accessibility in AddAccountModal by closing the modal on Escape key press and focusing the close button when opened. - Updated Content Security Policy in tauri.conf.json for enhanced security. - Improved Gmail recipe to use stable message IDs for better message tracking. - Introduced account ID sanitization in webview_accounts to prevent path traversal vulnerabilities. * fix(AddAccountModal): add missing import for improved functionality * chore(dependencies): bump OpenHuman version to 0.52.20 in Cargo.lock |
||
|
Mega MindandGitHub
|
43d44acc1a | fix: wire proactive welcome into conversations (#635) | ||
|
|
f098cfda40 |
test(coverage): batches 13–18 — Rust unit tests for 40+ modules (#530) (#618)
* test(coverage): batch 13–14 — core/all registry, proxy_config tool (#530) Add 36 new tests: - core/all: registry integrity (no duplicates, schema-controller parity), rpc_method_name, namespace_description, validate_params, schema lookup - proxy_config: parse_scope, parse_string_list (CSV, array, errors), parse_optional_string_update, env_snapshot, proxy_json, security gates All 4096 tests pass. * test(coverage): batch 15 — memory/store/unified/helpers pure functions (#530) Add 31 new tests for helpers module: - vec_to_bytes/bytes_to_vec roundtrip, cosine_similarity (identical, orthogonal, mismatched, zero), collapse_whitespace, normalize_search_text, tokenize_search_terms, normalize_graph_entity/predicate, json_string_array, merge_unique_string_arrays, json_i64 (int/float/missing/string), recency_score (current/old/future), chunk_document_content All 4127 tests pass. * test(coverage): batch 16 — terminal, supervision, security detect, trigger history (#530) Add 27 new tests across 4 modules: - accessibility/terminal: is_text_role, is_terminal_app, looks_like_terminal_buffer, extract_terminal_input_context, noise line detection - channels/runtime/supervision: compute_max_in_flight_messages clamping - security/detect: all sandbox backend fallback paths (landlock, firejail, bubblewrap, docker on non-linux), disabled-via-enabled-false - composio/trigger_history: list_recent with limit, empty store, field validation All 4154 tests pass. * test(coverage): batch 17 — learning, update, encryption, doctor, service schemas + skills/types (#530) Add 49 new tests across 6 modules: - learning/schemas: catalog, schema validation, unknown fallback - update/schemas: check/apply schemas, optional staging_dir, controller parity - encryption/schemas: encrypt/decrypt schemas, read_required helper - doctor/schemas: report/models schemas, read_optional helper (none/null/value/error) - service/schemas: 8 lifecycle functions, restart optional params, daemon_host - skills/types: ToolResult success/error/json/mixed, serde roundtrip, ToolContent All 4203 tests pass. * test(coverage): batch 18 — accessibility/keys key-state probes (#530) Add tests for is_tab_key_down/is_escape_key_down (platform-safe), macOS-specific constant validation. All 4216 tests pass. * test(coverage): batch 19 — service/restart, memory/store/factories (#530) Add 7 new tests: - service/restart: default source/reason, whitespace trimming, empty-string defaults, RestartStatus serde, startup delay noop - memory/store/factories: effective_memory_backend_name, migration-disabled error All 4223 tests pass. * test(coverage): batch 20 — tree_summarizer schemas, subconscious schemas (#530) Add 38 new tests across 2 modules: - tree_summarizer/schemas: catalog parity, all 5 functions, param helpers (read_required, read_optional, read_optional_timestamp), type_name, namespace_input - subconscious/schemas: catalog parity, all 10 functions, required input validation, field/field_req/field_opt helpers All 4261 tests pass. --------- Co-authored-by: Steven Enamakel <31011319+senamakel@users.noreply.github.com> |
||
|
Mega MindandGitHub
|
ec309c8f90 |
feat(onboarding): complete conversational onboarding stack and UX audit (#619)
* fix(onboarding): guard complete_onboarding against premature flag flip (#591) - Remove premature `chat_onboarding_completed = true` flip from `set_onboarding_completed` in config/ops.rs. The flag is now exclusively owned by the welcome agent via `complete_onboarding`. - Strip auto-finalize side-effect from `check_status` action. It is now pure read-only and returns `onboarding_status`, `exchange_count`, and `ready_to_complete` in the snapshot instead of `finalize_action`. - Add engagement guard to `complete` action: rejects with a descriptive error unless exchange_count >= 3 OR ≥1 Composio integration connected. Auth check also enforced before any write. - Add process-global `WELCOME_EXCHANGE_COUNT` (AtomicU32) with `increment_welcome_exchange_count()` / `get_welcome_exchange_count()`. Dispatch layer calls `increment_welcome_exchange_count()` each time a user message routes to the welcome agent. - Update `welcome_proactive.rs` snapshot call to new signature and prompt copy to reflect `onboarding_status: "pending"` / no pre-flip. - Add 7 pure-logic unit tests for `engagement_criteria_met`, plus tests for the new exchange counter and updated snapshot shape. Fixes #591 * style: cargo fmt for complete_onboarding.rs * feat(onboarding): inject CONNECTION_STATE block into welcome-agent turns (#593) Add `build_connection_state_block()` to dispatch.rs: fetches current Composio integration status (with a 3s timeout for graceful degradation) and formats it as a `[CONNECTION_STATE]...[/CONNECTION_STATE]` block. After `resolve_target_agent` selects the welcome agent, the block is appended to the last user message in history so the agent always has up-to-date connection state without spending a tool call. This captures OAuth completions that happened mid-conversation (user clicked auth link in chat, authenticated in browser, came back). Scoped strictly to welcome-agent turns (chat_onboarding_completed=false). Orchestrator turns are unaffected. Fixes #593 Part of #599 * style: cargo fmt for dispatch.rs * feat(welcome): parallel template messages with LLM inference for faster perceived welcome (#592) Show two template messages immediately while the LLM runs in the background, cutting the perceived wait from ~15s to ~0s: - Template 1 (t≈0ms): time-of-day greeting that names any connected channels, built from the status snapshot without extra I/O - Template 2 (t=4s): "Getting everything ready for you..." loading indicator, published via tokio::join! alongside the LLM future - LLM response: published when inference completes, opened directly with personalised setup content (no duplicate greeting) `run_proactive_welcome` now fires three `ProactiveMessageRequested` events rather than one. The two template helpers (`time_of_day_greeting`, `build_template_greeting`) are pure functions covered by 6 new unit tests. `prompt.md` gains a proactive-invocation section explaining that greeting templates are pre-delivered and the agent must skip them. * feat(welcome): add composio_authorize tool and inline auth link guidance (#594) Wire `composio_authorize` into the welcome agent's tool allowlist and teach the prompt how to offer OAuth links directly in chat: - agent.toml: add "composio_authorize" to the named tools list so the welcome agent can call it during multi-turn conversations - prompt.md: new "Offering inline auth links" section covering the consent-first flow (offer → user agrees → call tool → markdown link → confirm success via [CONNECTION_STATE] block on next turn) - prompt.md: toolkit slug reference table for the common services - prompt.md: auth link rules (no speculative calls, no bare URLs, one service at a time, await CONNECTION_STATE before confirming) - prompt.md: two new "What NOT to do" bullets for composio_authorize misuse patterns * feat(welcome): conversational onboarding prompt for issue #595 - Rewrite prompt.md: multi-turn flow, Gmail-first, no silent first turn - Align with check_status/complete and ready_to_complete semantics - Update proactive injection copy to match new tool wording Made-with: Cursor * feat(onboarding): add completion readiness reason Made-with: Cursor * docs(ux): add onboarding and welcome audit for #563 Made-with: Cursor * docs(welcome): clarify OAuth link behavior in onboarding prompt Updated the onboarding prompt to explicitly state that clicking the Gmail connection link opens the user's default browser for authentication. Added guidance to return to the chat after completing the authorization process. * fix(onboarding): resolve CI test and proactive greeting review Made-with: Cursor |
||
|
YellowSnnowmannandGitHub
|
e8639adae0 |
fix: Improve deep link authentication state management and improve logout (#608)
* feat(deepLink): implement deep link authentication state management - Added a new module for managing deep link authentication state, including processing states and error handling. - Integrated deep link authentication state management into the desktop deep link listener, enhancing the handling of authentication flows. - Introduced functions to begin, complete, and fail deep link authentication processing, improving user feedback during the authentication process. * refactor(settings): simplify logout process and enhance error handling - Removed onboarding completion flag from the logout process to streamline functionality. - Improved error handling during logout to provide user feedback if the logout fails. - Updated comments to clarify the behavior of session clearing and routing after logout. - Integrated deep link authentication state management into the settings component for better user experience. * refactor(settings, deepLink): enhance logout and session management - Simplified the logout process by removing unnecessary flags and improving error handling. - Introduced a new function to manage signed-out state in the core state provider. - Streamlined session token application in the deep link listener for better authentication flow. - Updated comments for clarity on session clearing and routing behavior post-logout. * chore(dependencies): update OpenHuman to version 0.52.16 in Cargo.lock files * refactor(format): format the code |
||
|
|
c26ca795ca |
fix: keep chat processing alive across tab switches (#587)
* fix(chat): keep in-flight responses alive across tab switches Made-with: Cursor * chore: satisfy lint and format push gates Made-with: Cursor * fix: address CodeRabbit review on chat runtime PR Made-with: Cursor * feat(chat): add explicit inference turn lifecycle in chat runtime slice Made-with: Cursor * feat(chat): implement thinking summary feature in chat responses Added a new mechanism to accumulate and send model reasoning text as a separate message during chat interactions. This includes handling "thinking_delta" events to gather reasoning content, formatting it for clarity, and ensuring it is sent before the main response. Updated the StreamingState struct to include a thinking accumulator for this purpose. This enhancement improves user experience by providing insight into the model's reasoning process. * feat(telegram): update bot username handling for staging and production environments Refactored the Telegram bot username resolution logic to differentiate between staging and production environments. Introduced constants for default usernames based on the application environment and updated the GitHub Actions workflow to set the appropriate environment variables. This change enhances the flexibility and clarity of bot username management in the application. * fix: resolve CodeRabbit review issues on feat/thinking-telegram-summary - bus.rs: fix UTF-8 char boundary panic in format_thinking_summary truncation - threadSlice.ts: remove premature activeThreadId clear from addInferenceResponse.rejected - Conversations.tsx: add composerBlocked global lock, clear tool timeline in safety timeout - ChatRuntimeProvider.tsx: replace stale toolTimelineRef/inferenceStatusRef reads with live store.getState() calls in all event handlers - LocalAIDownloadSnackbar.tsx: reset dismissed/collapsed on not-downloading → downloading transition edge - MemoryGraphMap.tsx: derive activeSelectedNode to guard stale selectedNode after relations refresh; add debug logs to useMemo graph recompute Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: use render-phase update for download dismiss reset in LocalAIDownloadSnackbar Replace effect-based setState with the React render-phase update pattern so the not-downloading → downloading transition resets dismissed/collapsed without triggering the react-hooks/set-state-in-effect lint warning. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: apply prettier and cargo fmt auto-formatting Formatting changes applied by the pre-push hook during the previous commit. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor: replace endInferenceTurn with clearRuntimeForThread in Conversations component Updated the Conversations component to replace the endInferenceTurn dispatch with clearRuntimeForThread. This change simplifies the handling of thread runtime state during error scenarios and timeout conditions, ensuring a cleaner state management approach. --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
|
d4f5b9a357 |
fix(overlay): fullscreen visibility, voice server reliability, and resize (#528) (#585)
* fix(overlay): shrink initial overlay window to match idle orb dimensions (#528) The Tauri overlay window was 248×228 px while the idle orb renders at 50×50. The excess transparent area wasted compositing resources and created an invisible click-absorbing region. Reduce to 60×60 to tightly frame the idle orb with minimal padding. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(overlay): add native drag support with position persistence (#528) - Mouse-down on the orb initiates Tauri startDragging() for native window drag - Dragged position is saved to localStorage and survives mode changes (idle ↔ active) so the orb stays where the user placed it - Double-click resets to the default bottom-right corner - Cursor changes to grab/grabbing for affordance - Skip default repositioning when a saved position exists Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(overlay): reclass NSWindow to NSPanel for fullscreen visibility (#528) macOS fullscreen apps run in separate Spaces where standard NSWindow cannot follow. Use object_setClass() to reclass the Tauri overlay window from NSWindow to NSPanel at runtime, then configure it with NonactivatingPanel style mask and Transient collection behavior — matching the working Swift accessibility helper pattern. Key configuration that makes this work: - object_setClass(NSWindow → NSPanel) — in-place reclass, no reparenting - NSWindowStyleMask::NonactivatingPanel — critical for panel behavior - NSWindowCollectionBehavior::Transient (not Stationary) — follows Spaces - Window level 25 (NSStatusWindowLevel) — floats above fullscreen apps - setFloatingPanel(true), setHidesOnDeactivate(false) Previous approaches that failed: 1. CGShieldingWindowLevel + CanJoinAllSpaces — hidden (NSWindow limitation) 2. Window level i32::MAX-17 + Stationary — hidden (Space membership issue) 3. CGS private API CGSSetWindowTags sticky bit — blocked on Sonoma 4. object_setClass WITHOUT NonactivatingPanel mask — hidden 5. Create new NSPanel + reparent webview — CRASH (Tao delegate panic) Also removes unused objc2-core-graphics and objc2-foundation deps. Ref: https://github.com/tauri-apps/tauri/issues/11488 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(sidecar): make dev signing non-fatal in stage script codesign failures no longer call process.exit(), preventing yarn tauri dev from hanging when the dev signing identity is missing or the keychain rejects the request. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(voice): prevent race condition and fix restart after stop - Atomically transition Stopped → Idle at start of run() to prevent duplicate run() calls during slow globe listener compilation - Wrap CancellationToken in Mutex so run() creates a fresh token on each start — a cancelled token cannot be reused after stop() - Reset state to Stopped if hotkey listener fails to start Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(voice): capture server errors from spawned run() task Store errors from the background server.run() task via set_last_error() so they surface in voice_server_status RPC responses instead of being silently lost. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(overlay): enable programmatic resize and shrink idle dimensions - Change overlay window from 60x60 to 50x50 to match idle orb size - Remove minWidth/minHeight constraints that blocked dynamic resize - Set resizable: true so setSize() calls work for bubble expansion Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(overlay): simplify window resize and bubble rendering - Clear min/max constraints before resizing to avoid clamping - Replace CSS transition-based bubble visibility with conditional mount for more reliable rendering when mode changes Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: fix fmt and remove unused bubbles variable Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(voice): enforce lock ordering to prevent race between run() and stop() Acquire cancel lock before state lock in run() — same order as stop() — so stop() cannot cancel a stale token between setting Idle and swapping. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(overlay): restore saved position on resize and format with prettier Parse and apply saved drag coordinates instead of just using their presence as a sentinel. Also reformats for prettier compliance. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(sidecar): fail-fast on dev signing failure in CI environments Add CI detection so signing failures abort the build in CI but remain non-fatal for local development. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: fix cargo fmt on Tauri shell import Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
YellowSnnowmannandGitHub
|
cca6c08ab0 |
Fix: discord (#580)
* feat(discord): implement managed link functionality and enhance DiscordConfig component - Added support for managed Discord linking, including new API methods for initiating and checking link status. - Enhanced the DiscordConfig component to manage link tokens and implement polling for link completion. - Introduced new state management for link tokens and improved error handling during connection processes. - Updated the definitions to include a new auth action for managed links, streamlining the onboarding experience for users. - Added tests for the new Discord link functionality to ensure robust integration and error handling. This commit significantly improves the user experience for connecting Discord accounts, providing clearer feedback and handling for managed links. * feat(discord): implement managed link functionality and enhance DiscordConfig component - Added support for managed Discord linking, including new API methods for initiating and checking link status. - Enhanced the DiscordConfig component to manage link tokens and implement polling for link status, improving user experience during the linking process. - Introduced new state management for link tokens and error handling, ensuring robust interaction with the Discord API. - Updated the channel definitions to include a new auth action for managed links, streamlining the integration process. - Added tests for the new API methods and updated existing tests to cover the new functionality. This commit significantly improves the integration with Discord, allowing users to link their accounts more effectively and providing better feedback during the linking process. * chore(release): update OpenHuman version to 0.52.9 in Cargo.lock - Bumped the version of the OpenHuman package from 0.52.7 to 0.52.9 in both Cargo.lock files for the main application and the Tauri app. - This update ensures that the latest features and fixes from the OpenHuman package are included in the project. * fix(tests): update DiscordConfig test to reflect new Connect button count - Adjusted the test for the DiscordConfig component to expect three Connect buttons instead of two, aligning with recent changes in the component's authentication modes. - This update ensures that the test accurately reflects the current functionality and improves test reliability. * refactor(discord): modularize channel permission checks and enhance logging - Extracted the `check_channel_permissions_at_base` function to modularize the permission checking logic for better readability and maintainability. - Updated the API calls to use a dynamic base URL instead of a hardcoded one, improving flexibility. - Enhanced logging for non-success responses from Discord API calls, providing more context for debugging. - Minor adjustment in the hallucination check logic to improve clarity in variable usage. * fix(logging): enhance error handling in DiscordConfig and SocketProvider - Updated error handling in DiscordConfig to log specific errors when link checks fail, improving debugging capabilities. - Enhanced SocketProvider to log non-fatal RPC connection failures, providing clearer context for potential issues with the sidecar or backend connectivity. * feat(discord): add validation functions for Discord link start and complete responses - Introduced `expectDiscordLinkStart` and `expectDiscordLinkComplete` functions to validate the structure of responses from Discord link API calls. - Updated `channelConnectionsApi` methods to utilize these new validation functions, improving error handling and response consistency. - Added tests to ensure proper unwrapping and validation of Discord link responses, enhancing overall reliability of the API integration. * fix(channelConnections): improve handling of lastError and capabilities in touchConnection function - Updated the touchConnection function to conditionally assign lastError and capabilities based on their presence in the patch object, ensuring more accurate state management. - This change enhances the reliability of connection updates by preventing unintended overwrites of existing values. * test(channelConnections): add test to clear lastError when explicitly set to undefined - Introduced a new test case to verify that the lastError state is cleared when an explicit patch sets it to undefined. This enhances the reliability of the state management in the channelConnectionsSlice reducer. * test(tests): add tests for upstream_unhealthy detection and failure reason precedence - Introduced multiple test cases to verify the detection of upstream unavailability and service unavailability errors. - Ensured that the `failure_reason` function correctly prioritizes upstream_unhealthy over other error states, enhancing the reliability of error handling in the system. * feat(discord): add OAuth success handling in DiscordConfig component - Implemented a new useEffect to handle successful OAuth events for Discord, updating the channel connection status and capabilities accordingly. - This enhancement improves the integration with Discord by ensuring that the application responds correctly to OAuth success events, providing a better user experience. * feat(deepLink): enhance OAuth deep link handling to include skillId - Updated the deep link handling logic to also retrieve the skillId from the URL parameters, improving flexibility in OAuth success scenarios. - Added a new simulation example for skillId in the setupDesktopDeepLinkListener function to aid in testing and development. * refactor(format): ran format * fix(discord): update permissions mock and improve message dispatch test - Revised the permissions mock to include additional endpoints for better accuracy in testing Discord API interactions. - Adjusted the message dispatch test to utilize a deterministic stub for the agent's response, ensuring consistent timing and behavior during tests. - Reduced the delay in the SlowProvider to enhance test performance while maintaining robustness. * chore(dependencies): update OpenHuman package version to 0.52.13 in Cargo.lock |
||
|
|
d545c193b9 |
feat(agent): fuzzy-filter skills_agent toolkit actions by task prompt (#579)
* feat(agent): fuzzy-filter skills_agent toolkit actions by task prompt Narrow large Composio toolkits (e.g. github ~500 actions) down to the handful relevant to a given delegation prompt before registering them as native tools on a spawned skills_agent. Falls back to the full catalogue when the filter yields fewer than MIN_CONFIDENT_HITS hits to avoid starving the sub-agent on under-specified prompts. Filter is only invoked when both `definition.id == "skills_agent"` and a `toolkit=` argument is present, so orchestrator and other sub-agents are unaffected. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test(composio): mock toolkits route in ops integration test --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
|
b2c74458d3 |
fix(voice): enable GPU detection and Metal acceleration for whisper (#558) (#571)
* fix(device): expand GPU detection with Intel Mac and NVIDIA probes (#558) Add Intel Mac detection (no Metal GPU for whisper), nvidia-smi probe for Windows/Linux NVIDIA GPUs, and diagnostic tracing at each decision point in detect_gpu(). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * build(cargo): enable whisper-rs Metal feature on macOS (#558) Add target-specific dependency for macOS that enables the `metal` feature on whisper-rs, compiling whisper.cpp with Metal GPU support. Cargo merges features from both declarations so non-macOS builds are unaffected. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(whisper): configure GPU params from device profile (#558) Accept has_gpu and gpu_description in load_engine() and explicitly set use_gpu and flash_attn on WhisperContextParameters instead of relying on the compile-time default. Log the selected acceleration backend at startup. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(local_ai): pass GPU info to whisper engine load paths (#558) Thread DeviceProfile has_gpu and gpu_description through both the bootstrap (startup) and speech (lazy) whisper engine load calls so the engine can configure Metal or CUDA acceleration at runtime. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
CodeGhost21andGitHub
|
13ce2cdcbf |
test(coverage): raise Rust unit test coverage to ≥80% on 10 critical modules (issue #530) (#567)
* test(coverage): raise 9 critical modules to ≥80% line coverage Pushes unit test coverage to meet issue #530's 80% target on 9 modules: socket (17% → 80%), credentials (46% → 81%), composio (55% → 81%), memory (75% → 80%), tools (73% → 83%), plus previously-passing cron, context, learning, embeddings. Additions span ~400+ new tests across 44 files: - Pure-function tests for types, schemas, parsers, URL builders, and error-classification helpers. - Mock-backend integration tests using axum for HTTP-dependent code (composio client + ops, socket ws_loop against a local WebSocket server). - End-to-end RPC handler tests using tempdir + OPENHUMAN_WORKSPACE env override (credentials, memory, config, cron). Also fixes 4 pre-existing flaky tests on macOS dev machines by pinning absolute paths (/bin/ls, /bin/sleep, /bin/sh, /usr/bin/touch) in cron scheduler tests so sh -lc does not pick up homebrew-shadowed binaries that macOS SIP refuses to execute, and relaxing a screen_intelligence capture-permission hang assertion. Remaining work: voice, config, screen_intelligence, channels, local_ai still below 80% (62%, 61%, 54%, 68%, 35% respectively). * test(coverage): add config module tests (62% → ~71%) Adds tests for: - config/ops.rs: env_flag_enabled, core_rpc_url_from_env, snapshot_config_json, workspace_onboarding_flag_exists/_set, set_browser_allow_all, agent_server_status - config/schemas.rs: catalog parity, all 21 registered schema keys, helpers - config/schema/proxy.rs: normalize_*, parse_proxy_scope, parse_proxy_enabled, validate_proxy_url, service_selector_matches, ProxyConfig defaults - config/schema/load.rs: apply_env_overrides for api_key, model, temperature, reasoning_enabled, web_search.*, storage.*; resolve_config_dir_for_workspace - config/settings_cli.rs: settings_section_json all sections (model/memory/runtime/browser/unknown) * test(coverage): config reaches 80.49%, socket back to 80.10% - config/ops.rs: apply_model/memory/runtime/browser/analytics/screen_intelligence_settings roundtrips via in-memory Config; load_and_apply_dictation/voice_server_settings activation-mode validation; get_dictation/voice_server/onboarding readers; workspace_onboarding_flag_set/resolve error and happy paths. - config/schema/load.rs: apply_env_overrides coverage for OPENHUMAN_MODEL, OPENHUMAN_TEMPERATURE (range clamp), OPENHUMAN_REASONING_ENABLED, OPENHUMAN_WEB_SEARCH_*, OPENHUMAN_STORAGE_* env branches. - config/schema/proxy.rs: normalize_* helpers, parse_proxy_scope/enabled, ProxyConfig defaults, validate_proxy_url schemes + hosts, service_selector_matches wildcards. - config/schemas.rs: every registered controller key, namespace parity. - config/settings_cli.rs: all section projections (model/memory/runtime/browser/unknown). - Shared `TEST_ENV_LOCK` at config::mod level so config::ops and config::schema::load test modules serialize OPENHUMAN_WORKSPACE mutations. - socket: a couple more schema assertions to keep module at 80%+. * test(coverage): channels partial push (68.12% → 68.91%) - channels/controllers/schemas.rs: every registered key resolves, required input coverage for describe/send_message/telegram_login_check. - channels/providers/web.rs: catalog parity, chat/cancel schema required inputs, unknown fallback, key_for, event_session_id_for stability, normalize_model_override trimming/empty, broadcast channel subscribe, field-builder helpers. - channels/providers/discord/api.rs: auth_header prefix, BotPermissionCheck serde with empty/full missing_permissions lists, permission bit flags are single-bit and distinct. Channels remains below the 80% issue target — the bulk of remaining uncovered code lives in telegram/channel.rs (574 lines), ops.rs (462), lark.rs (433) and runtime/startup.rs (350), which depend on live HTTP / socket / runtime bootstrap state that would require extensive mock infrastructure to exercise from unit tests. * test(coverage): partial push on local_ai (34→45%), screen_intelligence (54→62%), voice (62→63%) - local_ai/schemas.rs: catalog parity, every registered key resolves, field-builder helpers, deserialize_params happy/error, download_force optional. - local_ai/ops.rs: local-ai-disabled error paths for prompt/vision_prompt/ embed/summarize/transcribe/tts/chat; empty-messages rejection; suggestions return empty when disabled (graceful degradation). - local_ai/install.rs: find_system_ollama_binary env-override happy/missing/ empty cases; PATH-based lookup stub. - screen_intelligence/schemas.rs: catalog parity, all 15 registered keys resolve to non-unknown, unknown fallback. - screen_intelligence/ops.rs: accessibility_status/doctor_cli_json/ capture_image_ref/stop_session/vision_recent error-free behaviour. - voice/server.rs: truncate_for_log ellipsis + multibyte; try_global_server after init; additional hallucination-pattern coverage. Remaining gap on these modules lives almost entirely in: - local_ai/service/ollama_admin.rs (625 lines) — real HTTP + Ollama subprocess - local_ai/service/{assets,public_infer,vision_embed}.rs — same - voice/{server,audio_capture}.rs deep paths — audio hardware - screen_intelligence/{engine,processing_worker}.rs — active capture session * test(coverage): channels providers (+0.9%) — qq ensure_https, lark parsers - qq: ensure_https accept/reject, QQ_API_BASE/AUTH_URL constants, constructor. - lark: parse_post_content zh_cn/en_us fallback + links/mentions; invalid JSON returns None; strip_at_placeholders for @_user_N tokens; group should_respond_in_group mention gating. * test(coverage): push all 4 remaining modules - discord/api.rs: list_bot_guilds_at_base/list_guild_channels_at_base test seams with mock axum server; parse happy-path, error status, channel filter+sort, empty list. - channels/controllers/ops.rs: parse_allowed_users for string CSV/array/ newline/@-prefix/case-insensitive dedup/non-string; credential_provider; list_channels/describe_channel; connect_channel unknown-channel and non-object credentials. - local_ai/ollama_api.rs: `ollama_base_url()` honours OPENHUMAN_OLLAMA_BASE_URL env var so tests can point at mock servers; DEFAULT_OLLAMA_BASE_URL preserved. - local_ai/service/public_infer.rs: mock-backend tests for inference/prompt happy path, non-success status, suggest_questions parsing, disabled- local-ai short-circuits for summarize/prompt/suggest_questions/ inline_complete. - voice/schemas.rs: overlay_notify cancelled→released, unknown state errors, missing state errors, server_start handler, TranscribeParams + TtsParams deserialize happy/error paths, server_start all-optional invariant, description completeness. * test(coverage): local_ai HTTP mock tests (49→53%) - ollama_api: ollama_base_url() helper now used by ollama_admin/has_model + ollama_healthy (in addition to public_infer + vision_embed). - public_infer: inference against mock /api/generate happy/error/empty; suggest_questions parses line-separated output; disabled short-circuits for summarize/prompt/suggest/inline_complete. - vision_embed: mock /api/embed with /api/tags preflight; empty-input rejection; disabled short-circuits for embed and vision_prompt. - ollama_admin: has_model matches exact + prefixed tags; errors on 5xx /api/tags; ollama_healthy true on 200 and false on unreachable URL. * test(coverage): more local_ai + voice gains - local_ai/ollama_admin: diagnostics against mock Ollama (unreachable + missing models + all models present), list_models happy/error paths. - voice/dictation_listener: start_if_enabled early-returns for disabled/ empty-hotkey/unparseable-hotkey; normalize_hotkey_for_rdev coverage for Shift+Alt, lowercase, function keys, whitespace trimming. * test(coverage): local_ai schemas handlers + voice flakiness fix - local_ai/schemas: handle_device_profile; handle_presets tier+device shape; handle_apply_preset invalid/custom/valid paths; handle_set_ollama_path nonexistent/empty-to-clear paths. - voice/schemas: relax server_status/stop assertion to tolerate other tests in the same binary having initialised the global voice server (it's a OnceLock, so state is shared across the whole test process). * test(coverage): channels incremental push (71→73%) - presentation.rs: split_sentences, group_sentences, merge_short, segment_delay monotonic/bounded, is_structured_content detection, segment_for_delivery edge cases. - runtime/dispatch.rs: contains_any, starts_with_any, full coverage of select_acknowledgment_reaction across all 7 categories + deterministic + empty/single-char inputs. - commands.rs: doctor_channels with telegram/discord/slack/imessage/ multiple-config branches. - discord/api: check_channel_permissions mock-server tests — admin bypass, all-missing, everyone-allow, channel overwrite deny, member lookup failure. Added check_channel_permissions_at_base seam. - lark: should_refresh_last_recv, LarkChannel::new, is_user_allowed wildcard/empty allowlist, parse_event_payload edge cases (unsupported type / empty sender / missing event / post type). - email_channel: is_sender_allowed full matrix (empty/wildcard/exact/ @-prefix/bare-domain/subdomain-confusion), strip_html empty/tags-only/ unclosed/whitespace collapse. - voice/schemas: tolerate event interleaving in broadcast-channel test (schema bus is process-global). * test(coverage): address review findings — assertions, determinism, observability Tighten assertions so regressions surface: - credentials/ops: assert decrypt migrate path returns Ok - credentials/session_support: assert trimmed profile name directly - computer/mouse: check Err branch in single-axis scroll tests - filesystem/git_operations: assert exact error substrings and verify `git init` success before each test - channels/controllers/ops: exact credential_provider key + concrete parse_allowed_users expectation with accurate normalisation note; merged the three duplicate list/describe tests into existing coverage - tools/impl/browser/screenshot: cfg-branched support-matrix assertions - tools/impl/agent: replace misleading stub test with one that actually exercises dispatch_subagent's graceful-failure paths - local_ai/install: cfg-branched build_install_command expectations - local_ai/service/public_infer: exercise empty-response reject path via inference() (allow_empty=false) and assert the error - screen_intelligence: only take the macOS slow-path skip on macOS Test determinism: - local_ai/install: serialise env mutations via a module Mutex and RAII EnvGuard that restores prior values on drop - composio/ops: poll TCP readiness with backoff before returning the mock-backend URL - memory/global: bind TempDir at test scope so its workspace outlives any lazy-init reference Consistency: - local_ai/service/ollama_admin: route every Ollama HTTP call and the diagnostics URL through ollama_base_url(); drop the stale OLLAMA_BASE_URL import so /api/pull, /api/tags (runner check), /api/show, and the health message all honour the env override Observability: - local_ai/service/vision_embed: tracing::debug at entry/response and tracing::error on send/non-success - local_ai/service/ollama_admin::list_models: entry/response/parse logging including raw body on parse failure - channels/providers/discord/api: tracing::debug with endpoint, status, body, and context before every non-success bail! New coverage: - channels/providers/lark: anchor href-only fallback in parse_post_content - local_ai/ollama_api: five-case env-override suite for ollama_base_url (unset / normal / trimmed / trailing slashes / empty|whitespace) Miscellaneous: - voice/server: OnceCell-backed comment (was OnceLock); drop duplicate initial-status test; supply the HallucinationMode argument to two stale hallucination tests so the crate type-checks |
||
|
|
933c233704 |
fix(voice): add hallucination filter to chat voice path (#553) (#556)
* fix(voice): add hallucination filter to chat voice path and improve detector (#553) The chat voice transcription pipeline (ops.rs voice_transcribe_bytes) had no hallucination filtering, unlike the desktop dictation server which has had it since inception. This caused Whisper to inject repetitive garbage text ("it... it... it...", "Thank you. Thank you. Thank you.") into chat voice input, especially on short/silent recordings. Changes: - Extract hallucination detection into shared voice/hallucination.rs module - Add hallucination filter to voice_transcribe_bytes (chat voice path) - Improve detector: strip punctuation before word comparison, add dominant-word ratio check (>40%), catch "it... it... it..." patterns - Add 14 unit tests covering exact match, repetition, ratio, and legitimate speech cases Closes #553 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(voice): introduce HallucinationMode enum with split pattern lists Split monolithic HALLUCINATION_PATTERNS into ALWAYS_HALLUCINATION (blank-audio, YouTube phrases — filtered in all modes) and DICTATION_ONLY_PATTERNS (single-word noise like "yes", "okay" — filtered only in desktop dictation). Add repeating n-gram detection for looping phrases ("Thank you. Thank you. Thank you.") and raise dominant-word ratio from >40%/3 to >60%/5 to prevent false positives on emphatic speech like "no no no don't do that". Addresses CodeRabbit review on PR #556. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(voice): use Conversation mode for chat, Dictation mode for desktop Wire HallucinationMode into both voice paths: - ops.rs (chat voice): HallucinationMode::Conversation — conservative filtering allows short replies like "yes", "okay", "thank you" - server.rs (desktop dictation): HallucinationMode::Dictation — aggressive filtering drops single-word noise artifacts - Update server.rs hallucination_detection test for new signature Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(voice): redact user transcript content from debug logs Remove raw text interpolation (normalized, first, word, pattern) from hallucination detection debug logs to prevent leaking sensitive speech content. Retain non-PII metadata (repeat counts, ratios, n-gram length) for diagnostics. Addresses CodeRabbit review on PR #556. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
|
08d9fd2d4d |
fix(voice): recover buffered hotkey events after select! race (#527) (#545)
* fix(voice): return receiver count from publish_transcription publish_transcription now returns the number of active TRANSCRIPTION_BUS subscribers that received the message. When no receivers are connected (e.g. Socket.IO bridge not yet subscribed), the function logs a warning instead of silently discarding the broadcast. This makes it possible to diagnose "transcription produced but never delivered" scenarios from logs alone. Closes #527 (partial) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(voice): recover buffered hotkey events after select! race On warm CPAL init (second+ recording), audio_capture::start_recording() completes fast enough that the pending_ready branch and the hotkey_rx branch are both ready simultaneously inside tokio::select!. select! picks one pseudo-randomly — when it picks pending_ready, the Released event sits unprocessed in hotkey_rx. The recording is stored as "live" with no deferred stop, then the next loop iteration processes the buffered Released and stops the recording almost immediately, producing a near-zero-length clip that the duration gate silently drops. Fix: after pending_ready resolves, call hotkey_rx.try_recv() to check for a buffered stop event that lost the race. If found, apply the deferred stop mechanism (MIN_RECORDING_AFTER_SETUP = 1500ms) instead of treating the recording as live. Also adds pipeline_id (UUID prefix) to all process_recording_bg log lines for end-to-end correlation, and labels each pipeline stage (stop_recording, gate_duration, gate_silence, transcribe, deliver) so dropped recordings are diagnosable from logs. Closes #527 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Steven EnamakelandGitHub
|
4cf608c2be |
feat: native embeddings module with Ollama and vector store (#521)
* feat: replace fastembed with candle for local embeddings - Introduced a new `CandleEmbedding` provider using the `candle` ML framework, eliminating C++ dependencies and enhancing performance. - Updated configuration to default to `candle` for embedding provider settings, replacing the previous `fastembed` references. - Added new modules for embedding providers, including `candle_embed`, `noop`, and `openai`, to support various embedding strategies. - Enhanced the `EmbeddingProvider` interface to accommodate the new `CandleEmbedding` implementation. - Refactored related tests to ensure comprehensive coverage of the new embedding functionalities and maintain backward compatibility with existing configurations. * feat: update embedding provider to Ollama - Replaced the default embedding provider from Candle to Ollama, enhancing local embedding capabilities with improved model management and GPU acceleration. - Updated configuration defaults for embedding model and dimensions to align with Ollama specifications. - Introduced a new Ollama embedding module, including necessary constants and functionality for embedding requests. - Refactored related code and tests to ensure compatibility with the new provider, maintaining backward compatibility with existing configurations. * refactor: remove Candle embedding provider and related dependencies - Deleted the `CandleEmbedding` module and its associated files, streamlining the embedding provider architecture. - Updated `Cargo.toml` and `Cargo.lock` to remove references to Candle-related packages, ensuring a cleaner dependency tree. - Refactored the `EmbeddingProvider` interface to eliminate support for the Candle provider, maintaining compatibility with existing providers. - Adjusted tests and documentation to reflect the removal of the Candle embedding functionality, ensuring clarity and consistency across the codebase. * feat: add SQLite-backed vector store for embeddings - Introduced a new `store` module to implement a local vector store backed by SQLite, enabling efficient storage and retrieval of text embeddings. - Added functionality for inserting, updating, and searching embeddings using cosine similarity, enhancing the embedding management capabilities. - Updated the `mod.rs` file to include the new `store` module and expose relevant functions for cosine similarity and vector operations. - Enhanced documentation to provide usage examples and clarify the integration of the vector store with existing embedding providers. * Add fs2 dependency for improved file locking in ComposeIO trigger history * test: boost embeddings module coverage to 97%+ Add tests for batch insert mismatch error path, invalid metadata JSON handling, disk store parent directory creation, and fake embedding dimensions accessor. 95 tests total across the module, all files at 97-100% line coverage. * style: apply cargo fmt to embeddings module * feat: enhance embedding provider creation with error handling - Updated the `create_embedding_provider` function to return an `anyhow::Result`, allowing for immediate error reporting on unrecognized provider names. - Added support for a "none" provider that returns a no-op embedding. - Enhanced tests to cover new error handling paths and validate behavior for known and unknown providers, improving overall test coverage. * feat: enhance Ollama and OpenAI embedding providers with improved handling for blank inputs and response validation - Updated the `embed` method in both `OllamaEmbedding` and `OpenAiEmbedding` to skip blank inputs while preserving their positions in the output as zero-vectors. - Added validation to ensure the response count matches the input count, with appropriate error handling for dimension mismatches. - Enhanced tests to cover new behavior for blank inputs and response validation, ensuring robustness in embedding functionality. * fix: address PR review findings for embeddings module - Factory returns Result for unknown providers instead of silent noop - Ollama: preserve positional alignment when blank texts are filtered - Ollama: validate response count and dimensions before returning - OpenAI: strict parsing errors on non-numeric embedding values - OpenAI: skip Authorization header when api_key is empty - OpenAI: validate response count and dimensions - OpenAI/Ollama: add tracing at entry, success, and error paths - Store: add store_meta table to persist and validate embedding provider/dimensions on open — errors on dimension mismatch - Store: propagate row decode errors instead of filter_map(ok) - Store: add tracing to search, insert, delete, and count paths - Update factories.rs caller for Result-returning factory 101 tests pass, all files at 97%+ coverage. |
||
|
Steven EnamakelandGitHub
|
a115758c6c |
Improve local voice readiness and add Composio trigger history (#516)
* feat: enhance local AI asset state management and error handling - Updated the Home component to improve readiness checks for local AI assets, ensuring a more robust UI experience. - Refactored the LocalAiService to provide clearer state management for STT and TTS models, including handling on-demand downloads and error logging. - Enhanced the voice status function to accurately reflect the availability of transcription backends, improving overall system reliability. - Introduced warnings for on-demand model downloads to inform users about potential delays in functionality. * feat: implement polling for voice server status in OverlayApp - Added a polling mechanism to periodically check the voice server status every 2 seconds, ensuring the overlay remains in sync with the server state. - Introduced logic to activate or dismiss the speech-to-text (STT) mode based on the server's recording or transcribing state, enhancing user experience and responsiveness. - Utilized the existing `callCoreRpc` function to fetch server status, improving reliability in state management during brief disconnects or reconnections. * feat: refine prompt handling in LocalAiService - Updated the prompt construction logic in `LocalAiService` to enhance clarity and functionality. - When `no_think` is set, the system prompt now includes a directive for the model to respond with only the final answer, improving response accuracy. - Refactored the prompt and system parameters to ensure they are correctly formatted and passed to the `OllamaGenerateRequest`, enhancing overall request handling. * feat: enhance STT readiness checks in VoicePanel and useVoiceSkillStatus - Updated the STT readiness logic in `VoicePanel` to account for both 'ready' and 'ondemand' states, improving the accuracy of readiness assessments. - Refined the `useVoiceSkillStatus` hook to ensure it only blocks when the local AI's STT state is explicitly 'missing', enhancing overall system reliability and user experience. * feat(composio): implement ComposeIO trigger history component and hook - Added `ComposeioTriggerHistory` component to display a list of ComposeIO trigger events with formatted timestamps and payloads. - Introduced `useComposeioTriggerHistory` hook to manage fetching and state of trigger history entries, including error handling and loading states. - Updated `Webhooks` page to integrate the new component and hook, replacing previous webhook activity display with ComposeIO trigger history. - Created utility functions for formatting timestamps and payloads for better readability in the UI. - Established backend support for fetching trigger history through new Tauri commands, ensuring robust data handling and storage. * style: apply repo formatting fixes * feat: add fs2 dependency and enhance ComposeIO trigger history handling - Introduced the `fs2` crate to manage file locking, improving the reliability of file operations in the ComposeIO trigger history. - Updated `ComposioTriggerHistoryStore` to utilize exclusive file locks during archive writing, ensuring data integrity. - Enhanced error handling for file operations, providing clearer logging for failures related to file access and locking. - Refactored the initialization logic for global trigger history to prevent duplicate setups, improving overall stability. |
||
|
Steven EnamakelandGitHub
|
403f239ca5 |
refactor: remove QuickJS skills runtime (#508)
* refactor: remove quickjs skills runtime * style: apply repo formatting * refactor: clean up error reporting and connection handling - Removed the 'skill' source option from the error report structure to streamline error reporting. - Refactored the ConnectionsPanel component to simplify connection status badge rendering and improve clarity. - Updated the CronJobsPanel to enhance logging for cron job loading processes. - Adjusted SkillCard component to use a more consistent type for icons. - Deleted outdated end-to-end tests for Gmail and Notion skills, improving test suite maintainability. * fix: remove unnecessary ESLint disable comment in Conversations component - Cleaned up the Conversations component by removing the ESLint disable comment for exhaustive dependencies in the useEffect hook, improving code clarity and maintainability. * fix: remove unnecessary whitespace in Conversations component - Eliminated an extra line of whitespace in the Conversations component, enhancing code readability and maintainability. * refactor: streamline SkillCard imports for improved clarity - Combined import statements in the SkillCard component to enhance code readability and maintainability. |
||
|
Steven EnamakelandGitHub
|
73f8d1287a |
refactor: remove hardware-related components and streamline service management (#502)
* feat(config): introduce pre-login user directory structure - Added support for a pre-login user directory to encapsulate configuration, memory, and state before any user logs in. This ensures that all initial data is scoped under a dedicated user directory (`users/local`), preventing direct writes to the root `.openhuman` path. - Implemented the `pre_login_user_dir` function to return the appropriate path for the pre-login user. - Updated configuration loading logic to defer disk state creation until the first successful login, enhancing user data management and isolation. - Added tests to verify the correct behavior of the pre-login directory structure. * refactor: remove hardware-related components and streamline service management - Deleted hardware configuration and related tools from the codebase, including `HardwareConfig`, `HardwareTransport`, and associated memory management tools. - Introduced a new `service.ts` module for managing service and daemon commands, consolidating service-related functionalities. - Updated import paths across the application to reflect the removal of hardware references and the addition of the new service management module. - Refactored the `build_system_prompt` function to remove hardware access instructions, focusing on action instructions instead. - Cleaned up the Cargo.toml and Cargo.lock files by removing unused dependencies related to hardware management. * chore: apply formatting and tauri lockfile sync * refactor(tests): extract config file writing logic into a reusable function - Introduced a `write_config_file` function to encapsulate the logic for creating directories and writing configuration files, improving code reuse and readability. - Updated test cases to utilize the new function for writing configuration files, ensuring consistency and reducing duplication. - Added handling for pre-login user directory structure to ensure configuration is correctly written to the appropriate paths. |
||
|
Steven EnamakelandGitHub
|
57d307ad1e |
feat(agent): simplify harness + trim bundled prompts (#500)
* Add built-in agent definitions and prompts for orchestrator, planner, code executor, skills agent, researcher, critic, archivist, and tool maker - Introduced a new module for built-in agent definitions, allowing for easy addition of agents through a structured format. - Created TOML configuration files for each agent, detailing their properties such as id, display name, usage context, and tool specifications. - Developed corresponding prompt files that outline the responsibilities and rules for each agent, enhancing their functionality and usability. - Implemented a loading function to parse these definitions into usable agent structures, ensuring all agents are correctly initialized and validated. * Remove deprecated archetypes and related components from the multi-agent harness - Deleted the `archetypes.rs`, `dag.rs`, `executor.rs`, and `types.rs` files, which contained the definitions and implementations for agent archetypes, task DAGs, and execution logic. - Updated the `builtin_definitions.rs` and `definition.rs` files to remove references to the now-removed archetypes, streamlining the agent definition process. - Refactored the `mod.rs` file to eliminate unused modules and clarify the structure of the multi-agent harness. - Adjusted comments and documentation to reflect the removal of the DAG orchestration flow, emphasizing the current sub-agent delegation model. * Remove unused `ArchetypeConfig` from schema exports in `mod.rs` to streamline configuration management. * Implement context guard and memory management features in the harness module - Introduced `ContextGuard` to monitor context utilization and trigger auto-compaction when usage exceeds defined thresholds. - Added `scrub_credentials` function to sanitize sensitive information from tool outputs. - Implemented history management functions to trim conversation history and build compaction transcripts. - Created `build_tool_instructions` to generate tool usage guidelines for the system prompt. - Developed `memory_context` functions to manage user memory and relevant context for conversations. - Enhanced the `tool_loop` to integrate the new context management and tool invocation logic. This update improves the agent's ability to manage context effectively, ensuring efficient memory usage and secure handling of sensitive data. * Refactor agent module imports to use the harness instead of loop_ - Updated import paths in `dispatcher.rs`, `memory_loader.rs`, `mod.rs`, `dispatch.rs`, and `startup.rs` to replace references from the `loop_` module to the `harness` module. - This change enhances module organization and aligns with the recent architectural updates in the agent structure. * Remove cost tracking and context assembly modules from the agent structure - Deleted `cost.rs` and `context_assembly.rs` files, which handled token cost tracking and context assembly for the multi-agent harness, respectively. - Updated `mod.rs` files to remove references to the deleted modules, streamlining the agent's organization and focusing on essential components. - This cleanup enhances maintainability and aligns with the current architectural direction of the project. * Remove identity module and related configurations from the agent structure - Deleted the `identity.rs` file, which contained the AIEOS identity handling logic and related structures. - Updated `mod.rs` and other files to remove references to the deleted identity module, streamlining the agent's organization. - This change simplifies the codebase and aligns with the decision to rely solely on OpenClaw markdown files for identity management. * Enhance agent prompts with improved formatting and clarity - Added spacing for better readability in the prompts of the Archivist, Code Executor, Critic, Orchestrator, Researcher, Skills Agent, and Tool Maker agents. - Updated the table formatting in the Orchestrator prompt for consistency and clarity. - These changes improve the overall presentation and usability of agent documentation, making it easier for users to understand agent responsibilities and rules. * Remove REPL functionality and related components from the agent structure - Deleted the `repl.rs` file, which contained the implementation for the interactive REPL. - Removed references to the REPL in `cli.rs`, `mod.rs`, and other related files, streamlining the agent's organization. - Eliminated unused REPL session handling logic from the agent schemas and local AI operations, enhancing maintainability and focusing on essential components. - This cleanup aligns with the current architectural direction of the project, simplifying the codebase. * Refactor imports and clean up configuration exports - Updated import paths in `startup.rs` to include `host_runtime` from the correct module. - Streamlined the export statements in `mod.rs` and `schema/mod.rs` for better organization and clarity. - Removed unnecessary line breaks in the export lists to enhance readability and maintainability of the configuration schema. * Remove unused history management functions and clean up agent module exports - Deleted the `history.rs` and `session.rs` files, which contained functions for managing conversation history and session handling. - Updated `mod.rs` to remove references to the deleted modules and streamline the agent's organization. - Cleaned up import statements in `mod.rs` and other files to enhance clarity and maintainability of the codebase. * Add agent harness module with builder, runtime, and turn management - Introduced a new `harness` module for the `Agent`, encapsulating the `AgentBuilder`, runtime accessors, and turn lifecycle management. - Implemented the `AgentBuilder` fluent API for constructing `Agent` instances with customizable configurations. - Developed the `turn` method to handle user interactions, tool execution, and context management within the agent. - Created a `runtime` module for public accessors and utility functions related to agent operations. - Added comprehensive unit and integration tests to ensure the functionality of the new agent structure and its components. * Remove classifier and traits modules from the agent structure - Deleted the `classifier.rs` and `traits.rs` files, which contained the classification logic and core agent traits, respectively. - Updated `mod.rs` to remove references to the deleted modules, streamlining the agent's organization. - This cleanup enhances maintainability and focuses on essential components of the agent architecture. * Refactor agent prompt structure and remove unused files - Updated the `prompt.rs` file to streamline the orchestrator's logic by removing unnecessary tool documentation references and simplifying the file list. - Deleted several prompt files (`AGENTS.md`, `BOOTSTRAP.md`, `CONSCIOUS_LOOP.md`, `MEMORY.md`, `README.md`, `TOOLS.md`) to declutter the project and focus on essential components. - Adjusted the `harness/mod.rs` file to change module visibility from public to crate-level for better encapsulation. - These changes enhance maintainability and align with the current architectural direction of the project. * Refactor bootstrap file handling and update tests - Simplified the list of bundled prompt files in `prompt.rs` by removing references to `AGENTS.md` and `TOOLS.md`, focusing on essential identity files. - Adjusted the logic for injecting `MEMORY.md` to be optional, ensuring it only appears if it exists. - Updated test cases in `common.rs`, `identity.rs`, and `prompt.rs` to reflect the changes in the bootstrap file structure and ensure accurate validation of the new logic. - These modifications enhance clarity and maintainability of the codebase while aligning with the current project architecture. * Implement agent delegation tools and browser automation features - Introduced new tools for agent delegation, including `ArchetypeDelegationTool`, `AskClarificationTool`, `DelegateTool`, and `SkillDelegationTool`, enhancing the agent's ability to manage tasks through specialized sub-agents. - Added `SpawnSubagentTool` for delegating tasks to sub-agents, allowing for more complex workflows and improved task management. - Implemented browser automation capabilities with `BrowserOpenTool`, enabling secure opening of approved URLs in the Brave Browser. - Enhanced the `BrowserTool` with pluggable backends for improved automation and user interaction. - Added `ImageInfoTool` for extracting metadata from images, supporting future multimodal capabilities. - Organized tools into a new `impl` module structure for better maintainability and clarity in the codebase. * Refactor imports and clean up tool implementations - Updated import statements across various tool implementations to ensure consistency and clarity, particularly in the `traits.rs` file. - Removed unnecessary line breaks and adjusted module paths for better organization. - Cleaned up test files by removing trailing whitespace, enhancing code readability and maintainability. - These changes streamline the codebase and improve the overall structure of tool-related components. * Update built-in definitions and prompt handling for clarity and consistency - Revised the test for built-in agent definitions to dynamically reference the length of `BUILTINS`, enhancing maintainability. - Clarified documentation for the `system_prompt` field in `AgentDefinition`, emphasizing the default behavior and usage of inline prompts. - Simplified the `build_system_prompt` function by removing unnecessary conditional logic, ensuring a more straightforward return of the prompt string. * Enhance agent definition loading and documentation clarity - Updated the `load_file` function to reject definitions with missing or empty `system_prompt`, ensuring custom definitions are properly validated. - Added a new test to verify that definitions lacking a `system_prompt` are correctly rejected, improving robustness. - Clarified documentation for built-in definitions and TOML parsing, emphasizing compile-time guarantees and runtime checks. - Improved the logic for checking the existence of `MEMORY.md` to prevent errors from stray directories, enhancing file handling reliability. |
||
|
Steven EnamakelandGitHub
|
31297ad19d |
refactor(memory): remove GLiNER/GLiREL ingestion phase (#499)
* refactor(memory): replace GLiNER model with heuristic extraction - Removed GLiNER-related code and dependencies from the memory ingestion pipeline, transitioning to a heuristic-only extraction approach. - Updated documentation and comments to reflect changes in extraction methods. - Adjusted tests to ensure compatibility with the new heuristic extraction configuration. - Bumped version of the tokenizers dependency and updated Cargo.lock accordingly. * chore: apply formatting and tauri lockfile sync |
||
|
|
0cd0f7a670 |
feat(voice): sync overlay orb with chat voice button state (#487) (#490)
* feat(voice): sync overlay orb with chat voice button state (#487) The overlay orb already reacts to hotkey-based dictation via Socket.IO events, but the chat "Start Talking" button used local React state only. Add a new RPC method `openhuman.overlay_stt_notify` that the chat button calls at each voice state transition, which publishes to the existing DICTATION_BUS / TRANSCRIPTION_BUS broadcast channels — so the overlay reflects recording/transcribing/idle from both input paths with zero changes to the Socket.IO bridge or overlay event handlers. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(voice): address CI formatting and CodeRabbit review feedback - Run cargo fmt and prettier to fix formatting violations - Use typed enum OverlaySttState instead of raw String for state param (serde rejects invalid states at deserialization, eliminating the unknown state branch) - Require `text` field for transcription_done state (return error if missing instead of silently ignoring) - Replace raw transcript logging with metadata-only (has_text, text_len) to avoid logging sensitive user speech content - Use "Voice input active" aria-label (covers recording + linger phases) - Convert notifyOverlaySttState to arrow function with async/await per repo TS conventions Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Steven EnamakelandGitHub
|
30eec2ad88 |
docs: comprehensive documentation for core Rust modules (#470)
* feat(core): enhance RPC controller and dispatch logic - Added comprehensive documentation for the core RPC controller and dispatch modules, detailing their purpose and functionality. - Introduced new functions for managing registered controllers and schemas, including validation and invocation methods. - Improved the structure of the `RpcOutcome` type to standardize response formats across domain-specific handlers. - Enhanced the local AI operations module with additional functionalities for agent interactions, model management, and audio processing. - Updated the dispatcher to better route RPC calls to their respective handlers, ensuring a more robust and maintainable architecture. These changes improve the clarity and usability of the RPC system, facilitating easier integration and interaction with various components of the OpenHuman platform. * docs: comprehensive documentation for core Rust modules * chore(dependencies): update OpenHuman to version 0.52.0 in Cargo.lock and add knip dependency in package.json |
||
|
Steven EnamakelandGitHub
|
acc6246e59 |
Refactor core-polled app state and screen intelligence status (#464)
* refactor(accessibility): remove device control and predictive input features from accessibility settings - Updated accessibility-related components and tests to eliminate device control and predictive input features. - Adjusted AccessibilityPanel and ScreenIntelligencePanel to reflect the removal of these features. - Modified related tests to ensure consistency with the updated accessibility status structure. - Cleaned up accessibility session parameters and state management to focus solely on screen monitoring. * refactor(accessibility): streamline featureOverrides state initialization - Simplified the initialization of featureOverrides state in AccessibilityPanel and ScreenIntelligencePanel components for better readability. - Consolidated parameter definitions in startAccessibilitySession to enhance clarity and maintainability. - Removed unnecessary re-exports in the screen_intelligence engine module to clean up the codebase. * chore(dependencies): update OpenHuman to version 0.51.19 in Cargo.lock * chore(dependencies): update OpenHuman version to 0.51.19 in Cargo.lock * feat(restart): implement core process restart functionality - Added a new `SystemRestartRequested` event to the `DomainEvent` enum to handle restart requests. - Introduced a `RestartSubscriber` that listens for restart events and manages the process respawn. - Created a `service_restart` function to publish restart requests via the event bus. - Updated service schemas to include a new `restart` controller with parameters for source and reason. - Enhanced documentation to reflect changes in behavior and added necessary code comments. * feat(accessibility): add last restart summary to Screen Intelligence Panel - Introduced `lastRestartSummary` to the accessibility state and updated relevant components to display the last successful core restart information. - Modified `PermissionsSection` and `ScreenIntelligencePanel` to include the new summary. - Updated tests to validate the display of the last restart summary and ensure proper state management during core restarts. - Refactored accessibility slice to handle the new restart summary in state updates. * feat(core): enhance startup process with restart delay and subscriber registration - Added a call to apply startup restart delay from environment variables in `run_core_from_args`. - Updated the `bootstrap_skill_runtime` function to register a `RestartSubscriber` for handling restart requests, ensuring consistent respawn logic across triggers. - Introduced a new `core_process` field in the `AccessibilityEngine` to track the core process status, including its PID and start time. - Implemented a helper function to capture the core process start time using `OnceLock` for efficient initialization. * feat(screen-intelligence): refactor accessibility state management and UI components - Replaced direct Redux state access with a new `useScreenIntelligenceState` hook across multiple components, including `AccessibilityPanel`, `ScreenIntelligencePanel`, and their respective subcomponents. - Streamlined permission and session handling by consolidating related functions and removing unnecessary dispatch calls. - Updated tests to mock the new state management approach, ensuring consistent behavior and validation of UI elements. - Removed the `SessionAndVisionSection` component to simplify the structure and improve maintainability. - Introduced a new API file for screen intelligence to encapsulate related functionality and improve code organization. * refactor(tests): clean up and optimize test files for accessibility and screen intelligence panels - Removed redundant imports and streamlined the structure of test files for `AccessibilityPanel` and `ScreenIntelligencePanel`. - Consolidated core process state initialization in test mocks for better readability. - Updated dependency imports and ensured consistent mocking of state management hooks across tests. - Enhanced the `ScreenPermissionsStep` component by improving the dependency array in the useEffect hook for better performance. * refactor(store): remove unused authentication and user management code - Deleted the `UserProvider`, `authSlice`, `authSelectors`, `userSlice`, `teamSlice`, and related test files to streamline the codebase. - This cleanup enhances maintainability by removing legacy code that is no longer in use. - Updated the store configuration to reflect the removal of these slices and ensure proper state management. * refactor(webhooks): reorganize types and remove legacy state management - Moved `TunnelRegistration` and `WebhookActivityEntry` types to a new `types.ts` file for better organization. - Updated imports in `TunnelList` and `WebhookActivity` components to reference the new types location. - Refactored `useWebhooks` hook to eliminate Redux state management in favor of local state, enhancing performance and reducing complexity. - Removed unused `aiSlice`, `inviteSlice`, and `webhooksSlice` along with their associated tests to streamline the codebase. * refactor(daemon): migrate state management from Redux to a custom store - Introduced a new `store.ts` file to manage daemon state, replacing the previous Redux slice. - Updated components and hooks to utilize the new state management approach, enhancing performance and reducing complexity. - Removed the legacy `daemonSlice` and associated Redux logic, streamlining the codebase. - Adjusted imports in various components and hooks to reference the new store structure. * refactor(screen-intelligence): integrate core state management and enhance status handling - Replaced direct state management in `useScreenIntelligenceState` with a new core state approach, utilizing `useCoreState` for improved performance and consistency. - Updated status fetching and permission handling to leverage the core state snapshot, streamlining the logic and reducing redundant API calls. - Introduced a new `CoreRuntimeSnapshot` interface to encapsulate runtime statuses, including screen intelligence, local AI, autocomplete, and service states. - Adjusted related components and hooks to align with the new state management structure, enhancing maintainability and readability. - Updated tests to validate the new runtime state structure and ensure proper functionality across the application. * refactor(components): reorganize imports and streamline function formatting - Moved the import of `Tunnel` and `tunnelsApi` in `TunnelList.tsx` for better organization. - Reformatted function definitions in `store.ts`, `useDaemonHealth.ts`, `useDaemonLifecycle.ts`, `useWebhooks.ts` for improved readability. - Cleaned up the structure of test files in `coreRpcClient.test.ts` by consolidating object properties for clarity. - These changes enhance code maintainability and readability across the application. * test(screen-intelligence): fix duplicate hook imports * fix(tests): update ScreenIntelligenceDebugPanel test to use baseState for refresh status and vision calls * refactor(invites): simplify error message rendering in Invites component - Consolidated the conditional rendering of the load error message in the Invites component for improved readability. - This change enhances the clarity of the code without altering functionality. * refactor(daemon): streamline state management and function definitions - Removed the `healthTimeoutId` from the `DaemonUserState` interface and related functions to simplify state management. - Converted several functions in `store.ts` to arrow function syntax for consistency and improved readability. - Updated the `Invites` component to handle asynchronous loading and error states more effectively, ensuring that in-flight requests are properly managed. - Refactored the `CoreStateProvider` to enhance the refresh logic and prevent multiple simultaneous refreshes. - Introduced a new `register_domain_subscribers` function in `jsonrpc.rs` to centralize event bus subscriber registration, improving code organization and maintainability. * fix: add debug logging, atomic restart guard, and idempotent subscriber registration - CoreStateProvider: add namespaced debug logger for polling failure diagnostics - service/bus.rs: add AtomicBool gate to prevent duplicate restart spawns - service/bus.rs: use OnceLock for idempotent RestartSubscriber registration - Invites.tsx: add debug log in loadInviteCodes catch block * style: apply prettier formatting to CoreStateProvider * fix: sanitize error logging, serialize refresh, and demote restart logs - CoreStateProvider: sanitize error objects in poll failure logs to avoid leaking tokens/headers - CoreStateProvider: move in-flight guard into refresh() via shared promise so all callers (poll, updateLocalState, storeSessionToken) are serialized - CoreStateProvider: log refreshTeams errors instead of swallowing them - service/bus.rs: demote duplicate-restart log to debug, omit reason from log output to avoid free-form text emission * style: apply cargo fmt to service/bus.rs |
||
|
Mega MindandGitHub
|
371bcd34ea |
Feat/chat issue (#441)
* feat: display app version in settings panel * fix(onboarding): auto-refresh accessibility state after grant (#351) * style(onboarding): apply formatter for issue #351 fix * fix(onboarding): ESLint + typed mock for ScreenPermissionsStep; clear flag in handler Consolidate tauriCommands imports and drop redundant mock cast. Handle granted accessibility in focus/visibility callback instead of a follow-up effect. Made-with: Cursor * feat(env): add support for custom dotenv path and update dependencies - Introduced an optional environment variable `OPENHUMAN_DOTENV_PATH` to specify a custom path for dotenv files, enhancing configuration flexibility. - Updated `Cargo.toml` to include the `dotenvy` dependency for improved dotenv file handling. - Enhanced the `.env.example` file with a new comment for the custom dotenv path. - Added data-testid attributes and button types in `SkillDebugModal` and `Skills` components for better testability. - Created new tests for Gmail and Notion third-party skills to ensure proper functionality of sync and debug tools. - Added documentation for memory sync functions to clarify usage patterns and function details. * fix: address CodeRabbit review on PR #441 - Dotenv: treat empty OPENHUMAN_DOTENV_PATH as unset; propagate from_path errors - Document OPENHUMAN_DOTENV_PATH parent-env requirement in .env.example - Memory docs: MD040 fence language; clarify skill namespace vs integration id - QuickJS bootstrap: modern helpers, generic platform.notify log, template URLs - Skills UI: type=button on close/settings; async waitFor in sync tests - Gmail OAuth e2e: workspace env matches MemoryClient; env/engine drop guards; redact secrets from logs - Add replace_global_engine for test teardown Made-with: Cursor |
||
|
Steven EnamakelandGitHub
|
3851d1ef67 |
fix(voice): anti-hallucination, clipboard paste, Fn key reliability (#380)
* fix(dictation): update hotkey default value and documentation - Changed the default global hotkey for dictation from "CmdOrCtrl+Shift+D" to "Fn" in both the configuration schema and the associated documentation. - Updated the hotkey parsing function to recognize "Fn" as a valid key, enhancing the flexibility of hotkey configurations. - Added a test case to ensure the "Fn" key can be parsed correctly, improving the robustness of the hotkey handling functionality. * fix(voice): update default activation mode and hotkey in configuration - Changed the default activation mode for voice and dictation from "tap" to "push" in the respective configuration schemas. - Updated the default hotkey for voice commands from "ctrl+shift+space" to "Fn" across various modules and documentation. - Adjusted related tests to reflect the new defaults, ensuring consistency in behavior and expectations. * feat(voice): integrate embedded global voice server startup - Added a new asynchronous function `start_if_enabled` to the voice server module, which initializes the embedded voice server based on configuration settings. - Updated the server run logic to check if the voice server should auto-start, enhancing the startup process for the core application. - Integrated the new server startup function into the main server run logic, ensuring the voice server is launched if enabled in the configuration. * feat(voice): add VoicePanel for managing voice server settings - Introduced a new `VoicePanel` component to handle voice server configurations, including startup options, hotkeys, and runtime controls. - Updated routing in the settings page to include the new voice settings section. - Enhanced the `useSettingsNavigation` hook to support navigation to the voice settings. - Added tests for the `VoicePanel` to ensure functionality and reliability of the voice server management features. * refactor(dictation): update documentation and improve component initialization - Revised comments in `DictationHotkeyManager` to clarify the component's mounting process within the app tree. - Removed unused imports and unnecessary state management from `ServiceBlockingGate`, streamlining the component's logic. - Updated tests for `ServiceBlockingGate` to reflect changes in behavior, ensuring accurate rendering of child components based on service status. - Enhanced the `Cargo.lock` file by updating dependencies to their latest versions for improved stability and security. * fix(voice): update default skip_cleanup setting and enhance VoicePanel options - Changed the default value of `skip_cleanup` in the voice server configuration from `false` to `true` to improve transcription handling. - Reordered options in the `VoicePanel` component to ensure "Natural cleanup" is displayed alongside "Verbatim transcription" for better user clarity. - Updated tests to reflect the new default settings and ensure proper functionality of the VoicePanel component. * feat(window): add window management commands for Tauri application - Introduced a new module `window.ts` containing functions for managing window visibility and state in a Tauri application. - Implemented commands to show, hide, toggle visibility, minimize, maximize, close, and set the title of the main window. - Added checks to ensure commands are only executed in a Tauri environment, enhancing compatibility with web contexts. * feat(tauriCommands): add comprehensive Tauri command modules - Introduced multiple new modules for Tauri commands, including `accessibility`, `autocomplete`, `config`, `conscious`, `core`, `cron`, `hardware`, `localAi`, and `window`. - Each module contains functions for managing specific functionalities such as accessibility permissions, autocomplete suggestions, configuration settings, and hardware interactions. - Implemented checks to ensure commands are executed only in a Tauri environment, enhancing compatibility and reliability. - This addition significantly expands the command capabilities of the Tauri application, providing a robust framework for future development. * feat(voice): enhance audio transcription with initial prompt support - Added functionality to transcribe audio with an optional initial prompt, allowing for vocabulary bias and improved conversational continuity. - Updated the `transcribe_pcm_f32` and `transcribe_wav_file` functions to accept an `initial_prompt` parameter, enhancing recognition of specific vocabulary. - Implemented peak RMS energy tracking during audio recording for silence detection, ensuring recordings below a defined threshold are skipped. - Enhanced the voice server configuration to include a silence threshold and custom dictionary for better transcription context. - Introduced methods to build and manage recent transcripts for improved continuity across consecutive recordings. - Updated tests to validate new features and ensure proper functionality of the transcription process. * feat(voice): enhance VoiceServerConfig with silence detection and custom dictionary - Added `silence_threshold` to the `VoiceServerConfig` for improved silence detection, allowing recordings with low RMS energy to be skipped. - Introduced `custom_dictionary` to bias transcription towards specific vocabulary, enhancing recognition of names and technical terms. - Updated the `voice_transcribe` and `voice_transcribe_bytes` functions to utilize the new `initial_prompt` parameter for better context during transcription. - Adjusted the `transcribe_pcm_i16` function to accept additional parameters for improved flexibility in handling audio input. * feat(voice): add silence threshold and custom dictionary features to VoicePanel - Implemented a new input for setting the silence threshold, allowing recordings with low RMS energy to be skipped. - Added functionality for a custom dictionary, enabling users to add specific vocabulary words to improve transcription accuracy. - Updated the VoiceServerSettings interface and related functions to support the new features, ensuring seamless integration with existing settings. - Enhanced the UI in the VoicePanel to facilitate user interaction with the new settings. * feat(voice): propagate silence threshold and custom dictionary to voice server command - Added `silence_threshold` and `custom_dictionary` parameters to the `run_voice_server_command` function, ensuring these settings are utilized during voice server operations. - Enhanced integration with the existing voice server configuration to support improved transcription accuracy and silence detection. * fix(tauriCommands): update import paths for coreRpcClient - Adjusted import paths for `callCoreRpc` in multiple Tauri command modules to ensure correct referencing from the updated directory structure. - This change enhances module organization and maintains consistency across the codebase. * feat(dependencies): update Cargo.lock and Cargo.toml for new packages and versions - Added new dependencies including `arboard`, `fax`, `fax_derive`, `gethostname`, `half`, `quick-error`, `tiff`, and `x11rb` to enhance functionality and support for clipboard operations, transcription improvements, and system interactions. - Updated existing dependencies to their latest versions for better performance and compatibility. - Modified `VoicePanel` to utilize the updated settings and ensure proper handling of voice server configurations. - Enhanced the text input mechanism to use clipboard-paste for improved reliability in text insertion. * fix(voice): update skip_cleanup default value and enhance logging - Changed the default value of `skip_cleanup` in `VoiceServerConfig` from `true` to `false` to align with expected behavior and improve transcription handling. - Added detailed logging in the transcription cleanup process to provide better insights into the LLM state and cleanup decisions. - Removed unused functions related to unreliable key releases in hotkey handling to simplify the codebase. - Updated tests to reflect the new default settings for `skip_cleanup` and ensure proper functionality across components. * style: apply linter formatting fixes * fix(voice): remove unused warn import in hotkey module * test(voice): add silence threshold and custom dictionary to VoicePanel tests - Updated tests for the VoicePanel component to include new parameters: `silence_threshold` and `custom_dictionary`. - Ensured that the tests reflect the latest configuration settings for improved transcription accuracy and functionality. |
||
|
|
11f718d8bc |
feat(voice): standalone voice dictation server with hotkey support (#368)
* feat: add standalone voice dictation server with hotkey support - Introduced a new `voice` subcommand to the CLI for running a standalone voice dictation server that listens for a hotkey, records audio, transcribes it using Whisper, and inserts the result into the active text field. - Implemented configuration options for the voice server, including hotkey combination, activation mode (tap or push), and an option to skip LLM post-processing. - Added audio capture functionality using the `cpal` crate and integrated hotkey listening with the `rdev` crate for global key event handling. - Enhanced the configuration schema to include voice server settings and updated the main configuration structure accordingly. - Updated relevant modules and tests to ensure consistent behavior and functionality across the application. This feature enhances user interaction by allowing voice dictation directly into any active text field, improving accessibility and usability. * feat: add voice dictation server with hotkey support - Introduced a standalone voice dictation server that listens for a configurable hotkey to start recording audio, transcribes it using whisper, and inserts the transcribed text into the active text field. - Added CLI support for the `voice` command, allowing users to manage the voice server's configuration, including hotkey and activation mode settings. - Implemented configuration structures for the voice server, including options for automatic start, hotkey combination, activation mode, and cleanup behavior. - Enhanced audio capture functionality using the `cpal` library for microphone input and integrated text insertion using the `enigo` library for simulating keyboard input. - Updated relevant modules and schemas to support the new voice server features, ensuring a cohesive integration within the OpenHuman platform. * refactor: streamline voice server command and enhance audio capture functionality - Updated the `run_voice_server_command` function to initialize the configuration with environment overrides instead of loading from a file, improving performance and flexibility. - Refactored the audio capture logic in `start_recording` to enhance thread management and error handling, ensuring a more robust audio stream setup. - Improved the handling of audio stream creation and playback, ensuring that all cpal objects are managed on the same thread as required, enhancing stability during recording operations. * fix: remove unused import in voice server module - Eliminated the `HotkeyListenerHandle` import from the `server.rs` file, streamlining the code and improving clarity by removing unnecessary dependencies. * feat(voice): auto-enable LLM cleanup when local model is ready The postprocessor now checks the local LLM state and automatically enables transcription cleanup when the model is downloaded and ready, even if not explicitly configured. Falls back gracefully to raw text when the LLM is unavailable. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: apply cargo fmt + prettier formatting Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(voice): dictation config, hotkey lifecycle, and WebSocket streaming (#332) Add the foundational infrastructure for voice dictation (EPIC #332): **Rust core:** - New `DictationConfig` schema with serde defaults and env var overrides (enabled, hotkey, activation_mode, llm_refinement, streaming, interval) - RPC controllers: `config_get_dictation_settings` / `config_update_dictation_settings` - WebSocket endpoint `/ws/dictation` for streaming PCM16 transcription with periodic partial inference and final LLM refinement - Microphone permission declaration (`NSMicrophoneUsageDescription`) in Tauri macOS bundle config **Frontend:** - `useDictationHotkey` hook: fetches config from core RPC, auto-registers global hotkey, listens for `dictation://toggle` events - `DictationHotkeyManager` headless component mounted in App.tsx - Fix voice RPC response type mismatch: voice handlers return flat results (no `{result, logs}` wrapper), so remove incorrect `CommandResponse<T>` wrapping from `openhumanVoiceStatus`, `openhumanVoiceTranscribe`, `openhumanVoiceTranscribeBytes`, and `openhumanVoiceTts` Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(tauri): remove invalid infoPlist config that breaks tauri dev The `infoPlist` field in tauri.conf.json expects a string path, not an inline object. Remove it for now — microphone permission will be added via a proper Info.plist supplement in the production build pipeline. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * style: apply Prettier formatting to dictation files Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * format files * feat(dictation): integrate dictation listener and event broadcasting - Added a global dictation hotkey listener that activates based on configuration. - Implemented a web channel bridge to handle dictation events and broadcast them to connected clients. - Updated the voice module to include the new dictation listener functionality. This enhances the voice dictation capabilities by ensuring real-time event handling and client communication. * update code * format * feat(voice): enhance voice server configuration and functionality - Updated `Cargo.toml` to mark voice-related dependencies as optional. - Introduced `VoiceActivationMode` enum for better control over voice server activation. - Refactored voice server command handling and dictation event broadcasting to support new features. - Added conditional compilation for voice features across various modules, ensuring they are only included when enabled. This commit improves the modularity and configurability of the voice server, allowing for more flexible integration and usage. * refactor: clean up whitespace and formatting in core and voice modules - Removed unnecessary blank lines in `cli.rs`, `jsonrpc.rs`, `schemas.rs`, and `socketio.rs` to improve code readability. - Adjusted import order in `mod.rs` for better organization. This commit enhances the overall code quality by ensuring consistent formatting across multiple files. * chore: update Dockerfile and test workflow to install additional system dependencies - Added installation of system dependencies (cmake, ALSA, X11) in the Dockerfile for improved build support. - Updated the GitHub Actions workflow to reflect the new dependencies, ensuring consistent environment setup for testing. This commit enhances the build environment by including necessary libraries for audio and GUI support. * format * fix claude * format --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: oxoxDev <nikhil@tinyhumans.ai> |
||
|
sanil-23andGitHub
|
d73ccda13d |
fix: patch whisper-rs-sys for Windows MSVC static CRT (/MT) (#276)
The upstream whisper-rs-sys builds whisper.cpp via CMake which defaults to /MD (dynamic CRT), but Rust and all other C deps use /MT (static CRT). This causes LNK2038/LNK1169 linker errors on Windows. Patch whisper-rs-sys from tinyhumansai/whisper-rs-sys fork which adds config.static_crt(true) and overrides all per-config CMake flags (Debug/Release/MinSizeRel/RelWithDebInfo) from /MD to /MT. Closes #273 |
||
|
|
2545ae374a |
refactor: extract accessibility middleware module (#184)
* feat(autocomplete): add target_role to EngineState for improved context validation - Introduced a new field `target_role` in `EngineState` to store the AXRole of the text element when suggestions are generated. - Updated the `AutocompleteEngine` to validate the focused element against the expected app and role before applying suggestions. - Enhanced the `apply_text_to_focused_field` function to include role validation, ensuring better accuracy in text insertion. - Improved the handling of suggestion context and error states during autocomplete operations. This update enhances the reliability of the autocomplete feature by ensuring that suggestions are applied only when the correct context is maintained. * fix(package): update dev:app script to include core staging step - Modified the `dev:app` script in `package.json` to run `yarn core:stage` before executing `tauri dev`, ensuring that the core sidecar is properly staged during development. - This change enhances the development workflow by automating the staging process, reducing manual steps for developers. This update improves the reliability of the development environment setup. * feat(autocomplete): implement core autocomplete engine and supporting modules - Introduced a new `AutocompleteEngine` struct to manage the state and operations of the autocomplete feature, including starting, stopping, and refreshing suggestions. - Added `EngineState` to track the current status, phase, and context of the autocomplete process. - Implemented helper functions for managing focus and overlay notifications, enhancing user interaction with suggestions. - Created utility functions for terminal context extraction and text sanitization, improving the accuracy of suggestions. - Developed a comprehensive set of types and structures to support autocomplete operations, including suggestion handling and status reporting. This update lays the foundation for a robust autocomplete feature, enhancing user experience through improved context awareness and interaction. * refactor(autocomplete): remove unused functions and improve clipboard handling - Deleted the `normalize_ax_value` and `parse_ax_number` functions as they were not utilized in the codebase, streamlining the autocomplete module. - Enhanced the `clipboard_save` function to improve readability by formatting the string conversion of clipboard output, ensuring better handling of empty or "missing value" cases. This update simplifies the code and improves the overall maintainability of the autocomplete functionality. * refactor(autocomplete): remove unused functions and improve clipboard handling - Deleted the `normalize_ax_value` and `parse_ax_number` functions as they were not utilized in the codebase, streamlining the autocomplete module. - Enhanced the `clipboard_save` function to improve readability by formatting the string conversion of clipboard output, ensuring better handling of empty or "missing value" cases. This update cleans up the code and optimizes the clipboard handling logic for the autocomplete feature. * feat(autocomplete): enhance focus handling and text insertion methods - Implemented a unified Swift helper for querying focused text elements, improving performance and reliability on macOS. - Added fallback mechanisms to use osascript for focus queries when the helper is unavailable, ensuring consistent functionality. - Refactored text insertion logic to prioritize the unified helper, with osascript and AXValue as fallback options, enhancing user experience during text application. - Cleaned up and organized focus-related functions, improving code readability and maintainability. This update significantly enhances the autocomplete feature's ability to interact with focused text elements, providing a more robust and responsive user experience. * feat(accessibility): introduce comprehensive accessibility module for macOS - Added a new `accessibility` module that centralizes focus queries, screen capture, key state detection, and permission management for macOS. - Implemented a unified Swift helper process to enhance performance and reliability in querying focused text elements and managing overlays. - Introduced various functionalities including screen capture, text insertion into focused fields, and permission detection for accessibility features. - Enhanced user experience by providing robust methods for interacting with accessibility APIs, ensuring consistent behavior across different contexts. This update significantly improves the accessibility capabilities of the application, providing a more responsive and user-friendly interface for macOS users. * feat(accessibility): introduce screen capture and focus query modules - Added a new `accessibility` module to centralize platform-specific accessibility functionalities, including screen capture and focus queries. - Implemented `capture.rs` for screen capture using platform-native tools, supporting both windowed and fullscreen modes. - Developed `focus.rs` to handle accessibility focus queries, utilizing a unified Swift helper for improved performance on macOS. - Introduced helper functions for managing overlays and permissions, enhancing user interaction and accessibility features. This update significantly enhances the application's accessibility capabilities, providing robust tools for screen capture and focus management. * refactor(accessibility): remove unused accessibility functions and streamline modules - Deleted unused functions from the accessibility module, including `focused_text_context`, `is_text_role`, `normalize_ax_value`, and `parse_ax_number`, to enhance code clarity and maintainability. - Updated module documentation to reflect the current structure and purpose, ensuring consistency across the accessibility and screen intelligence modules. This update simplifies the codebase and improves the overall organization of accessibility-related functionalities. * feat(image-processing): implement image compression and resizing for vision LLM - Added a new module `image_processing` to handle the compression and resizing of screenshots before sending them to the vision LLM. - Implemented the `compress_screenshot` function, which decodes PNG data-URIs, resizes images to fit within a specified maximum dimension, and re-encodes them as JPEGs. - Updated the `AccessibilityEngine` to utilize the new image processing functionality, ensuring that images sent for analysis are optimized for size and quality. - Introduced new dependencies in `Cargo.toml` for image handling and compression. This update enhances the efficiency of image processing in the application, reducing token usage and improving inference speed. * feat(tests): add end-to-end tests for screen intelligence vision pipeline - Introduced a new test file `screen_intelligence_vision_e2e.rs` to validate the complete flow of the screen intelligence vision pipeline. - Implemented tests that cover generating images, compressing and resizing them, simulating LLM responses, and persisting results to memory. - Utilized temporary directories and environment variable management to ensure test isolation and reliability. - Enhanced the testing framework by including helper functions for image creation and mock responses, improving the overall test coverage and robustness. This update significantly strengthens the testing capabilities of the screen intelligence module, ensuring that the entire pipeline functions correctly under various scenarios. * style: apply cargo fmt and import ordering fixes Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(accessibility): add non-macOS stub for validate_focused_target The function was gated with #[cfg(target_os = "macos")] but exported unconditionally from mod.rs, causing compilation failure on non-macOS. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(accessibility): enhance screen capture and error handling - Updated the screen capture functionality to include a timestamp helper in the documentation. - Improved error handling when reading resized screenshots, ensuring temporary files are removed on failure. - Added logging for raw errors returned by the helper in the focus context, providing better debugging information. - Refactored clipboard handling in the paste functionality to preserve multi-line text, enhancing usability. - Introduced a constant for terminal application names to simplify terminal detection logic. This update improves the robustness and clarity of the accessibility module, enhancing both screen capture and focus handling capabilities. --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
|
cf344facf9 |
feat(voice): dedicated voice assistance module for STT/TTS (#178)
* feat(voice): add dedicated voice assistance module for STT/TTS Extracts speech-to-text (whisper.cpp) and text-to-speech (piper) into a dedicated `src/openhuman/voice/` domain module with its own RPC namespace (`openhuman.voice_*`). Adds proactive availability checking via `voice_status` so the UI can show clear errors when binaries/models are missing instead of failing silently at transcription time. - New module: voice/types.rs, voice/ops.rs, voice/schemas.rs, voice/mod.rs - 4 RPC endpoints: voice_status, voice_transcribe, voice_transcribe_bytes, voice_tts - 21 unit tests + 1 integration test (json_rpc_e2e) - Frontend updated to use voice_* endpoints with status check on mode switch Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: fix cargo fmt in voice/ops.rs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test(e2e): add voice mode integration spec Tests switching to voice input mode, verifying status check fires, recording button renders, and switching back to text mode restores text input. Also checks reply mode toggle visibility. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove unused waitForText import in voice-mode e2e spec Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(voice): in-process whisper engine and LLM post-processing - Add whisper-rs (0.16) for in-process whisper.cpp inference, eliminating cold-start latency from subprocess-per-call (~1-3s) to warm inference (~50ms). Model is loaded once during bootstrap and reused across calls. Falls back to whisper-cli subprocess if in-process loading fails. - Add LLM post-processing layer that passes raw transcription through Ollama to fix grammar, punctuation, and filler words. Accepts optional conversation context to disambiguate names and technical terms. Gracefully degrades to raw whisper output if Ollama is unavailable. - Update voice RPC endpoints with new optional params (context, skip_cleanup) and return both cleaned text and raw_text. - Update frontend to pass conversation history as context for voice transcription cleanup, and update TypeScript interfaces to match. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: apply cargo fmt formatting fixes Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(build): make whisper-rs optional behind `whisper` feature flag The whisper-rs crate requires cmake to compile whisper.cpp from source, which is not available in the CI environment. Move it behind an optional cargo feature so CI builds succeed without cmake. The whisper_engine module now compiles as a no-op stub when the feature is disabled, returning "whisper feature not compiled in" errors. Desktop builds can opt in with `--features whisper`. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: cargo fmt whisper_engine.rs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(ci): make whisper-rs mandatory and install cmake in CI Revert whisper-rs from optional to mandatory dependency. Add cmake installation to all CI workflows (build, typecheck, test, release) and the CI Docker image so whisper-rs can compile whisper.cpp from source. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: cargo fmt whisper_engine.rs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address code review findings across voice module - whisper_engine: validate WAV sample rate (must be 16kHz) and channel count (1 or 2) before feeding audio to whisper - speech: offload load_engine and transcribe_in_process to tokio::task::spawn_blocking to avoid blocking the Tokio runtime - ops: use RAII guard for WHISPER_BIN env var in test to prevent races and ensure restore on panic; log temp file cleanup failures instead of silently ignoring; sanitize paths in debug logs to basenames only - postprocess: add test for disabled cleanup config returning raw text - voice-mode.spec: assert failure when neither voice CTA nor unavailable message appears; make reply mode test runnable in isolation with auth/nav setup Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: apply cargo fmt Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
|
906b55b63e |
feat(observability): add Sentry error reporting to Rust core (#131)
* chore(workflows): comment out Windows smoke tests in installer and release workflows * feat(observability): integrate Sentry for error reporting and add configuration support - Added Sentry integration for error reporting in the application, initializing it in the main function. - Introduced a new environment variable `OPENHUMAN_SENTRY_DSN` to configure Sentry DSN. - Updated the observability configuration schema to include a field for Sentry DSN. - Enhanced logging to include Sentry event filtering based on log levels. - Implemented secret scrubbing to protect sensitive information in error reports. * feat(analytics): add support for anonymized analytics settings - Introduced new environment variable `OPENHUMAN_ANALYTICS_ENABLED` to enable or disable anonymized analytics and crash reports. - Updated the PrivacyPanel component to sync analytics consent with the core configuration. - Added new functions to handle analytics settings updates and retrieval in the Tauri backend. - Enhanced observability configuration to include analytics settings, defaulting to enabled. - Updated relevant schemas and handlers for analytics settings in the backend. * refactor(tauriCommands): improve formatting and structure of analytics settings function - Reformatted the `openhumanUpdateAnalyticsSettings` function for better readability by adjusting the parameter structure. - Enhanced the output formatting in the `handle_get_analytics_settings` function for improved clarity in the JSON response. * feat(analytics): sync analytics consent to core RPC and improve anonymization copy Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Al629176andGitHub
|
e7a7f90fb6 |
Merge pull request #113 from sanil-23/issue-60-ingestion-relex
Add GLiNER relex ingestion pipeline (#60) |
||
|
sanil jainandClaude Opus 4.6
|
6964abbf5f |
Fix CI: remove openssl dep, skip ORT init in ingestion tests, fix fmt
- Replace openssl with aes-gcm for AES-256-GCM decryption in rest.rs - Remove openssl/openssl-sys from Cargo.toml and Cargo.lock - Use ci_safe_config() in ingestion tests to skip ORT model loading (avoids Mutex poisoned panic on CI without libonnxruntime) - Remove serial_test dependency (no longer needed) - Fix cargo fmt issue in rest.rs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
cyrus@tinyhumans.ai
|
6f6cdc5631 | merge: resolve Cargo.lock conflict with main | ||
|
cyrus@tinyhumans.ai
|
2570195604 |
feat(local-ai): add guided model tier selection by device capability
Add tiered model presets (Low/Medium/High) with device-aware recommendations so users can pick a local AI model that fits their machine without editing raw JSON config. Detect RAM, CPU, GPU via sysinfo crate and recommend a tier. Persist selection to config.toml, with env var override and graceful degradation hints on bootstrap failure. - Rust: presets.rs (tier definitions, recommendation logic), device.rs (hardware detection), 3 new RPC methods, env var override, bootstrap hints - Frontend: tier selector UI in Settings > Local AI Model with device info, loading/error states, and "Advanced" toggle for existing controls - Tests: 7 Rust unit tests + comprehensive JSON-RPC E2E test - Also fixes pre-existing lint warning in SkillSetupWizard.tsx Closes #80 |