mirror of
https://github.com/tinyhumansai/openhuman.git
synced 2026-07-28 13:32:23 +00:00
b329e45cdb08c2cbc185aa0de99ba30db4dd9679
193
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
b329e45cdb |
feat(skills): uninstall for user-scope SKILL.md skills (#781) (#833)
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
|
478c92e533 |
feat(commands): ⌘K palette + keyboard system + chat UX polish (#745)
- Implement command palette (⌘K) using cmdk and Radix Dialog with a global action registry. - Add a keyboard shortcut system with a capture-phase hotkey manager and scope stack. - Integrate seed navigation actions for Home, Chat, Intelligence, Skills, and Settings. - Improve chat UX with a `useStickToBottom` hook for auto-scroll and fixed hydration errors. - Refactor command UI to display shortcuts inline and remove the redundant help overlay. - Fix Vite HMR connection issues within the Tauri/CEF environment. Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai> |
||
|
|
b40af6c294 |
fix(security): command injection in npm postinstall + weak RNG fallback (#837)
- Replace `execSync` with `execFileSync` in npm install script to prevent command injection. - Pass PowerShell paths via environment variables to avoid shell metacharacter interpolation. - Add `-NoProfile` and `-NonInteractive` flags to PowerShell extraction for cleaner installs. - Upgrade `makeAccountId` to prioritize `crypto.getRandomValues` over `Math.random` for suffixes. - Update internal thread title logic to import `collapse_whitespace` from the correct location. Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai> |
||
|
|
53befcd56f |
chore: drop dead crypto + stale rules, Windows doctor disk probe (#856)
Two cleanups surfaced during a codebase review: ## Changes - **Drop dead integration-token crypto** — `integrationTokensCrypto.ts` + helpers; no remaining consumers - **Trim stale `.claude/rules`** — rules that contradict current CLAUDE.md - **Windows doctor disk probe** — detect low-disk conditions on Windows - **Refresh stale binary-size claim** in docs Net: +82 / −3,596 (mostly dead-code removal). ## Test plan - [ ] `yarn typecheck` green — no dangling imports to crypto module - [ ] `cargo check` green - [ ] Doctor runs on Windows host Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai> |
||
|
|
1408666706 |
feat(notifications): native OS notifications via CEF shim + scanner fallback (#850)
Co-authored-by: oxoxDev <nikhil@tinyhumans.ai> Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai> |
||
|
Steven EnamakelandGitHub
|
643ec33dec | feat(ui,webview): compact tab bar, hide inactive tabs, harden CEF link handling (#868) | ||
|
Cyrus GrayandGitHub
|
c78f496686 | fix(webhooks,cron): wire trigger pipeline so agents auto-respond reliably (#747) | ||
|
|
3b83fba93a |
feat(home): next-steps card + drop stub /agents route (#788)
Co-authored-by: Jwalin Shah <jshah1331@gmail.com> |
||
|
|
7670ae9f12 |
feat(notifications): core→shell DomainEvent bridge (Phase 1d of #395) (#782)
Co-authored-by: Jwalin Shah <jshah1331@gmail.com> Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai> |
||
|
YellowSnnowmannandGitHub
|
29a3962b55 | Feat: add email login auth (#832) | ||
|
|
c8ecfd8b66 |
[codex] Add daemon lifecycle retry and visibility tests (#796)
Co-authored-by: Jwalin Shah <jshah1331@gmail.com> |
||
|
|
f2ad7a3a55 |
polish(settings): trust-surface visual pass — Connections + Recovery Phrase (#766)
Co-authored-by: Jwalin Shah <jshah1331@gmail.com> |
||
|
|
e94afc61ea |
test(frontend): lock provider regression coverage (#791)
Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Jwalin Shah <jshah1331@gmail.com> |
||
|
|
a84c42860c |
test(app): expand frontend regression coverage for store slices and core RPC (#787)
Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
3dc0d0db64 |
feat(notifications): native OS banners for agent/chat/socket events (Phase 1c of #395) (#780)
Co-authored-by: Jwalin Shah <jshah1331@gmail.com> Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai> |
||
|
|
5113b0fd8b |
fix(rewards): add retry on snapshot load failure + tab-switch logging (#768)
Co-authored-by: Jwalin Shah <jshah1331@gmail.com> |
||
|
|
c78c957821 |
feat(notifications): in-app notification center (Phase 1b of #395) (#769)
Co-authored-by: Jwalin Shah <jshah1331@gmail.com> Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai> |
||
|
|
44fda19734 |
fix(recovery): prompt users to download latest build after crash loops (#778)
Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai> |
||
|
|
83aa0166d6 |
feat(onboarding,ui): trust-first onboarding + Button primitive + honest privacy surface (#759)
Co-authored-by: Jwalin Shah <jshah1331@gmail.com> Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai> |
||
|
|
9a6f9cd110 |
fix(onboarding): show onboarding immediately after bootstrap (#777)
Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai> |
||
|
|
325f019853 |
refactor(core): move CLI adapters out of src/core/ into per-domain cli modules (#758)
* refactor(voice): move standalone CLI adapter into voice domain
Move the blocking `openhuman voice` / `openhuman dictate` dictation-server
subcommand out of `src/core/cli.rs` and into a new `src/openhuman/voice/cli.rs`
owned by the voice domain. The core CLI dispatcher now routes to
`voice::cli::run_standalone_subcommand` and no longer imports voice internals.
Why this shape: the standalone server blocks forever on the hotkey listener,
which doesn't fit the controller registry's request/response contract. A
domain-owned CLI adapter is the right home for long-lived operational
commands; the controller registry stays for RPC-style capabilities.
Establishes the exposure rule documented in PLAN.md for the rest of the
backend overhaul.
- src/openhuman/voice/cli.rs: new 118-line adapter (flag parsing, tokio
runtime, run_standalone call)
- src/openhuman/voice/mod.rs: declare `pub mod cli`
- src/core/cli.rs: delete 92-line run_voice_server_command, dispatch only
- PLAN.md: new scope doc + exposure rule
* refactor(text_input): move CLI adapter into domain
Move src/core/text_input_cli.rs to src/openhuman/text_input/cli.rs and
drop the stale `pub mod text_input_cli` from src/core/mod.rs. Core CLI
dispatcher now routes `text-input` to the domain-owned adapter.
The moved file has a mixed shape that validates the exposure rule from
the previous voice commit:
- `run` starts a long-lived HTTP JSON-RPC server (domain cli.rs shape)
- `read / insert / ghost / dismiss` are short-lived UX wrappers around
`text_input::rpc::*` (pretty-print, flag parsing — CLI affordance, not
transport duplication)
Both shapes legitimately belong in the domain. The controller registry
stays for transport-agnostic capabilities; CLI UX wrappers live next to
their domain RPC.
- git mv src/core/text_input_cli.rs src/openhuman/text_input/cli.rs
- src/openhuman/text_input/mod.rs: declare `pub mod cli`
- src/core/mod.rs: drop `pub mod text_input_cli`
- src/core/cli.rs: update dispatch path
* refactor(tree_summarizer): move CLI adapter into domain
git mv src/core/tree_summarizer_cli.rs src/openhuman/tree_summarizer/cli.rs
and drop the stale `pub mod tree_summarizer_cli` from src/core/mod.rs.
Core CLI dispatcher now routes `tree-summarizer` to the domain-owned
adapter.
All five subcommands (ingest / run / query / status / rebuild) are short
-lived UX wrappers over registry handlers that already exist in
tree_summarizer/schemas.rs — no long-running server, no business logic
duplicated. Straight relocation keeps transport generic.
- git mv src/core/tree_summarizer_cli.rs src/openhuman/tree_summarizer/cli.rs
- src/openhuman/tree_summarizer/mod.rs: declare `pub mod cli`
- src/core/mod.rs: drop `pub mod tree_summarizer_cli`
- src/core/cli.rs: update dispatch path
* refactor(screen_intelligence): move + split CLI adapter into domain
Move src/core/screen_intelligence_cli.rs (699 lines, 7 subcommands) into
a domain-owned cli/ submodule, split by responsibility so no single file
exceeds the project's 500-line soft limit.
Layout:
screen_intelligence/cli/mod.rs (204) dispatch + shared helpers
(CliOpts, parse_opts,
bootstrap_engine, logging,
print_help)
screen_intelligence/cli/server.rs (79) run_server (long-running
capture+vision loop)
screen_intelligence/cli/session.rs (150) status / start / stop
screen_intelligence/cli/capture.rs (173) capture / vision (inspect pair)
screen_intelligence/cli/doctor.rs (107) readiness diagnostics
Classification before moving:
- no business-logic duplication vs schemas.rs handlers
- heavy fns (doctor 103, capture 97, start 89) were 80%+ pretty-printing
and CLI-side orchestration (polling loop, flag-driven save) — legit
CLI UX, not extracted domain logic
- shared helpers stayed pub(super) inside mod.rs; no new domain APIs
introduced
Core transport changes:
- delete src/core/screen_intelligence_cli.rs
- drop `pub mod screen_intelligence_cli` from src/core/mod.rs
- src/core/cli.rs dispatch routes to domain cli::
After this commit, src/core/cli.rs contains only dispatch lines for all
four migrated domains (voice, text_input, tree_summarizer,
screen_intelligence) — no embedded domain logic.
* refactor(screen_intelligence): address external review — extract capture save + tighten cli visibility
Follow-ups from codex + gemini review of the branch:
1. Extract CaptureFrame construction + disk save from the CLI.
`capture.rs --keep` was building a CaptureFrame inline (stamping
timestamps, copying context fields) and calling
`AccessibilityEngine::save_screenshot_to_disk` directly. Added
`AccessibilityEngine::save_capture_test_result(workspace_dir,
&CaptureTestResult, reason) -> Option<Result<PathBuf, String>>`
so the CLI only passes the result through and picks a reason
label. Domain owns the frame shape, not the CLI.
2. Tighten CLI module visibility across all four migrated domains.
`pub mod cli` -> `pub(crate) mod cli` and
`pub fn run_*_command` -> `pub(crate) fn` for voice, text_input,
tree_summarizer, screen_intelligence. Only src/core/cli.rs needs
to see these; they're transport plumbing, not domain public API.
No behavior change.
* style: cargo fmt
* address review comments from coderabbitai
- screen_intelligence/cli/server: use effective keep_screenshots (opts.keep
|| config flag) for both SiServerConfig and status output, so the server
behaves consistently with what the CLI reports.
- voice/cli: surface config load errors with a warning instead of silently
falling back to Config::default(); reject unknown --mode values instead
of silently coercing them to Push.
- PLAN.md: narrow the Phase 4 enforcement grep so it forbids domain
imports/non-dispatcher references in the transport layer while still
allowing crate::openhuman::<domain>::cli::run_*(...) dispatch calls,
which this PR legitimately introduces.
* style: cargo fmt voice cli
* style: cargo fmt + bump tauri-cef vendor
* fix(screen_intelligence): narrow save_capture_test_result to pub(crate) (addresses @coderabbitai nitpick on engine.rs:486)
AccessibilityEngine::save_capture_test_result is only called from the
domain's own cli/capture.rs (crate-internal). Making it pub leaks it
into the public API surface unnecessarily — tighten to pub(crate) to
match the visibility-tightening done across the rest of this PR.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* update
* chore: update .gitignore to include scheduled_tasks.lock in app/.claude
* ran formatter
---------
Co-authored-by: Jwalin Shah <jshah1331@gmail.com>
Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
|
||
|
|
8212438112 |
feat(e2e): agent-observable artifact capture for onboarding + privacy flow (#762)
Adds a canonical WDIO spec that captures screenshots, page-source dumps, and mock request logs at named checkpoints, plus an afterTest hook that dumps failure artifacts. Wrapper script prints the run dir so coding agents (and humans) can inspect the flow from disk. Co-authored-by: Jwalin Shah <jshah1331@gmail.com> Co-authored-by: WOZCODE <contact@withwoz.com> |
||
|
|
bc9e223306 |
feat(privacy): backend-backed capability privacy metadata + PrivacyPanel (#760)
* feat(about_app): add capability privacy metadata
Adds optional `CapabilityPrivacy { leaves_device, data_kind, destinations }`
to the about_app capability catalog so the in-app Privacy surface can be
backend-backed instead of hand-maintained. Twelve representative capabilities
are annotated for the first audited set (raw/local, derived/backend,
credentials, diagnostics, model download); remaining entries default to
None and are simply not surfaced. Wire format stays backward compatible
via skip_serializing_if.
* feat(settings): drive privacy panel from about_app capabilities
Replaces the hand-maintained privacy rows with data fetched from
openhuman.about_app_list. Only capabilities that ship privacy metadata
are rendered; loading and RPC failure both degrade gracefully and the
analytics toggle plus explanatory copy remain intact. Adds a small typed
client (utils/tauriCommands/aboutApp.ts) and focused vitest coverage for
render, omission of unannotated entries, and RPC failure.
---------
Co-authored-by: Jwalin Shah <jshah1331@gmail.com>
|
||
|
|
3bb714bf96 |
feat(webview): native OS notifications from embedded webview apps (#714) (#727)
* feat(webview_accounts): native OS notifications from embedded webviews (#714) Forward CEF notification intercept payloads to tauri-plugin-notification, prefixing the title with the provider label so the source of each toast is obvious at a glance. Honour `silent` (skip toast, still record route), `icon` (passed through to the native builder), and `tag` (used as the dedup key, with a monotonic timestamp fallback for untagged payloads). Record a NotificationRoute keyed by `{provider}:{account_id}:{tag_or_uuid}` so a future click hook (UNUserNotificationCenter / notify-rust on_response) can route the OS click back to the source account. Entries are cleared on webview_account_close / _purge to bound map growth. Expose webview_notification_permission_state / _request commands mapping tauri::plugin::PermissionState onto the web API triple. Non-cef stubs return "default" so the frontend can call the same invoke names on both runtimes. Wire notification:allow-* capabilities so the plugin can be invoked from the webview. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(accounts): wire notification permission + click bridge (#714) Round-trip the OS notification permission once per session on first account open via the new invoke pair. Attach a dormant notification:click listener that dispatches setActiveAccount and brings the main window to front when a platform click hook starts emitting the event — contract matches the Rust NotificationRoute shape so the emit side is a one-liner. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: sync Cargo.lock to 0.52.26 after version bump Lockfile picked up the pending 0.52.26 version bump from Cargo.toml while building the notification feature. No dependency graph change. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(notifications): add notification bypass for embedded webview apps (#679) - Add NotificationBypassPrefs (global DND, per-account mute, bypass-when-focused) to WebviewAccountsState with thread-safe AtomicBool window focus tracking - Evaluate all three bypass conditions inside forward_native_notification before showing OS toast; each suppression path logs at debug with [notify-bypass] prefix - Add four new Tauri commands: webview_notification_set_dnd, webview_notification_mute_account, webview_notification_get_bypass_prefs, webview_set_focused_account - Wire window focus tracking in setup hook via on_window_event Focused handler - Frontend: add setAccountMuted, setGlobalDnd, getBypassPrefs, setFocusedAccount helpers in webviewAccountService; sync focused account on open + click - Add NotificationsPanel settings page with Global DND toggle - Register NotificationsPanel at /settings/notifications Closes #679 * feat(notifications): integrate notifications feature into app - Added Notifications page and routing to AppRoutes. - Introduced NotificationRoutingPanel in Settings for managing notification settings. - Updated SettingsHome to include navigation for notification routing. - Integrated notifications reducer into the store for state management. - Enhanced Rust backend to support notification handling from embedded webviews. This commit lays the groundwork for a comprehensive notification system within the application. * refactor(notifications): clean up code formatting and structure - Simplified JSX structure in NotificationCard for better readability. - Consolidated fetchNotifications call in NotificationCenter for cleaner syntax. - Improved formatting in NotificationRoutingPanel and notificationsSlice for consistency. - Enhanced Rust code readability by streamlining function signatures and logic. These changes enhance code maintainability and readability across the notifications feature. * refactor(webview_accounts): simplify webview_notification_set_dnd function signature - Removed unnecessary line breaks in the webview_notification_set_dnd function for improved readability. * feat(notifications): implement provider-level notification settings management - Added `getNotificationSettings` and `setNotificationSettings` functions to manage notification settings for providers. - Enhanced `NotificationRoutingPanel` to display and update settings for Gmail, Slack, Discord, and WhatsApp. - Introduced new RPC endpoints for retrieving and updating notification settings. - Updated database schema to store notification settings persistently. This commit establishes a robust system for managing notification preferences, improving user control over notifications. * refactor(notifications): improve code formatting and readability - Enhanced formatting in NotificationRoutingPanel for better clarity. - Streamlined function signatures in notificationService and Rust backend. - Improved readability of assertions in tests by adjusting line breaks. These changes contribute to a more maintainable and comprehensible codebase for the notifications feature. * chore(vendor): bump tauri-cef to fix Slack notification permission banner Updates the tauri-cef submodule to 55db2d6 which adds a navigator.permissions.query shim in the CEF render process. Slack checks this API (not just Notification.permission) to decide whether to show its "needs your permission" banner — the shim returns "granted" for notifications queries so the banner no longer appears. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore(vendor): bump tauri-cef for cargo fmt fixes Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore(vendor): bump tauri-cef — native V8 permissions.query shim Switches from context.eval() to a proper PermissionsQueryV8Handler so the navigator.permissions.query fix actually runs in on_context_created. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(notifications): patch navigator.permissions.query in ua_spoof.js The V8 set_value_bykey approach in cef-helper's on_context_created does not stick on CEF platform objects (Chromium's V8 binding layer silently ignores property writes on native wrappers like Permissions). The init script path via frame.execute_java_script runs in the fully-initialised JS context where navigator.permissions IS writable, matching how ua_spoof.js already overrides navigator.userAgent successfully. Slack checks navigator.permissions.query({ name: 'notifications' }) before showing its "needs permission" banner — patching it here to return "granted" removes the banner. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(notifications): use Object.defineProperty to shim navigator.permissions Two-layer fix for the Slack "needs permission to enable notifications" banner: 1. cef-helper (submodule update to 99a2686): context.eval() in on_context_created installs Object.defineProperty(navigator, 'permissions', ...) before any page JS runs. 2. ua_spoof.js: same Object.defineProperty pattern as belt-and-suspenders for frames that reload or trigger permission checks after on_load_end. Simple property assignment on Blink platform objects is silently ignored; Object.defineProperty on the navigator wrapper itself (the same mechanism already used for navigator.userAgent) works correctly. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(notifications): address CodeRabbit review issues on PR #727 - Move raw_title out of log::info! (PII risk) — log title_chars at info, raw_title at debug only - Fix permissionChecked set before async invoke in ensureNotificationPermission so transient failures allow retry on next account open * fix(cef): enable webview-data-url feature for CEF placeholder URL The CEF backend uses a data: URL as the initial webview location so CDP can attach before the real provider URL loads. Tauri's add_child rejects data: URLs unless the webview-data-url feature is enabled. * fix(notifications): address remaining CodeRabbit issues on PR #727 - cdp/emulation: bump Chrome UA 124→136 to pass Slack browser check - cdp/session: inject Page.addScriptToEvaluateOnNewDocument to stub Notification.permission as "granted" and silence provider banners - notifications/mod.rs + core/all.rs: wire notifications domain into the controller registry (fixes unknown-method in json_rpc_e2e tests) - notifications/schemas: add skipped bool output to ingest schema - notifications/store: add tracing::warn on datetime parse failure - notificationService: union return type for ingestNotification - webviewAccountService: narrow union before accessing result.id - NotificationCenter: drive loading/error from fetch effect; track allProviders separately so filter pills don't collapse on selection - NotificationRoutingPanel: rollback optimistic update on save failure - useSettingsNavigation: add notifications/notification-routing routes - scripts/install.sh: remove silent dry-run exit 0 on asset failure - scripts/setup-dev-codesign.sh: remove unconditional -legacy flag - docs/SUMMARY.md: remove worktree path, fix macOS capitalisation, remove self-referential deletion note * chore: apply prettier + cargo fmt + fix useEffect dep warning Auto-apply formatting changes flagged by the pre-push hook: - prettier reformatted NotificationCenter.tsx and notificationService.ts - cargo fmt reformatted all.rs, openhuman/mod.rs, notifications/schemas.rs - NotificationRoutingPanel: move providers array to module scope so useEffect dependency array is satisfied without exhaustive-deps warning * feat(notifications): enhance notification management and permissions - Added new commands for managing notification preferences, including setting global Do Not Disturb (DND), muting specific accounts, and retrieving current bypass preferences. - Implemented a notification permission state handler to ensure consistent behavior across different environments. - Updated the JavaScript shim for notification permissions to handle both Notification and PushManager states, ensuring compatibility with various providers. - Refactored the WebviewAccountsState to include a new structure for managing notification bypass preferences, improving the overall notification handling logic. * update agents * fix(notifications): complete schema + navigation metadata for ingest/settings routes - app/src/components/settings/hooks/useSettingsNavigation.ts: resolve the new `/settings/notifications` and `/settings/notification-routing` URLs to their SettingsRoute values and feed them into breadcrumbs so the new panels don't silently fall through to `'home'`. Addresses CodeRabbit on useSettingsNavigation.ts:34. - src/openhuman/notifications/schemas.rs: add the optional `reason` output on `notification.ingest` (populated alongside `skipped=true` by the runtime) and the normalized `settings` output on `notification.settings_set` so schema-driven clients see the full response shape. Addresses CodeRabbit on schemas.rs:103 and schemas.rs:217. - src/core/all.rs: add a `notification` namespace_description so CLI help covers the new controllers, plus a test assertion. Addresses CodeRabbit on src/core/all.rs:149. * fix(notifications): trace DB entry, surface empty update matches, warn on bad scored_at - Add `tracing::trace!` checkpoints around the `with_connection` DB open and schema migration so notification-delivery issues are reconstructible from logs. - `update_triage` and `mark_read` now inspect `Connection::execute`'s affected-row count: log a `warn!` when the update matched zero rows (row deleted between ingest and scoring / client passed a stale id), `debug!` on the normal path. - `scored_at` parsing no longer silently drops malformed values — log a `warn!` with the raw value and parse error before treating the row as unscored, matching the existing behavior for `received_at`. Addresses CodeRabbit on store.rs (lines 72, 172, 294). * fix(webview): respect silent notifications, multi-host CDP fallback, shim idempotency - webview_accounts/mod.rs: honor the Web Notification `silent` flag. Previously we only logged it and still called `builder.show()`, so pages that marked a notification silent still produced an OS toast. Mirror event still fires so the in-app center updates; only the OS toast is suppressed. Also picks up a prior cargo-fmt rewrap. - cdp/target.rs: `browser_ws_url()` now continues the host loop when `resp.json()` fails instead of early-returning via `?`. A malformed response from the first host (CDP_HOST) no longer prevents the `localhost` fallback from being tried. - webview_accounts/ua_spoof.js: guard the Notification wrapper behind `window.__OH_NOTIF_SHIM` so repeated evaluations of the script (Page.addScriptToEvaluateOnNewDocument + frame-level re-injections) don't stack wrappers onto the same page globals or re-proxy `Function.prototype.toString`. Addresses CodeRabbit on webview_accounts/mod.rs:377, cdp/target.rs:33, and ua_spoof.js:176. * update agents * update --------- Co-authored-by: oxoxDev <nikhil@tinyhumans.ai> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai> |
||
|
|
581f37965c |
feat(skills): SKILL.md skills UI — browse, create, install from URL (#681) (#740)
* feat(skills/core): add read_skill_resource with size + traversal guards (#681) Introduces `read_skill_resource(skill_id, relative_path)` in the skills ops module. Used by the new `skills.read_resource` RPC (landed in a follow-up commit) to let the UI preview files bundled alongside a SKILL.md without having to shell out to the Node runtime. Guards rejecting each known attack surface have their own unit test: - empty skill_id / empty relative_path - unknown skill - absolute paths - `..` traversal escapes (checked after canonicalization against the skill root, reusing the pattern from the Node exec allowlist) - directory targets - symlinked leaves (reject via `symlink_metadata` before open) - files over the 128 KB cap - non-UTF-8 content (binary allowlist is text-only) Happy-path test covers a small text resource under the skill root. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(skills/core): wire skills.read_resource RPC + namespace (#681) Adds `skills` RPC namespace with `skills.list` and `skills.read_resource` handlers. `read_resource` delegates to `read_skill_resource` (previous commit) and surfaces path-traversal / size / encoding errors back to the caller verbatim so the UI can render the error string as-is. - `src/openhuman/skills/schemas.rs` (new): controller + schema definitions, plus unit tests for schema name stability, round-trip of the minimum `SkillSummary` fields, and controller list/schema length parity. - `src/openhuman/skills/mod.rs`: declare `pub mod schemas` and re-export `all_skills_controller_schemas`, `all_skills_registered_controllers`, and `skills_schemas`. - `src/core/all.rs`: register the controllers + schemas and add a namespace description so the RPC discovery endpoint surfaces it. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(app/api): typed skillsApi client for list + read_resource (#681) Thin typed wrapper around the `skills.list` and `skills.read_resource` RPCs added in the previous commit. The client normalises the backend response shape (bytes + UTF-8 content) and rethrows backend error strings verbatim so the preview pane can render them unchanged. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(app/ui): SkillResourceTree groups bundled resources by top dir (#681) Presentational component that takes the `resources: PathBuf[]` from a loaded Skill and renders it as a grouped list (scripts, assets, references, etc. based on the first path segment). Selecting a leaf calls `onSelect(relativePath)` so the parent drawer can drive preview state. Stateless — no fetching, no effects. Styling follows the stone/coral design tokens used across the Skills page. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(app/ui): SkillResourcePreview size-gated viewer + specs (#681) Presentational component that fetches a single bundled resource via `skillsApi.readSkillResource`. Backend caps payloads at 128 KB and either returns UTF-8 text or a plain error string, so the preview pane has three visual states: loading, error, success. - On error (e.g. "path escape", ">128KB", "non-UTF-8"), renders the backend message verbatim in a coral panel. - On success, renders a monospace pre block with the byte count in the footer. - `key={id:path}` on the mount site (in SkillDetailDrawer, next commit) drives a remount when the selected resource changes — so no setState-in-effect hack is needed to reset loading state. Vitest specs cover: loading state, success rendering with byte footer, error rendering for traversal / oversize / encoding strings, cancelled fetch guard on unmount. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(app/ui): SkillDetailDrawer right-side detail panel + specs (#681) Slide-in right-hand drawer that displays frontmatter metadata (description, version, author, license, tags, allowed_tools) and hosts SkillResourceTree + SkillResourcePreview. Opened by clicking a skill card on the Skills page (wired in the next commit). - Focus management: on mount, focuses the close button via `window.requestAnimationFrame` and restores the previously focused element on unmount. - Esc + backdrop click dismiss. - Preview pane is conditionally rendered and keyed on `${skill.id}:${selectedResource}` so changing the selected resource remounts the previewer (avoids setState-in-effect pattern). Vitest specs cover: render with frontmatter, resource tree click opens preview, close button / Esc / backdrop dismiss paths, focus restoration, empty-resources case. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(app/ui): open SkillDetailDrawer on skill card click (#681) Wires the Skills page to the new drawer: clicking a skill card sets `selectedSkill` state, which mounts `SkillDetailDrawer`. Dismissing the drawer clears the state. Cards gain an explicit "View details" affordance for discoverability. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(skills/core): add skills.create RPC for scaffolded SKILL.md authoring (#681) Adds `create_skill` in ops.rs plus the `skills.create` controller + handler in schemas.rs. Writes a minimal SKILL.md (with optional license/author/tags/ allowed-tools frontmatter) under the selected scope, scaffolds scripts/ references/assets subdirs, and re-discovers the skill to return the parsed SkillSummary. Legacy scope rejects; Project scope requires the trust marker; User scope is always allowed when a home directory is available. Hardened against path traversal the same way read_skill_resource is: canonicalize the scope root, canonicalize the target dir, reject unless the target starts with the root. Slug derivation is ASCII-only (collapse whitespace/ -/_ to a single hyphen, drop other chars, trim hyphens, enforce MAX_NAME_LEN). Tests (hermetic via create_skill_inner): - user-scope happy path (slug, metadata, SKILL.md on disk, subdirs) - slug collision rejection - invalid name (no alphanumerics) rejection - project-scope without trust marker rejection - project-scope with trust marker happy path - legacy-scope rejection - empty-description rejection - slugify edge cases Closes part of #681 (backend scope for create flow). * feat(skills/core): add skills.install_from_url RPC via npx skills add (#681) Introduces `install_skill_from_url(url, timeout_secs?)` — a JSON-RPC method that shells out to `npx --yes skills add <url>` under the managed Node runtime so the UI can install published SKILL.md packages directly. Security posture: - https scheme only (no http, file, ssh, git+https…) - Rejects `localhost`, `*.localhost`, `*.local`, RFC1918 private IPv4, loopback, link-local, multicast, broadcast, unspecified, 100.64/10 CGN, 0.0.0.0/8, and IPv6 loopback/unspecified/multicast, fc00::/7 ULA, fe80::/10 link-local. Explicitly covers 169.254.169.254 cloud metadata. - Trims + caps URL at 2048 chars; parses with the `url` crate. - IPv6 brackets stripped from `host_str()` before address parse. Process posture: - Reuses `NodeBootstrap` so the managed toolchain resolves first. - `env_clear()` + explicit PATH injection (bootstrap bin_dir first) + a narrow safe-env allow-list (HOME, TERM, LANG, LC_ALL, LC_CTYPE, USER, SHELL, TMPDIR). Matches the npm_exec pattern from #723. - Default 60s wall-clock timeout, capped at 600s. - Captures stdout/stderr; returns both on success or failure. - Diff-based `new_skills`: snapshots discovered skills pre-install and reports slugs that appear post-install. Surface: - JSON-RPC: `openhuman.skills_install_from_url` params: { url: string, timeout_secs?: number } result: { url, stdout, stderr, new_skills[] } - Wired into `all_skills_controller_schemas()` and `all_skills_registered_controllers()`. Tests (5 new unit tests, all pass — 51/51 in skills::): - validate_install_url_accepts_public_https - validate_install_url_rejects_non_https_scheme (http, file, ftp, ssh, git+https, javascript) - validate_install_url_rejects_empty_and_oversized - validate_install_url_rejects_private_and_loopback (20 URLs inc. CGN, cloud metadata, IPv6 ULA/link-local/loopback/multicast) - validate_install_url_rejects_malformed (missing scheme, empty host, non-https scheme, unparseable bracketed host) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(app/api): skillsApi.createSkill + installSkillFromUrl wrappers (#681) Adds typed frontend wrappers for the two new skill-authoring RPC methods: - `skillsApi.createSkill(input)` — scaffolds a new SKILL.md skill via `openhuman.skills_create`. Accepts camelCase `allowedTools` and rekeys it to the `allowed-tools` spelling the SKILL.md frontmatter convention expects, matching `SkillsCreateParams` in `src/openhuman/skills/schemas.rs`. Optional fields are only sent when explicitly provided so the Rust `#[serde(default)]` defaults apply cleanly. - `skillsApi.installSkillFromUrl(input)` — installs a published skill package via `openhuman.skills_install_from_url`. Accepts camelCase `timeoutSecs` and rekeys it to `timeout_secs`. Normalizes the response (snake_case `new_skills` -> camelCase `newSkills`, missing list -> []). Both wrappers reuse the existing `unwrapEnvelope` helper so they tolerate either a bare RPC payload or the `{ data: … }` envelope some transports emit. Adds `CreateSkillInput`, `InstallSkillFromUrlInput`, and `InstallSkillFromUrlResult` type exports for downstream modal components. Tests (vitest, 6 new specs, all pass): - createSkill forwards inputs and rekeys allowedTools - createSkill omits optional fields when absent - createSkill unwraps envelope responses - installSkillFromUrl forwards url and rekeys timeoutSecs - installSkillFromUrl omits timeout_secs + defaults newSkills to [] - installSkillFromUrl unwraps envelope responses Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(app/ui): CreateSkillModal for scaffolding SKILL.md skills (#681) Adds a centered white modal that scaffolds a new SKILL.md skill via `skillsApi.createSkill`, matching the settings-modal design rules (520px desktop, 16px radius, backdrop+blur, Escape/click-out to close, focus capture). Form fields mirror the Rust `SkillsCreateParams` schema: - name (required) — display name, also slugified into the on-disk directory; a live slug preview surfaces what will hit disk - description (required) — short prose; written as the `description:` field in the generated YAML frontmatter - scope (user | project radio) — `legacy` is hidden because that layout is read-only and being phased out - license (optional) — free-form SPDX-style string - author (optional) - tags (optional, CSV) — normalised client-side; empty entries dropped - allowedTools (optional, CSV) — rekeyed to `allowed-tools` on the JSON-RPC wire by `skillsApi.createSkill` The slug preview mirrors `slugify_skill_name` on the Rust side (lowercase ASCII alnum + `-`, collapse repeats, trim edge hyphens) so the user sees what the Rust slugifier will produce; the Rust side stays authoritative when the skill is persisted. On success `onCreated(skill)` fires with the freshly-discovered `SkillSummary`, letting the parent grid insert the new row without a full refetch. On failure the Rust error string is surfaced verbatim in a coral-styled alert and the submit button re-enables. Vitest specs cover: required-field rendering, live slug preview, submit-disabled gating, Escape close, wire-format rekey of `allowedTools` → `'allowed-tools'`, `onCreated` dispatch, and error-banner recovery. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(app/ui): InstallSkillDialog for npx skills add <url> (#681) Adds a centered white modal that installs a published skill package via `skillsApi.installSkillFromUrl`. The Rust side shells out to `npx --yes skills add <url>` under the managed Node toolchain, with an allow-list on the URL (https only, no private/loopback/link-local/ multicast/cloud-metadata hosts) and a wall-clock timeout (default 60s, max 600s). UI contract: - Single URL input plus optional timeout in seconds. - Client-side `isLikelyValidUrl` fails fast on non-https URLs so the user doesn't pay a round-trip for shape errors the Rust side would reject anyway; the Rust side remains authoritative. - Timeout field validates `1 <= n <= 600` client-side to mirror the server-side clamp range. - While the RPC is in flight we render a spinner with "Running `npx skills add`…" copy and disable close / backdrop dismiss so we don't orphan the subprocess. - On success we surface the list of `newSkills` (ids that appeared post-install) plus captured stdout/stderr panes inside collapsible <details> elements, then hand the full result back to the caller via `onInstalled` so the parent can refetch the skills list and auto-select the new row. - On failure the Rust error string is rendered verbatim in a coral alert and the submit button re-enables. Vitest specs cover: required-field rendering, URL shape gating (empty, malformed, http://, https://), timeout range validation, `timeoutSecs` forwarding on submit, success panel with newSkills rendering, blank timeout omitted from payload, and error-banner recovery. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(app/ui): wire New skill + Install from URL into Skills page (#681) Adds a header row on the Skills page with two buttons: - **New skill** → opens `CreateSkillModal` - **Install from URL** → opens `InstallSkillDialog` Extracts the existing `listSkills` effect into a reusable `refreshDiscoveredSkills` helper so both new flows can reconcile their results against the freshly-discovered `SkillSummary` rows rather than relying on the optimistic payload from the RPC alone. Create flow: - Optimistically appends the returned `SkillSummary` to `discoveredSkills` (dedupe by id). - Auto-opens the detail drawer for the new skill so the user lands in context — matches the install flow's UX. - Follows up with `refreshDiscoveredSkills()` so version/author/ warnings picked up by the Rust discoverer end up in state too. Install flow: - Always refreshes the list (the install can add multiple skills if the package declares several). - Auto-opens the detail drawer for the first newly-installed skill when at least one id is reported back; otherwise leaves the grid in its refreshed state. Both buttons sit in a flush `max-w-lg` header above the existing search bar, styled consistent with `UnifiedSkillCard` CTAs — ocean primary for "New skill" (positive action), neutral stone for "Install from URL" (secondary). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(skills/core): direct SKILL.md fetch replaces npx skills add (#681) Installer no longer shells out to the vercel-labs/skills CLI. It now fetches SKILL.md over HTTPS, validates YAML frontmatter, and writes into the user's skills dir. Size cap (1 MiB), timeout clamp (1-600s), GitHub blob->raw URL normalization, and path-traversal guards are covered by unit tests. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * refactor(app/ui): install dialog copy + categorized errors for direct fetch (#681) Dialog subtitle, helper text, and in-flight indicator reflect the new direct SKILL.md fetch flow. Errors from the core are categorized into friendly titles (URL rejected, too large, timeout, parse failure, already installed, write failed) with the raw backend message tucked under a details disclosure. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(app/ui): dialog specs cover direct-fetch fixtures + error categorization (#681) Fixtures updated to raw GitHub SKILL.md URLs. New cases assert the categorization helper surfaces the right title for invalid SKILL.md, unsupported URL form, and unknown backend errors (raw text hidden under details). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(skills/core): install_from_url writes to user scope (#681) Project scope (`<ws>/.openhuman/skills/`) is gated on a `<ws>/.openhuman/trust` marker that the workspace rarely has, so freshly-installed skills were invisible to `skills.list` until the user opted the workspace into trust. Route installs to `~/.openhuman/skills/<slug>` — the user-scope root that `discover_skills` always scans — so "Install from URL" surfaces the new skill immediately. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs(skills): align install_from_url docs with direct-fetch impl (#681) `install_from_url` stopped shelling out to `npx --yes skills add <url>` when it was rewritten to fetch SKILL.md over HTTPS directly, but schema descriptions and SDK wrappers still described the old subprocess flow. Fix the module-level rustdoc, the JSON-RPC schema `description`/`comment` fields, and the TS client wrapper doc comments so the surface documents what actually runs. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(skills/core): DNS-to-private-IP SSRF guard + install rollback (#681) Two fixes surfaced by CodeRabbit review on PR #740: * `validate_install_url` only inspected literal-IP hosts, so a public-looking hostname like `evil.example.com` with an A record pointing at `127.0.0.1` / `169.254.x` / etc. would still be handed to `reqwest`. Resolve the host via `tokio::net::lookup_host` before the GET and reject if any returned address falls in loopback / private / link-local / multicast / unspecified ranges. Document the remaining DNS-rebinding gap (pinning to a `SocketAddr` + custom reqwest resolver is tracked separately). * If `std::fs::write` or `std::fs::rename` fails after `create_dir_all` succeeded, the empty/partial target directory used to survive and permanently block retries under the same slug. Wrap the write+rename in a rollback that removes the temp file + the just-created directory on failure (best-effort; cleanup errors are logged and the original write error is surfaced). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * style(skills): cargo fmt break long Host::Ipv6 conditional (#681) CI's cargo fmt (stable) rewraps the long `.map(..).unwrap_or(false)` chain that passed local fmt. Apply the break so the pre-push hook and the upstream lint job agree. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Cyrus GrayandGitHub
|
8ab8c59961 |
fix(onboarding): remove referral code step from onboarding flow (#772)
Simplify onboarding from 4 steps to 3: Welcome → Skills → Context Gathering. The referral step added unnecessary friction; the feature may return later so ReferralApplyStep.tsx is preserved but unused. Closes #752 |
||
|
Cyrus GrayandGitHub
|
100b5b781c |
fix(voice): friendly STT error with direct settings link (#761)
* fix(voice): show friendly error with settings link when STT model is missing Replace technical "whisper.cpp binary not found" error with a user-friendly message and a "Set up" link that navigates directly to Settings > Local AI Models so non-technical users can easily download the required speech model. * style: apply prettier formatting |
||
|
|
6a28e75e94 |
feat(routing): pluggable local inference (MLX, llama.cpp, LM Studio) + openhuman:// deep-link + OpenSSL 3 PKCS12 fix (#750)
* feat(routing,local-ai): pluggable local inference + openhuman:// deep-link
Salvaged from recovery commit cfb1fd9f (Apr 19) — originally auto-backed up
by claude-mem recovery tool, not yet PR'd.
## Local AI routing becomes pluggable (factory.rs, health.rs)
Previously the IntelligentRoutingProvider hard-wired Ollama as the only local
inference backend. This commit lets operators point at any OpenAI-compatible
local server (llama.cpp / llama-server, LM Studio, MLX's mlx_lm.server /
mlx-omni-server, vLLM — anything exposing /v1/chat/completions + /v1/models).
Config surface:
- `OPENHUMAN_LOCAL_INFERENCE_URL` env var — full /v1 base URL. When set,
health is probed via GET {base}/models (OpenAI-compat) instead of Ollama's
/api/tags.
- `LocalAiConfig.provider` accepts "llamacpp" or "llama-server" as aliases
that default to http://127.0.0.1:8080/v1.
- Default (no env, no override) stays Ollama — zero config change for
existing users.
Motivation: Ollama's embedded llama.cpp cannot yet load Gemma 4 E2B and
other recent models; MLX is significantly faster on Apple Silicon.
Operators should be able to pick the backend; end users see nothing.
## Deep-link scheme (Info.plist)
Adds CFBundleURLTypes entry registering `openhuman://` for the macOS bundle
so browser-issued deep links (e.g. openhuman://auth?token=...) launch the
installed app. See .claude/rules/14-deep-link-platform-guide.md for the
flow — requires a .app bundle (not `tauri dev`).
## OpenSSL 3.x PKCS12 fix (setup-dev-codesign.sh)
Adds `-legacy` to the `openssl pkcs12 -export` call. Required on OpenSSL 3.x
because the modern SHA-256 MAC / AES-256-CBC defaults are not yet supported
by macOS `security` tool, which silently fails to import the cert.
Notes:
- Dropped the CLAUDE.md.new noise and the mic-description copy downgrade
from the original recovery commit; kept only the load-bearing changes.
- Cargo.lock files reset to upstream/main to avoid dependency drift — the
new code does not introduce new deps.
Co-Authored-By: WOZCODE <contact@withwoz.com>
* fix(routing,codesign): address CodeRabbit review on PR #750
- setup-dev-codesign.sh: probe for openssl `-legacy` support before
using it so older OpenSSL/LibreSSL installs don't silently fail;
drop the stderr suppression on pkcs12 so import errors surface.
- routing/factory.rs: trim provider before alias matching, lower the
local-inference diagnostic to `debug` and drop raw URLs from the
log fields, and honor `OPENHUMAN_OLLAMA_BASE_URL` via the existing
`ollama_base_url()` helper in the default Ollama branch.
- local_ai/mod.rs: re-export `ollama_base_url` for the routing factory.
* style(app/src-tauri): apply rustfmt to merged main
`cargo fmt --check` in the merge workspace flagged two hunks inherited
from upstream main:
- app/src-tauri/src/lib.rs: move `mod notification_settings;` between
the cfg(cef) imessage_scanner and slack_scanner declarations so the
ordering matches rustfmt's reordering output.
- app/src-tauri/src/webview_accounts/mod.rs: wrap the long
`try_state::<NotificationSettingsState>()` binding.
No behavior change.
* format
---------
Co-authored-by: Jwalin Shah <jshah1331@gmail.com>
Co-authored-by: WOZCODE <contact@withwoz.com>
Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai>
|
||
|
Steven EnamakelandGitHub
|
e405e2cc99 |
Prune obsolete config API keys (#739)
* feat(memory_recipes): introduce memory recipes module for source-type-specific ingestion - Added a new `memory_recipes` module that includes functionality for handling various source types, such as email and chat, with dedicated recipes for ingestion. - Implemented core types and the `Recipe` trait to facilitate the ingestion process, allowing for structured handling of raw content and metadata. - Created specific recipes for email and chat, enhancing the ability to process and normalize content before storing it in memory. - Updated the core controller registration to include new endpoints for memory recipes, enabling JSON-RPC interactions for ingesting content and listing available recipes. - Enhanced task management by integrating task storage and query helpers, allowing for better tracking and management of action items extracted during ingestion. These changes significantly improve the ingestion capabilities of the application, providing a robust framework for handling diverse content types. * refactor: remove API key references from configuration and schema - Eliminated the `api_key` field from various configuration schemas and related structures, including `Config`, `ModelSettingsPatch`, and `ComposioConfig`, to streamline the configuration process. - Updated the handling of API keys in the application to rely solely on session JWTs, enhancing security and simplifying the authentication flow. - Adjusted related documentation and comments to reflect the removal of the API key, ensuring clarity in the new authentication approach. - Refactored multiple components and functions to remove dependencies on the API key, improving code maintainability and reducing complexity. These changes significantly enhance the security posture of the application by removing reliance on static API keys and promoting the use of session-based authentication. * chore: update .env.example and documentation for API key removal - Removed references to `OPENHUMAN_API_KEY` from the `.env.example` file to reflect recent changes in the authentication approach. - Updated documentation in `AGENTS.md` and `CLAUDE.md` to clarify that `OPENHUMAN_API_URL` now overrides `config.api_url`, ensuring consistency across configuration references. - Bumped the version in `Cargo.lock` to 0.52.23 to align with the latest changes. These updates enhance clarity in configuration management and documentation following the removal of static API keys. * fix: address config pruning review comments * feat(tests): add validation tests for SearchResponse deserialization - Introduced new tests to ensure that the SearchResponse struct correctly rejects JSON inputs missing required fields: searchId, results, and costUsd. - Updated DelegateTool instantiation to remove the unused fallback_credential parameter, simplifying the constructor and related tests. These changes enhance the robustness of the SearchResponse handling and improve code clarity by removing unnecessary parameters. * refactor: remove api_key params from provider factory signatures The OpenHuman backend now only uses the app-session JWT for auth — the api_key parameter was ignored (as _api_key) after the config pruning. Drop it from the signatures entirely instead of leaving dead None arguments at every call site. Also removes the unused ChannelContext.api_key field. * refactor: remove api_key/composio_key params from factory signatures Every caller was passing None after the config pruning: - create_embedding_provider / create_memory* — the api_key param was only threaded to the openai / custom embedding branches, but no caller has ever supplied a non-None value since the pruning. Embeddings use an empty key; the param is gone. - all_tools / all_tools_with_runtime — both composio_key and composio_entity_id only fed the ComposioTool registration block, which was already dead after the JWT migration. Drop both args and the block; the modern path is all_composio_agent_tools via build_composio_client(config). * style: apply cargo fmt |
||
|
CodeGhost21andGitHub
|
bdbb83772e |
feat(observability): Sentry release tracking, source maps, and end-to-end DSN plumbing (#734)
* ci(release): bake Sentry DSN into shipped tauri bundle
Released builds weren't reporting anything to Sentry. Root cause: the
tauri.conf.json `beforeBuildCommand` re-runs `vite build` inside
`cargo tauri build`. The prior `yarn build` step set `VITE_SENTRY_DSN`
for its own run, but the tauri step did not — so the rebuild produced
a DSN-less `dist/` that overwrote the good one, and the shipped web UI
initialized Sentry with an empty DSN (`initSentry` returns early when
`!SENTRY_DSN`).
Fix:
- `release.yml` / `build-desktop` — declare `VITE_SENTRY_DSN` and
`VITE_DEBUG` on the tauri-build step so the `beforeBuildCommand`
rebuild bakes them into the final bundle.
- `release-packages.yml` / `build-cli-linux-arm64` — guard against a
missing `vars.OPENHUMAN_SENTRY_DSN` so the Linux arm64 CLI tarball
cannot ship without error reporting baked in via `option_env!`.
The core sidecar's `option_env!("OPENHUMAN_SENTRY_DSN")` already gets
the value from the dedicated "Build sidecar core binary" step; the
tauri shell doesn't rebuild it (separate crate, not a workspace dep),
so the baked DSN survives into the bundled installer.
* feat(observability): Sentry release tracking + source maps (#405)
Tags every Sentry event with a canonical release identifier shared by
the frontend and Rust core, uploads source maps so stack traces are
symbolicated in the dashboard, and adds a CLI probe for repeatable
verification of any future release.
Release identifier
openhuman@<semver>[+<short_git_sha>]
- Frontend (`app/src/utils/config.ts::SENTRY_RELEASE`) builds the tag
from `VITE_BUILD_SHA`.
- Core sidecar (`src/main.rs::build_release_tag`) builds the same tag
from `option_env!("OPENHUMAN_BUILD_SHA")`, so events from both
surfaces group under one release. Cargo's fingerprint already tracks
`option_env!` changes.
Environment separation
- Frontend: new `APP_ENVIRONMENT` export (`development` | `staging` |
`production`) derived from `VITE_OPENHUMAN_APP_ENV`, passed to
`Sentry.init`.
- Core: `resolve_environment` honors `OPENHUMAN_APP_ENV` at runtime,
falling back to `debug_assertions` detection.
Source-map upload
- `@sentry/vite-plugin` added as an app devDependency.
- `vite.config.ts` emits source maps unconditionally and registers the
plugin only when `SENTRY_AUTH_TOKEN` is present, so local dev skips
silently. The plugin uploads `dist/**/*.js{,.map}` under the
canonical release name and then deletes the on-disk `.map` files so
they never ship to end users.
CI wiring (`release.yml` + `release-packages.yml`)
- `Build frontend` and `Build and package Tauri app` both receive
`VITE_BUILD_SHA`, `SENTRY_RELEASE`, `SENTRY_AUTH_TOKEN`, `SENTRY_ORG`,
`SENTRY_PROJECT_FRONTEND`. The tauri step needs the same env because
its `beforeBuildCommand` re-runs `vite build`.
- `Build sidecar core binary` receives `OPENHUMAN_BUILD_SHA` so
`option_env!` bakes the short SHA into the release tag.
- `build-cli-linux-arm64` mirrors `OPENHUMAN_BUILD_SHA` and
`OPENHUMAN_APP_ENV` for the arm64 CLI tarball.
Verification support
- New `openhuman sentry-test` CLI subcommand captures an `Error`-level
event against the currently-initialized client, flushes, and prints
the event UUID. Optional `--panic` flag exercises the panic
integration. Requires a DSN resolvable at runtime or baked in at
compile time; exits non-zero otherwise so misconfiguration is loud.
- `src/main.rs` now loads `.env` before `sentry::init`, so a DSN
defined only in the repo-local dotenv file (common dev case) is
honored by the startup-time Sentry client.
Docs
- `docs/sentry.md` covers the release identifier, environment table,
source-map pipeline, required CI variables, and a verification
runbook with troubleshooting tips.
- `.env.example` + `app/.env.example` document the new build-time vars.
|
||
|
|
b45e0ed262 |
refactor(chat): extract bubble components and helpers from Conversations.tsx (#755)
Split the 1,692-line Conversations.tsx by lifting presentational bubbles and pure helpers into app/src/pages/conversations/. Conversations.tsx now 1,394 lines; semantically identical (verified by Codex + Gemini review). - components/AgentMessageBubble.tsx (+BubbleMarkdown) - components/ToolTimelineBlock.tsx - components/LimitPill.tsx - utils/format.ts (AgentBubblePosition, formatRelativeTime, formatResetTime, getInlineCompletionSuffix, buildAcceptedInlineCompletion) Typecheck clean, 464 tests pass. Co-authored-by: Jwalin Shah <jshah1331@gmail.com> Co-authored-by: WOZCODE <contact@withwoz.com> |
||
|
|
68214579d6 |
feat(onboarding): expand SkillsStep to Gmail+Calendar+Drive+Notion with fallback (#743)
Broaden onboarding integration step from Gmail-only to four high-value toolkits (Gmail, Google Calendar, Google Drive, Notion). Canonicalize backend allowlist slugs and fall back to the curated static catalog when the allowlist is empty or hasn't loaded, so users always see actionable connect targets. - SkillsStep.tsx: 4-toolkit onboarding set, canonicalized slug filter, static fallback when backend allowlist returns nothing - SkillsStep.test.tsx: vitest coverage for fallback, allowlist subset, and connected-source forwarding on continue Co-authored-by: Jwalin Shah <jshah1331@gmail.com> |
||
|
|
4a5a00cb96 |
feat(notifications): webview notifications end-to-end (Phase 7) (#741)
* wip(notifications): core module skeleton + controller registration Phase 7 PR #714 work-in-progress. Shipped: - src/openhuman/webview_notifications/ — mod, types, dispatch, bus, schemas - pub mod declaration in src/openhuman/mod.rs - controllers entry in src/core/all.rs Remaining: schemas line in all.rs, Tauri shell edits (OpenHuman: prefix, feature flag gate, click event emit), new shell module for feature flag state + commands, frontend app/src/lib/webviewNotifications/ IPC wrapper, Redux click handler. * feat(notifications): register webview_notifications schemas in core registry Adds the schemas line next to the already-registered controllers entry so the domain participates in declared-schema validation. v1 controller list is empty (settings live shell-side), but the wiring is in place for future additions. * feat(notifications): add OpenHuman: prefix, feature flag, and click-routing event Updates the CEF notification hook in webview_accounts to: - Prefix OS toast titles with "OpenHuman:" so they visually disambiguate from natively installed apps (Slack, Gmail, Discord desktop) firing the same DM twice. - Gate delivery on a new shell-side feature flag (off by default). - Mirror each fire to the React frontend via a `webview-notification:fired` Tauri event carrying {account_id, provider, title, body, tag}, so the UI can route click-to-focus back to the originating webview. Adds a new notification_settings shell module holding the runtime toggle as an AtomicBool (lock-free read from the CEF callback thread) plus notification_settings_{get,set} Tauri commands for the settings UI to flip the flag. State lives shell-side rather than in the core sidecar so the toggle doesn't require a JSON-RPC round-trip. * feat(notifications): frontend IPC wrapper + Redux click routing Mirrors the Rust-side `webview-notification:fired` Tauri event into Redux so the UI can: - bump an unread badge on the originating account (sidebar surface reads `accounts.unread[accountId]`) - route a subsequent click intent back to the right embedded webview via `focusAccountFromNotification`, which sets `activeAccountId` and clears the unread counter in one action Plumbing: - `app/src/lib/webviewNotifications/` — typed subscribe/unsubscribe, idempotent `started` guard mirroring `webviewAccountService.ts`, `handleNotificationClick(accountId)` as the public click entrypoint so in-app toast UI or a future OS-notification click hook share one Redux intent - `accountsSlice`: `noteWebviewNotificationFired` and `focusAccountFromNotification` reducers (both no-op for unknown account ids — guards against stale payloads from a closed webview) - `App.tsx`: start the service at boot, right next to the existing `startWebviewAccountService()` call Tests: - `accountsSlice.webviewNotifications.test.ts` — reducer behavior - `service.test.ts` — fired dispatches + click focuses via real store Also: `cargo fmt` noise fixups on `src/core/all.rs` and `src/openhuman/webview_notifications/mod.rs` so the branch passes `cargo fmt --check` in CI. --------- Co-authored-by: Jwalin Shah <jshah1331@gmail.com> |
||
|
|
be8cd7b6eb |
feat(chat): show active token consumption and plan status in UI (#703) (#738)
Adds a small pill in the Conversations chat header with: - Session token counter — cumulative input + output tokens across all chat turns, accumulated from chat:done events via new recordChatTurnUsage slice action. - Plan usage badge — % of 5-hour or weekly budget consumed, driven by existing useUsageState. Color-coded sage/amber/coral by severity; click navigates to /settings/billing. No new backend endpoints — session counter is client-local, plan state reuses the existing creditsApi/billingApi fetches cached by useUsageState. Closes #703. Co-authored-by: Jwalin Shah <jshah1331@gmail.com> |
||
|
|
5e0bb97b3b |
feat(ui): hide TTS and autocomplete entry points (#717) (#737)
Removes user-facing entry points for Voice Dictation/TTS and Autocomplete per product deprioritization. Routes, panels, modals, and status hooks are retained so features can be re-enabled by reverting these hunks. Hidden surfaces: - Features settings menu: Autocomplete + Voice Dictation cards - Developer Options: Autocomplete Debug + Voice Debug cards - Skills page: Text Auto-Complete + Voice Intelligence built-in skill cards - Chat input: mic/voice mode toggle button Closes #717, #706. Co-authored-by: Jwalin Shah <jshah1331@gmail.com> |
||
|
|
0dfcb7d71d |
feat(onboarding): add beta notice banner (#729) (#736)
Shows a dismissible amber banner at the top of the onboarding flow on first launch. Dismissal is persisted to localStorage so it only shows once per installation. Co-authored-by: Jwalin Shah <jshah1331@gmail.com> |
||
|
|
8f927369d1 |
feat(telegram): CDP-driven Telegram Web K scanner (#630) (#638)
* feat(accounts): implement accounts management with webview integration - Added a new Accounts page for managing user accounts, including the ability to add and remove accounts. - Introduced AddAccountModal for selecting account providers and initiating account setup. - Implemented WebviewHost to display third-party web applications (e.g., WhatsApp Web) within the app. - Enhanced routing to include a protected route for the Accounts page. - Updated the BottomTabBar to include an Accounts tab for easy navigation. - Integrated Redux for state management of accounts, messages, and logs, ensuring a seamless user experience. - Updated dependencies in Cargo.lock to version 0.52.9 for compatibility. This commit significantly enhances the application's functionality by allowing users to manage accounts directly within the app, improving overall user engagement and experience. * refactor(accounts): clean up Accounts component layout and improve readability - Removed unnecessary comments and simplified the structure of the Accounts component for better clarity. - Adjusted the rendering logic to enhance the layout of the active account section, improving user experience. - Reformatted text in the no accounts message for better readability. - Streamlined the import statements by consolidating related imports, enhancing code organization. * feat(accounts): enhance account management with new providers and routing updates - Introduced support for additional account providers: Telegram, LinkedIn, Gmail, and Slack, expanding user options for account management. - Updated routing to replace the old /conversations path with /chat, streamlining navigation and improving user experience. - Refactored the App component to include an AppShell for better layout management, ensuring the bottom tab bar visibility aligns with the selected account. - Enhanced the BottomTabBar component to reflect the new routing and account options, improving accessibility and usability. - Implemented fullscreen logic for accounts, allowing for a more immersive experience when interacting with selected accounts. - Added utility functions for managing fullscreen states and account provider icons, enhancing code organization and maintainability. This commit significantly improves the application's account management capabilities, providing users with a more flexible and engaging experience. * feat(cef): integrate Chromium Embedded Framework support and enhance webview functionality - Added support for the Chromium Embedded Framework (CEF) as an alternative runtime, allowing for improved webview capabilities. - Updated Cargo.toml to include new dependencies and features for CEF integration, ensuring compatibility with existing Tauri plugins. - Enhanced the WhatsApp recipe to include ghost-text autocomplete functionality, improving user experience during message composition. - Implemented WebSocket observation in the WhatsApp recipe to capture and forward relevant message frames, enhancing real-time interaction. - Introduced user agent spoofing for specific providers to bypass fingerprinting checks, ensuring better compatibility with services like Slack and LinkedIn. - Refactored various components to accommodate the new runtime and improve overall code organization and maintainability. This commit significantly enhances the application's webview capabilities and user interaction with messaging services, providing a more robust and flexible experience. * feat(cef): add development command for CEF and update configuration - Introduced a new development command `dev:cef` in both package.json files to streamline the development process for the Chromium Embedded Framework (CEF). - Updated Cargo.toml to include the `tauri/devtools` feature alongside `tauri/cef`, enhancing debugging capabilities. - Modified tauri.conf.json to adjust visibility settings for the application window and refined the Content Security Policy (CSP) for improved security. - Enhanced resource paths in tauri.conf.json to support recursive file inclusion for better resource management. - Updated the Rust code to bypass macOS Keychain prompts when using CEF, improving user experience during development. This commit enhances the development workflow for CEF integration, providing better tools and configurations for developers. * fix(cef): update development command for CEF to include signing identity - Modified the `dev:cef` command in package.json to include the `APPLE_SIGNING_IDENTITY` environment variable, enhancing the development process for CEF on macOS. - This change improves the build process by ensuring proper code signing during development, streamlining the workflow for developers working with CEF integration. * feat(cef): enhance development command for CEF with safe storage setup - Updated the `dev:cef` command in package.json to include a call to a new script, `setup-chromium-safe-storage.sh`, which pre-seeds the "Chromium Safe Storage" keychain entry with a permissive ACL. - Added the `setup-chromium-safe-storage.sh` script to ensure that CEF/Chromium can read the keychain entry without prompting, improving the development experience on macOS. - This change streamlines the setup process for developers working with CEF integration, ensuring a smoother workflow. * feat(whatsapp): enhance message ingestion and IndexedDB integration - Introduced a new `IngestMessage` interface to standardize message structure for WhatsApp. - Updated `IngestPayload` to include additional fields for better message handling, including `provider`, `chatId`, and `day`. - Implemented a new function `persistWhatsappChatDay` to handle the ingestion of chat messages by day, improving data organization and retrieval. - Enhanced the WhatsApp recipe to utilize IndexedDB for direct data access, eliminating the need for DOM scraping and improving performance. - Updated the Tauri configuration to enable development tools for easier debugging of webview accounts. This commit significantly improves the application's ability to manage and ingest WhatsApp messages, providing a more robust and efficient user experience. * feat(cdp): integrate IndexedDB scanner for WhatsApp via Chrome DevTools Protocol - Added a new module for scanning IndexedDB using the Chrome DevTools Protocol (CDP), enabling direct access to WhatsApp data without DOM scraping. - Implemented a scanner that communicates with the embedded CEF instance to read and decrypt messages stored in IndexedDB. - Updated the Tauri application to manage the new scanner, ensuring it operates seamlessly with existing webview accounts. - Enhanced the Cargo.toml and Cargo.lock files to include necessary dependencies such as `tokio-tungstenite` and `futures-util` for asynchronous operations. - Refactored the WhatsApp recipe to utilize the new scanning capabilities, improving performance and data handling. This commit significantly enhances the application's ability to interact with WhatsApp's IndexedDB, providing a more efficient and robust user experience. * feat(cdp): enhance message diagnostics in IndexedDB scanner - Updated the ScanSnapshot struct to include new fields for message diagnostics: `messageKeyUnion`, `messageTypeBreakdown`, and `sampleByType`, providing a comprehensive overview of message structures and types. - Modified the scanner logic to capture and log detailed information about message types and their shapes, improving debugging capabilities. - Refactored the JavaScript scanner to aggregate message key signatures and counts, enhancing the analysis of message records. This commit significantly improves the application's ability to analyze and log message data from WhatsApp's IndexedDB, facilitating better debugging and data handling. * feat(cdp): implement fast DOM scraping and crypto key extraction for WhatsApp - Introduced a new fast-tick DOM scraping mechanism to extract rendered WhatsApp message bodies, enabling near real-time message updates without relying on IndexedDB. - Added scripts for capturing and logging CryptoKey operations within WhatsApp's workers, allowing for better analysis of key derivations and decryptions. - Enhanced the CDP scanner to interleave fast DOM scans with full IndexedDB scans, optimizing data retrieval and reducing UI spamming during idle periods. - Updated the ScanSnapshot struct to include new fields for DOM-scraped messages and crypto operation statistics, improving the overall diagnostic capabilities of the application. This commit significantly enhances the application's ability to interact with WhatsApp's messaging system, providing a more efficient and responsive user experience. * feat(whatsapp): replace cdp_indexeddb with whatsapp_scanner for enhanced message handling - Replaced the `cdp_indexeddb` module with `whatsapp_scanner` to streamline the scanning process for WhatsApp messages. - Updated the application to manage the new `ScannerRegistry` for WhatsApp, improving the integration with the Chrome DevTools Protocol. - Introduced new scripts for fast DOM scraping and full IndexedDB scanning, optimizing data retrieval and enhancing real-time message updates. - Added a new `dom_scan.js` for efficient extraction of rendered message bodies directly from the DOM, reducing reliance on IndexedDB. - Enhanced the `ScanSnapshot` struct to accommodate new fields for DOM-scraped messages, improving diagnostic capabilities. This commit significantly improves the application's ability to interact with WhatsApp, providing a more efficient and responsive user experience. * refactor(whatsapp): replace JavaScript DOM scanning with Rust-based DOM snapshot - Removed the `dom_scan.js` script and replaced it with a new Rust module `dom_snapshot.rs` that captures DOM snapshots directly via the Chrome DevTools Protocol, enhancing performance and reliability. - Introduced a new `idb.rs` module for scanning WhatsApp's IndexedDB, streamlining data retrieval and improving integration with the Rust backend. - Updated the `ScanSnapshot` struct to accommodate changes in data handling, ensuring compatibility with the new scanning methods. - Enhanced overall message handling capabilities, providing a more efficient and responsive user experience. This commit significantly improves the application's ability to interact with WhatsApp, leveraging Rust for better performance and reducing reliance on JavaScript for DOM operations. * feat(docs): add webview integration playbook for third-party messaging - Introduced a comprehensive playbook detailing the process for integrating third-party webviews (e.g., Instagram, Messenger) into the application. - Documented architecture, workflow, and best practices for building and debugging new integrations, leveraging Rust and Chrome DevTools Protocol. - Included step-by-step instructions for setting up scanners, monitoring logs, and optimizing message handling, ensuring a streamlined development experience for future integrations. This addition enhances the documentation, providing developers with a clear guide to implement and maintain webview integrations effectively. * docs(webview): improve table formatting for clarity - Enhanced the formatting of tables in the webview integration playbook to improve readability and consistency. - Adjusted column headers and alignment for better presentation of job intervals and costs, ensuring clearer communication of scanning processes and common pitfalls. This update aims to provide a more user-friendly documentation experience for developers integrating third-party webviews. * feat(slack): integrate Slack scanner for message extraction and management - Updated the OpenHuman package version to 0.52.15 in both Cargo.lock files. - Introduced a new Slack scanner module to extract messages, users, and channels from Slack's IndexedDB using the Chrome DevTools Protocol. - Added functionality to manage Slack accounts within the application, allowing for automatic opening of Slack webviews based on environment variables. - Enhanced the existing webview account management to support Slack integration, ensuring seamless interaction with the Slack API. This commit significantly improves the application's ability to interact with Slack, providing a robust framework for message handling and account management. * fix(slack): one-doc-per-channel + omit CDP indexName param CEF 146's IndexedDB.requestData rejects `indexName: ""` with "Could not get index"; the CDP spec says empty string means the primary-key index but this backend only accepts the field unset. Omit it entirely so the Slack Redux-persist dump actually comes back. Also switch memory grouping from (channel, day) → channel. Each Slack channel is now one long-running memory doc keyed by channel name (e.g. `general`, `team-product`, `elvin516`), falling back to channel id for non-slug names. Every transcript line carries its own `YYYY-MM-DD HH:MM` stamp and the header records the full date range. `infer_team_id` updated to Slack's real DB naming pattern `objectStore-<TEAM>-<USER>` (not `ReduxPersistIDB:` as initially assumed). * fix(onboarding): update navigation and test descriptions in OnboardingOverlay tests - Changed the navigation path from '/conversations' to '/chat' in the OnboardingOverlay tests to reflect the updated routing logic. - Updated test descriptions for clarity, ensuring they accurately describe the functionality being tested. These changes enhance the accuracy and readability of the onboarding tests, aligning them with the current application flow. * fix(conversations): add return statement to Conversations component - Introduced a return statement in the Conversations component to ensure proper rendering of the sidebar or page variant. - This change enhances the component's functionality by ensuring it returns the expected JSX structure. These modifications improve the overall structure and behavior of the Conversations component. * feat(accounts): add Discord integration and enhance AddAccountModal - Introduced Discord as a new account provider, including its icon and service details. - Updated the AddAccountModal to filter out already connected providers, improving user experience. - Enhanced the UI to display a message when all providers are connected, ensuring clarity for users. - Implemented context menu functionality for account management, allowing users to log out directly from the accounts list. These changes expand the application's capabilities by integrating Discord and refining account management features. * feat(discord): integrate Discord scanner for HTTP and WebSocket monitoring - Added a new `discord_scanner` module to capture Discord API calls and WebSocket frames using the Chrome DevTools Protocol (CDP). - Updated the `lib.rs` to manage the new Discord scanner alongside existing WhatsApp and Slack scanners. - Enhanced the `webview_accounts` module to support Discord account management, including scanner registration and cleanup. These changes expand the application's capabilities by enabling real-time monitoring of Discord interactions, enhancing user experience and functionality. * feat(google-meet): integrate Google Meet as a new account provider - Added Google Meet as a supported account provider, including its icon and service details. - Updated the account management logic to handle Google Meet interactions, including recipe integration for call monitoring and notifications. - Enhanced the UI to accommodate the new provider, ensuring a seamless user experience when managing accounts. These changes expand the application's capabilities by integrating Google Meet, allowing users to join calls and receive notifications directly within the app. * refactor(runtime): remove notification handling and composer autocomplete - Eliminated the notification interception logic and associated functions, streamlining the runtime code. - Removed composer autocomplete features, transferring responsibility for ghost-text overlays to the UI host. - Updated comments to reflect the changes and clarify the remaining functionality. These modifications simplify the runtime script, focusing on core features while delegating UI responsibilities. * feat(google-meet): enhance Google Meet integration with lifecycle event handling - Implemented lifecycle event handling for Google Meet, including events for call start, captions, and call end. - Introduced in-memory storage for caption snapshots during meetings, allowing for the generation of markdown transcripts upon call completion. - Added interfaces for payload structures related to Google Meet events, improving type safety and clarity in the codebase. - Updated the webview account service to manage active meetings and flush transcripts to memory, ensuring a seamless user experience. These changes significantly enhance the Google Meet integration, enabling real-time caption handling and transcript generation, thereby improving the overall functionality of the application. * feat(cef): integrate CEF-based notification handling and update dependencies - Added support for native OS notifications through the `tauri-runtime-cef` crate, enabling interception of browser notifications in embedded webviews. - Introduced a new submodule for `tauri-cef` to manage CEF dependencies and facilitate notification handling. - Updated the `.gitignore` to exclude CEF-related build artifacts and lock files. - Removed the deprecated `notification_scanner` module, streamlining the codebase and focusing on the new CEF integration. - Enhanced the `webview_accounts` module to register and manage CEF browser notifications, improving user experience with real-time alerts. These changes significantly enhance the application's notification capabilities, leveraging CEF for a more integrated and responsive user experience. * feat(cef): update CEF integration and dependency management - Added `cef` and `tauri-runtime-cef` as dependencies to enhance CEF support. - Updated `Cargo.toml` to reference `tauri-runtime-cef` from a local path, ensuring proper integration with the vendored CEF submodule. - Removed direct Git references for Tauri packages, streamlining dependency management by using local paths. These changes improve the application's CEF capabilities and simplify the dependency structure, facilitating better integration and maintenance. * feat(browserscan): add BrowserScan as a new account provider with associated resources - Introduced BrowserScan as a development-only account provider, including its icon and service details. - Updated the account provider types and management logic to accommodate BrowserScan, enhancing the application's capabilities. - Added a new recipe and manifest for BrowserScan, ensuring it integrates seamlessly into the existing webview account lifecycle. - Enhanced the UI to display BrowserScan, providing users with a bot-detection sandbox for testing purposes. These changes expand the application's functionality by integrating BrowserScan, allowing for improved testing and development workflows. * feat(google-meet): enhance Google Meet integration with new transcript handling and session recovery - Introduced a new service to handle Google Meet transcripts, enabling structured note extraction and proactive follow-up actions. - Implemented session recovery logic to manage in-progress meetings when navigating away from the call. - Updated the webview account service to log call events and captions, improving monitoring and debugging capabilities. - Enhanced the Google Meet recipe to persist meeting state across navigations, ensuring seamless user experience. These changes significantly improve the Google Meet integration, allowing for better management of meeting transcripts and user interactions. * refactor(App): reorganize imports and clean up code structure - Removed unused imports from App.tsx to streamline the code. - Adjusted the import order for better readability and consistency. - Enhanced the BottomTabBar component by simplifying the button rendering logic. - Cleaned up the AddAccountModal component by consolidating prop destructuring. - Improved formatting in various components for better code clarity. These changes enhance code maintainability and readability across the application. * update tauri * feat(google-meet): improve caption handling and speaker identification logic - Enhanced the logic for extracting speaker names from Google Meet rows, adding checks to filter out icon ligatures and irrelevant text. - Updated the caption processing to better identify and score caption regions, ensuring more accurate transcript generation. - Introduced new utility functions to differentiate between real captions and icon names, improving the overall reliability of the captioning feature. These changes significantly enhance the accuracy and usability of the Google Meet integration, providing users with clearer and more relevant caption data. * build(cef): bump vendor/tauri-cef to include full cef-helper source tree (#630) Picks up 1b58f715 which fixes the bundler to copy the entire cef-helper/src/ tree instead of only main.rs — required for our CEF helper's Web Notifications interception to link properly in downstream consumers. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * build(cef): patch cef-dll-sys to fix/146-location-windows (#630) The vendor tauri-cef workspace pins cef-dll-sys to the fix/146-location-windows branch via its own [patch.crates-io], but cargo patches do not propagate through path dependencies. Without pinning cef-dll-sys here too, helper processes crash with `CefApp_0_CToCpp called with invalid version -1` because the app-side bindings target a different CEF ABI than what the vendor's cef-helper was built against. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(telegram): scaffold CDP-driven Telegram Web K scanner module (#630) New telegram_scanner module mirrors the Slack/WhatsApp scanner shape from PR #629 but targets Telegram Web K's IndexedDB surface via CDP: - mod.rs: per-account poller + ScannerRegistry; connects to CDP on 127.0.0.1:9222, picks the Telegram target, and runs an IDB tick every 30s. Emits webview:event and POSTs openhuman.memory_doc_ingest so memory fills even when the main window is hidden. - idb.rs: IndexedDB walker — requestDatabaseNames / requestDatabase / requestData, with record caps per store. - extract.rs: peer-grouped message/user/chat extraction from the `tweb` snapshot. cef-only (wry has no remote-debugging port). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(telegram): register scanner registry and dev-auto env (#630) Wires telegram_scanner into the Tauri builder: - Registers ScannerRegistry as managed state (cef-only). - Adds OPENHUMAN_DEV_AUTO_TELEGRAM=<uuid> helper mirroring the Slack / Google Meet dev-auto flow — opens the Telegram Web K account webview 2s after startup so the CDP scanner has a target without manual UI clicks. Useful for iterating on the scanner end-to-end. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(telegram): dispatch scanner on telegram account open/close/purge (#630) Mirrors the slack/discord branches in webview_accounts: - open(provider="telegram"): look up the telegram ScannerRegistry and ensure a CDP scanner is running for this account. - close / purge: forget the account's scanner entry alongside the other providers so we don't leak poll loops. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * style: apply pre-push formatting * fix(telegram): bind CDP scanner to account-marked targets --------- Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Mega MindandGitHub
|
1792135aea |
fix(settings): tool capability toggles now persist and are enforced at runtime (#720)
* chore: update OpenHuman version to 0.52.26 and refine tool preference handling - Bumped the OpenHuman package version to 0.52.26 in Cargo.lock files. - Enhanced the ToolsPanel component to include user feedback on save status. - Implemented filtering of tools based on user preferences in the Rust backend, allowing for more customizable tool availability. - Added new utility functions to map UI tool IDs to Rust tool names for better integration. These changes improve user experience and maintain compatibility with the latest features. * style: apply cargo fmt to user_filter.rs |
||
|
Mega MindandGitHub
|
6dc007850f |
fix(chat): increase agent silence timeout from 2 to 10 minutes (#719)
The frontend silence timer was cutting off long-running agent tasks after 120s of no inference progress signals. Increased to 600s so extended thinking and multi-step tasks can complete without being force-aborted. Manual cancel remains the primary stop mechanism. Closes #715 |
||
|
Steven EnamakelandGitHub
|
ee3f6472ca |
feat: improve sub-agent tooling, conversation timeline UX, and Tauri setup (#646)
* refactor(tauri): update development scripts and configuration for CEF integration - Modified `package.json` scripts to consistently export `CEF_PATH` for all `cargo tauri` commands, ensuring a unified CEF binary distribution location. - Removed the overlay window configuration from `tauri.conf.json` and updated related Rust functions to reflect this change, while retaining helper functions for potential future use. - Updated documentation in `install.md` to clarify the importance of setting `CEF_PATH` for consistent CEF integration across builds. - Enhanced the `ensure-tauri-cli.sh` script to set `CEF_PATH` and ensure proper installation of the vendored CEF-aware `tauri-cli`. These changes streamline the development workflow and improve the reliability of CEF integration in the application. * refactor(release): enhance macOS signing script for nested frameworks and helper apps - Introduced a new `codesign_hardened` function to streamline the signing process with consistent options. - Improved the signing logic for nested frameworks and helper applications, ensuring all binaries are signed correctly. - Updated output messages for better clarity during the signing process, including detailed listings of bundle contents. - Disabled the summarizer payload threshold in the configuration to prevent recursive invocations until the issue is resolved. These changes improve the reliability and maintainability of the macOS signing and notarization workflow. * feat(orchestrator): add current_time tool and enhance agent capabilities - Introduced the `current_time` tool to provide the current date and time in UTC and local time zones, facilitating scheduling and reminders. - Updated `agent.toml` to include new tools: `current_time`, `cron_add`, `cron_list`, `cron_remove`, and `schedule`, enhancing the orchestrator's functionality. - Expanded documentation in `prompt.md` to guide users on utilizing the new direct tools effectively. These changes improve the orchestrator's ability to handle time-related queries and scheduling tasks directly, enhancing user experience. * feat(gmail): implement post-processing for Gmail responses to convert HTML to markdown - Added a new `post_process` module specifically for Gmail, which modifies action responses to convert HTML content into markdown format, improving usability and reducing context token usage. - Enhanced the `ComposioProvider` trait with a `post_process_action_result` method to allow providers to handle response modifications. - Introduced a `post_process` function that checks for a `raw_html` flag in the arguments to determine whether to apply the conversion. - Implemented tests to validate the HTML detection and conversion logic, ensuring the integrity of the post-processing functionality. These changes enhance the handling of Gmail responses, making them more suitable for further processing and display in the application. * refactor(gmail): improve HTML to markdown conversion and tool result budget - Enhanced the `post_process` module for Gmail to handle large HTML payloads more efficiently, implementing a fallback mechanism for oversized content. - Updated the `DEFAULT_TOOL_RESULT_BUDGET_BYTES` to `0`, disabling the budget temporarily while reworking the oversized-output path. - Refined the `extract_markdown_body` function to better manage HTML content, ensuring cleaner markdown output and improved performance. - Added utility functions for stripping HTML noise and handling large email bodies, enhancing the overall robustness of the email processing logic. These changes optimize the handling of Gmail responses, improving usability and performance in processing large HTML content. * feat(thread): implement thread title generation from user and assistant messages - Added a new `generateTitleIfNeeded` function in the `threadApi` to create a thread title based on the first user message and the assistant's reply. - Introduced a new `GenerateConversationThreadTitleRequest` struct to handle requests for title generation. - Updated the `ChatRuntimeProvider` to dispatch the title generation action after processing inference responses. - Enhanced the `threadSlice` with a new async thunk for generating thread titles, ensuring proper error handling and thread loading. - Added tests for the new title generation functionality to validate the integration with the threads RPC. These changes improve the user experience by automatically generating relevant thread titles, enhancing the organization of conversations. * feat(conversations): add thread title update functionality - Implemented `update_thread_title` method in `ConversationStore` to allow updating the title of existing conversation threads. - Added a corresponding public function `update_thread_title` for external access. - Enhanced tests to verify that thread titles are correctly updated and persisted in the store. These changes improve the management of conversation threads by enabling dynamic title updates, enhancing user experience and organization. * feat(docs): add comprehensive agent and subagent tool flow documentation - Introduced a new document detailing the runtime flow of the agent harness, including execution paths for main agents and tools. - Explained the differences between typed and fork subagents, and provided guidance for debugging harness and delegation issues. - Included a file map outlining key components and their roles within the Rust implementation. - Added a flow diagram to visually represent the interaction between agents, tools, and subagents. These changes enhance the understanding of the agent architecture and improve the documentation for developers working with the system. * feat(subagent): implement extraction tool and handoff cache for oversized results - Introduced `extract_from_result` tool to allow targeted queries against oversized tool outputs, improving efficiency by directly interacting with the extraction model. - Added `ResultHandoffCache` to manage oversized payloads, enabling progressive disclosure and reducing context length issues in sub-agent history. - Implemented hygiene helpers for cleaning tool outputs before caching, ensuring only relevant data is stored. - Enhanced the sub-agent runner with new modules for tool preparation and execution, streamlining the overall agent workflow. These changes enhance the sub-agent's ability to handle large tool results effectively, improving performance and user experience. * feat(conversations): enhance message bubble rendering and timeline entry formatting - Introduced new utility functions for parsing and rendering agent messages, including `splitAgentMessageIntoBubbles` and `parseMarkdownTable`, to improve the display of messages in conversation threads. - Implemented `BubbleMarkdown` and `TableCellMarkdown` components for better formatting of user and agent messages, ensuring consistent styling and interaction. - Enhanced the `formatTimelineEntry` function to provide clearer titles and details for tool timeline entries, improving the user experience during interactions with subagents. - Updated the `ChatRuntimeProvider` to utilize the new formatting functions, ensuring that tool timeline entries are displayed with relevant context and detail. These changes improve the overall presentation and usability of conversation messages and tool interactions, enhancing user engagement and clarity. * feat(subagent): enforce restrictions on sub-agent spawning tools - Introduced a filter to prevent sub-agents from invoking their own spawning tools, specifically `spawn_subagent` and `delegate_*`, to avoid recursion issues and ensure proper delegation by the top-level orchestrator. - Updated the `is_subagent_spawn_tool` function to identify these tools and integrated checks in both `run_typed_mode` and `run_fork_mode` to maintain the integrity of the sub-agent execution environment. - Enhanced logging to track the removal of restricted tools from the sub-agent's tool surface, improving observability and debugging capabilities. These changes strengthen the sub-agent architecture by enforcing strict boundaries on tool invocation, enhancing stability and performance. * feat(conversations): add ToolTimelineBlock component and enhance timeline entry formatting - Introduced the `ToolTimelineBlock` component to display tool timeline entries with improved formatting and user interaction, including auto-expansion for running entries. - Enhanced the `formatTimelineEntry` function to include user-friendly titles for specific tool actions, such as 'Viewing your Integrations'. - Updated the rendering logic in the `Conversations` component to filter and display visible messages more effectively, improving user experience during conversations. These changes enhance the clarity and usability of tool interactions within conversation threads, providing users with better context and engagement. * refactor(conversations): streamline component imports and enhance formatting consistency - Consolidated import statements in `Conversations.tsx` and `ChatRuntimeProvider.tsx` for improved readability. - Refactored the `ToolTimelineBlock` component to simplify its props structure. - Enhanced formatting consistency in the rendering logic of agent message bubbles and timeline entries, ensuring cleaner code and better maintainability. - Updated test cases for `splitAgentMessageIntoBubbles` and `formatTimelineEntry` to reflect formatting changes and ensure accuracy. These changes improve code clarity and maintainability while enhancing the overall user experience in conversation threads. * fix: satisfy pre-push lint on fix/tauri * fix(chat): refresh usage counters after responses * refactor(ChatRuntimeProvider): remove redundant import of requestUsageRefresh - Eliminated the duplicate import statement for `requestUsageRefresh` in `ChatRuntimeProvider.tsx`, streamlining the code for better readability and maintainability. * fix(chat): satisfy pre-push checks * refactor(conversations): improve error handling and remove unused title generation logic - Removed the unused `generateThreadTitleIfNeeded` function call from the `Conversations` component, simplifying the message dispatch logic. - Enhanced error handling during message dispatch by using `unwrap()` to catch and log errors, providing clearer feedback on send failures. - Updated the `ChatRuntimeProvider` to ensure proper error logging when generating thread titles, improving observability of issues related to title generation. These changes streamline the conversation handling process and improve the robustness of error management in the chat system. * refactor(conversations): improve formatting of title redaction and streamline test assertions - Reformatted the `redact_title_for_log` function for better readability by adjusting the formatting of the output string. - Simplified the assertion in the timezone test to enhance clarity and maintainability. These changes contribute to cleaner code and improved test structure in the conversations module. * refactor(tokenjuice): sort fact parts for improved formatting in inline summary - Modified the `format_inline` function to sort the fact parts before generating the summary string. This change enhances the readability and consistency of the output by ensuring that facts are presented in a stable order. These changes contribute to better formatted summaries in the token juice module. * fix: address remaining CodeRabbit review comments * refactor: streamline error handling and improve code readability - Updated error handling in prompt loading to use `std::io::Error::other` for better clarity. - Simplified conditional checks using `is_none_or` and `is_some_and` for improved readability. - Refactored string trimming logic to utilize array syntax for better clarity. - Enhanced default implementations for several structs to reduce boilerplate code. These changes contribute to cleaner code and improved maintainability across various modules. * refactor: improve code formatting and structure - Adjusted formatting in `post_process.rs` for better readability by aligning the conditional block. - Combined derive attributes in `tools.rs` for the `ComputerControlConfig` struct to reduce redundancy. - Streamlined entry retrieval in `compatible.rs` by condensing multiple lines into a single line for clarity. These changes enhance code readability and maintainability across the affected modules. |
||
|
Steven EnamakelandGitHub
|
8f4696bdfb |
feat(composio): per-toolkit tool curation, user scopes, and Gmail HTML→markdown (#643)
* feat(composio): add user scope management for toolkits - Introduced `get_user_scopes` and `set_user_scopes` functions to manage per-toolkit user scope preferences, allowing for read, write, and admin classifications. - Updated `all_controller_schemas` and `all_registered_controllers` to include new schemas for user scope management. - Implemented `evaluate_tool_visibility` to determine tool visibility based on user-defined scopes, enhancing security and control over tool actions. - Added `UserScopePref` struct to store user preferences and integrated it with memory storage for persistence. - Enhanced existing tools to respect user scope preferences during execution, ensuring actions align with user-defined permissions. These changes improve the flexibility and security of toolkit interactions, allowing users to customize their access levels for different actions. * feat(gmail): expand GMAIL_CURATED tools for enhanced email management - Updated the GMAIL_CURATED constant to include additional tools for reading and writing emails, such as GMAIL_LIST_MESSAGES, GMAIL_LIST_THREADS, GMAIL_GET_ATTACHMENT, and GMAIL_FORWARD_MESSAGE. - Improved organization of tools by categorizing them under distinct sections for reading messages, managing drafts, and handling labels. - Enhanced admin tools with new functionalities like GMAIL_BATCH_DELETE_MESSAGES and GMAIL_UNTRASH_THREAD, improving overall email management capabilities. These changes provide a more comprehensive toolkit for interacting with Gmail, enhancing user experience and functionality. * refactor(tool_scope, gmail): improve code formatting and organization - Reformatted the `ADMIN` and `GMAIL_CURATED` constants for better readability by aligning the entries vertically. - Enhanced the clarity of the `classify_unknown` function by improving the structure of the constants, making it easier to maintain and understand. - Updated test assertions in `toolkit_from_slug` for consistency in formatting, ensuring clearer test outputs. These changes improve the overall readability and maintainability of the codebase, aligning with ongoing refactoring efforts. * feat(github): add GitHub toolkit and curated tools for enhanced integration - Introduced a new GitHub provider module, including a curated catalog of GitHub actions tailored for common tasks such as repository management, issue tracking, and pull request handling. - Implemented the `catalog_for_toolkit` function to allow fallback to a static curated list for toolkits without a native provider, ensuring consistent tool visibility and access. - Updated the `evaluate_tool_visibility` function to prioritize curated tools from registered providers, enhancing the overall user experience and security by enforcing whitelist checks. These changes expand the capabilities of the Composio toolkit, providing users with a comprehensive set of tools for interacting with GitHub, while maintaining a focus on user-defined permissions and visibility. * refactor(tools): improve formatting and organization of curated tools - Reformatted the `GITHUB_CURATED`, `NOTION_CURATED`, and other tool constants for better readability by aligning entries vertically. - Enhanced the clarity of the code structure, making it easier to maintain and understand. - Updated related documentation to reflect the changes in formatting and organization. These changes improve the overall readability and maintainability of the codebase, aligning with ongoing refactoring efforts. * feat(catalogs): introduce curated catalogs for various toolkits - Added a new module `catalogs.rs` containing curated tool lists for Slack, Discord, Google Calendar, Google Drive, Google Docs, and more, enhancing the Composio toolkit's integration capabilities. - Updated the `mod.rs` file to include the new `catalogs` module and modified the `catalog_for_toolkit` function to support these curated lists, allowing for better organization and access to toolkit actions. - This addition improves the overall functionality and user experience by providing a comprehensive set of tools for interacting with popular platforms. * feat(post-process): implement HTML to markdown conversion for Gmail responses - Introduced a new `post_process` module to handle per-toolkit response modifications, specifically for converting HTML content to markdown format. - Enhanced the `ComposioExecuteTool` to apply post-processing on successful responses, improving the clarity and usability of data returned from Gmail. - Added tests to ensure the correct functionality of HTML detection and conversion, validating the integrity of the post-processing logic. These changes enhance the user experience by streamlining the handling of HTML content in responses, making it more suitable for further processing and display. * refactor(post_process): reorganize HTML detection constants for improved readability - Moved HTML detection markers in the `looks_like_html` function to a more structured format, enhancing clarity and maintainability. - This change aligns with ongoing efforts to improve code organization and readability within the post-processing module. * refactor(catalogs): enhance formatting and organization of curated tool constants - Reformatted the `SLACK_CURATED`, `DISCORD_CURATED`, and `GOOGLECALENDAR_CURATED` constants for improved readability by aligning entries vertically. - This change enhances the clarity and maintainability of the code, aligning with ongoing refactoring efforts to improve code organization. * feat(connect-modal): wire read/write/admin scope toggles to composio prefs Adds the three scope toggles to the connected-state of the ComposioConnectModal so users can gate which Composio actions the agent may invoke per integration. Loads the stored pref via `composio_get_user_scopes` once the modal lands in the connected phase and persists changes through `composio_set_user_scopes` with optimistic updates and rollback on error. - Adds `getUserScopes` / `setUserScopes` to composioApi. - Adds `ComposioUserScopePref` type mirror. - Renders accessible role="switch" toggles with hint text per row. * refactor(ComposioConnectModal): simplify SCOPE_ROWS definition - Streamlined the definition of SCOPE_ROWS in ComposioConnectModal by removing unnecessary line breaks, enhancing code readability and maintainability. - Updated the agent.toml configuration to include "composio_execute" in the tools list, expanding the capabilities of the integrations agent. * fix(composio): apply curated whitelist + scope pref to integrations_agent prompt The agent prompt for integrations_agent was rendering every action returned by the backend's `composio_list_tools` for each connected toolkit, bypassing the curation/scope filter that the meta-tool layer applies. Concretely the GitHub integrations_agent prompt was showing ~500 actions including non-curated entries like GITHUB_ADD_OR_UPDATE_TEAM_REPOSITORY_PERMISSIONS. Adds `is_action_visible_with_pref(slug, pref)` — a sync helper that mirrors the meta-tool layer's decision logic — and applies it in: - `fetch_connected_integrations_uncached` (bulk session-cached path) - `fetch_toolkit_actions` (per-toolkit spawn-time path) One pref load per toolkit (not per action) keeps the cost minimal. * refactor(fetch_connected_integrations): streamline action visibility filter Simplified the action visibility filter in `fetch_connected_integrations_uncached` by consolidating the filter logic into a single line. This change enhances code readability while maintaining the existing functionality of applying user preferences to the displayed actions. * fix(prompt): drop duplicate `### Available Tools` listing in text-mode preamble Text-mode subagent prompts were rendering the tool catalog twice: once in the prompt template's `## Tools` section (with the richer `Call as: NAME[arg|arg]` signatures from `prompts::ToolsSection::build`) and once in `### Available Tools` under `## Tool Use Protocol` (`Parameters: name:type, ...` format). For an integrations_agent toolkit spawn (~50 actions) this doubled the tool listing bytes for no informational gain. Keep only the protocol preamble (essential for text mode); the catalog stays in `## Tools`. Removes `summarise_parameters` and `first_line_truncated` which were the sole consumers, plus the now-unused `std::fmt::Write` import. * review: address PR feedback (UTF-8 boundary, structured logs, doc, debug) Real fixes: - post_process: walk back to UTF-8 char boundary before truncating to 4096 bytes; previous `&s[..4096]` could panic mid-codepoint. Adds regression test that places a 3-byte char straddling the cutoff. - composioApi: move misplaced `execute` docstring back above `execute` (it had drifted above the new `getUserScopes`). - schemas: include `get_user_scopes` / `set_user_scopes` in the `every_known_schema_key_resolves` test. Diagnosability: - schemas: structured `[composio:scopes]` debug/error logs at entry, exit, and every early-return in `handle_get_user_scopes` / `handle_set_user_scopes` (method + toolkit + pref fields). The memory-not-ready branch now logs an error before returning. - composioApi + ConnectModal: grep-friendly `[composio][scopes]` debug logs around getUserScopes / setUserScopes RPC round-trips and the toggle handler (old → new state, persisted result, errors). - user_scopes: load_or_default now logs the same normalized `key` that `load()` does, so traces correlate across both code paths. Cleanups: - tools: replace external `idx` + `Vec::retain` in `filter_list_tools_response` with a drain + zip + filter_map pattern. - tool_scope: document the no-underscore-in-toolkit-name assumption of `toolkit_from_slug`, and call out the `microsoft_teams` alias. - Cargo.toml: comment on the html2md vs htmd choice. Pushback (no change): - Reviewer asked to split the type-only import in ConnectModal into a separate `import type` line; ESLint's `no-duplicate-imports` rejects that, so the inline `type` form (functionally identical) stays. |
||
|
Steven EnamakelandGitHub
|
fff24a6c6f |
fix(prompts): v2 prompt pipeline, integrations_agent, and subagent session plumbing (#642)
* refactor(transcript): update session transcript paths and enhance directory structure
- Changed the source of truth path for session transcripts from `sessions/{DDMMYYYY}/{agent}_{index}.jsonl` to `session_raw/{DDMMYYYY}/{agent}_{index}.jsonl` to better reflect the organization of files.
- Updated the logic for creating and resolving transcript paths to accommodate the new directory structure, ensuring compatibility with legacy `.md` files.
- Improved documentation to clarify the changes in file organization and their implications for transcript management.
This refactor enhances the clarity and maintainability of session transcript handling by establishing a more logical file structure.
* refactor(prompts): update tool handling in prompt builders
- Replaced `Vec<String>` with `Vec<ToolSummary<'_>>` for available tools in multiple agent prompt builders, enhancing type safety and clarity.
- Introduced `render_tool_catalog` and `render_connected_integrations` functions to dynamically generate sections in prompts based on available tools and connected integrations.
- Updated the `build` function in various agent prompts to utilize the new rendering functions, ensuring that prompts accurately reflect the current context and available resources.
These changes improve the maintainability and functionality of the prompt generation process across different agents.
* refactor(prompts): streamline prompt builders for agent templates
- Updated the prompt builders for various agents to utilize the sibling `prompt.md` template directly, enhancing clarity and maintainability.
- Replaced `Vec<ToolSummary<'_>>` with `Vec<ToolSummary>` and `Vec<ConnectedIntegration>` for improved type safety in test cases.
- Adjusted the `build` function to ensure consistent formatting and handling of tool catalogs across different agents.
These changes simplify the prompt generation process and prepare the codebase for future enhancements.
* refactor(harness): unify tool filtering and prompt loading for debug consistency
- Exposed `filter_tool_indices` and `load_prompt_source` as `pub(crate)` to ensure that both the live runner and debug dump share the same filtering and loading logic, eliminating discrepancies.
- Enhanced documentation for both functions to clarify their purpose and usage, improving maintainability and understanding of the codebase.
These changes streamline the tool management process and enhance the reliability of debug outputs, ensuring consistency across different contexts.
* refactor(prompts): enhance prompt builders for agent templates
- Updated the prompt builders for various agents to return fully-assembled system prompts, incorporating section helpers from `crate::openhuman::context::prompt`.
- Replaced `Vec<ToolSummary<'_>>` with `Vec<ToolSummary>` and `Vec<ConnectedIntegration>` for improved type safety in test cases.
- Adjusted the `build` function to ensure consistent formatting and handling of user files, tools, and workspace sections across different agents.
These changes streamline the prompt generation process, improve maintainability, and prepare the codebase for future enhancements.
* refactor(prompts): enhance prompt context handling for dynamic sources
- Updated the prompt builders to support fully-assembled prompts from dynamic sources, allowing for more flexible prompt generation.
- Introduced `PromptTool` and `PromptContext` structures to replace `ToolSummary`, improving type safety and clarity in prompt construction.
- Refactored the handling of prompt sources in both the subagent runner and session builder to streamline the integration of dynamic prompts and legacy sources.
These changes improve the maintainability and functionality of the prompt generation process, ensuring accurate representation of available tools and context in agent interactions.
* refactor(prompts): improve prompt context and tool handling
- Enhanced the `PromptContext` structure to include additional fields for better context management, such as `skills`, `dispatcher_instructions`, and `tool_call_format`.
- Replaced `ToolSummary` with `PromptTool` for improved type safety and clarity in prompt generation.
- Updated the handling of dynamic prompt sources in both the subagent runner and debug dump, ensuring consistent integration and rendering of prompts.
- Introduced a mechanism to handle empty visible tool names, enhancing the robustness of prompt generation.
These changes streamline the prompt construction process and improve the overall maintainability of the codebase.
* refactor(prompts): reorganize prompt handling and introduce SystemPromptBuilder
- Moved prompt-related types and builders from `openhuman::context::prompt` to `openhuman::agent::prompts` for better modularity.
- Introduced `SystemPromptBuilder` to streamline the construction of system prompts, allowing for flexible section management.
- Updated module exports to maintain compatibility while enhancing the organization of prompt-related code.
These changes improve the clarity and maintainability of the prompt generation process, aligning it more closely with the agents that utilize these prompts.
* refactor(prompts): unify agent prompt handling and update CLI references
- Removed the "main" alias for the orchestrator in the prompt dumping process, treating it as just another registered agent.
- Updated the `debug-agent-prompts.sh` script to reflect this change, ensuring all agents are included uniformly.
- Revised documentation and error messages in `agent_cli.rs` to replace references to "main" with "orchestrator" for clarity.
- Enhanced the debug dump functionality to maintain consistency across agent prompts, improving overall maintainability and usability.
These changes streamline the prompt handling process and clarify the usage of agent identifiers in the CLI, aligning with the new architecture.
* refactor(prompts): remove CACHE_BOUNDARY references from agent prompts
- Eliminated the CACHE_BOUNDARY marker from various agent prompt files, streamlining the prompt generation process.
- Updated the build functions in multiple agents to ensure consistent handling of workspace rendering without the cache boundary.
- Refactored related prompt handling logic to enhance clarity and maintainability, aligning with the new architecture.
These changes simplify the prompt structure and improve the overall efficiency of prompt generation across agents.
* refactor(prompts): remove cache boundary references from tests and prompts
- Eliminated all instances of cache boundary references from the subagent runner and related tests, simplifying the prompt handling logic.
- Updated test assertions to reflect the removal of cache boundary checks, ensuring consistency across the testing framework.
- Refactored the session manager to streamline the system prompt assembly process without relying on cache boundaries.
These changes enhance the clarity and maintainability of the prompt generation process, aligning with the recent architectural updates.
* refactor(harness): clean up unused imports and streamline code
- Removed unnecessary imports from multiple files, including `RandomState`, `Hasher`, and `SerializeMap`, to enhance code clarity and maintainability.
- Simplified the structure of several modules by eliminating redundant use statements, ensuring a cleaner and more efficient codebase.
These changes contribute to a more organized and readable code structure, aligning with ongoing refactoring efforts.
* refactor(prompts): enhance agent prompt structures and integration handling
- Updated the `orchestrator`, `skills_agent`, and `welcome` prompts to streamline the rendering of connected integrations and delegation guides.
- Introduced dedicated functions for rendering integration information, ensuring clarity in the agent's voice and responsibilities.
- Removed redundant sections from the shared prompt builder, allowing each agent to manage its own prompt content more effectively.
- Improved test coverage for prompt generation, ensuring accurate representation of connected integrations and skills.
These changes enhance the maintainability and clarity of the prompt generation process, aligning with the recent architectural updates.
* refactor(session): update integration handling and clean up prompt parameters
- Revised documentation for `connected_integrations` in the `Agent` struct to clarify its role in the agent's prompt rendering.
- Updated the parameter name in `render_subagent_system_prompt_with_format` to `_connected_integrations` to indicate it is unused, enhancing code clarity.
- Cleaned up import statements in the context module for better organization and maintainability.
These changes improve the clarity of integration handling and streamline the code structure, aligning with ongoing refactoring efforts.
* refactor(agent_cli): simplify command options and improve documentation
- Removed the `--skill` option from the `dump-prompt` command, streamlining the command usage and focusing on essential parameters.
- Updated documentation to clarify the usage of the `dump-prompt` command and its parameters, enhancing user understanding.
- Cleaned up the `DumpFlags` structure by removing unused fields, contributing to a more maintainable codebase.
These changes improve the clarity and usability of the agent CLI, aligning with ongoing refactoring efforts.
* refactor(cli): streamline dotenv loading and clean up prompt rendering
- Introduced `load_dotenv_for_cli` to load environment variables for all CLI entrypoints, ensuring consistent configuration across commands.
- Updated documentation to clarify the purpose of the dotenv loading mechanism.
- Removed unnecessary blank lines in prompt rendering functions across multiple agents, enhancing code readability.
These changes improve the maintainability and clarity of the CLI and prompt handling, aligning with ongoing refactoring efforts.
* refactor(debug-agent-prompts): enhance environment loading and streamline workspace resolution
- Updated the script to load environment variables from a `.env` file, ensuring consistent configuration for prompt generation.
- Simplified workspace resolution by delegating to the binary's internal logic, improving reliability and reducing code duplication.
- Revised documentation to clarify the usage of command options and the impact of environment variables on prompt rendering.
These changes improve the maintainability and clarity of the debug agent prompts script, aligning with ongoing refactoring efforts.
* refactor(agent): remove category filter and simplify agent definitions
- Eliminated the `category_filter` from various agent definitions and related tests, streamlining the agent configuration.
- Updated the `run_list` function in `agent_cli.rs` to reflect the removal of category filtering, enhancing output clarity.
- Revised documentation and comments to remove references to the now-removed category filter, improving overall code maintainability.
These changes contribute to a cleaner and more efficient agent architecture, aligning with ongoing refactoring efforts.
* feat(agents): introduce integrations_agent and tools_agent for enhanced service handling
- Added the `integrations_agent` to manage service integrations via Composio, including a new TOML configuration and prompt structure.
- Introduced the `tools_agent` for general ad-hoc tasks using built-in OpenHuman tools, with its own configuration and prompt.
- Updated the loader to include both agents in the built-in agent list, increasing the total number of agents from 13 to 14.
- Revised orchestrator and welcome prompts to delegate integration tasks to the new `integrations_agent`, ensuring clarity in agent responsibilities.
- Enhanced tests to verify the registration and functionality of the new agents, improving overall test coverage.
These changes expand the capabilities of the agent architecture, allowing for more specialized handling of integrations and tool usage.
* refactor(agents): update references from skills_agent to integrations_agent
- Changed all instances of `skills_agent` to `integrations_agent` across various files, including prompts, CLI commands, and tool registrations.
- Updated documentation and comments to reflect the new agent name, ensuring clarity in agent responsibilities and usage.
- Revised debug scripts to align with the new prompt structure for the integrations agent.
These changes enhance consistency in the codebase and improve the clarity of agent interactions.
* refactor(agents): rename skills_agent to integrations_agent throughout the codebase
- Updated all instances of `skills_agent` to `integrations_agent` in various files, including tests, documentation, and comments.
- Ensured consistency in agent references to improve clarity in agent responsibilities and interactions.
- Revised related code structures to align with the new naming convention, enhancing overall maintainability.
These changes support the transition to the new agent architecture and improve code readability.
* refactor(prompts): implement dynamic prompt rendering for enhanced context handling
- Introduced `DynamicPromptSection` to allow prompts to be built dynamically using a function pointer, enabling real-time access to the `PromptContext`.
- Updated `SystemPromptBuilder` to support dynamic prompts, ensuring that late-arriving state like `connected_integrations` is accurately reflected in the rendered output.
- Revised the prompt handling logic in the agent builder to streamline the integration of dynamic prompts, improving overall flexibility and responsiveness.
These changes enhance the prompt generation process, aligning with the ongoing improvements in agent architecture and context management.
* refactor(prompts): refine delegation guide to display only connected integrations
- Updated the `render_delegation_guide` function to list only the toolkits that are actively connected, omitting unauthorized toolkits to prevent hallucinations during delegation.
- Revised related tests to ensure the prompt correctly reflects the current state of integrations, including scenarios where no integrations are connected.
- Introduced a new utility function to filter out welcome-only tools from non-welcome agents, enhancing the clarity and safety of tool visibility.
These changes improve the accuracy and focus of the delegation guide, aligning with the ongoing enhancements in agent prompt handling.
* refactor(planner): update tool usage and prompt guidelines for read-only operations
- Modified the `agent.toml` configuration to clarify that the planner operates in a read-only mode, specifying that it does not mutate the workspace or memory.
- Revised the prompt guidelines to reflect the read-only nature of the planner, emphasizing the need for explicit nodes for any required writes to be handled by downstream agents.
These changes enhance the clarity of the planner's operational constraints and improve the overall structure of the planning process.
* refactor(config): remove web search enable flag and update related configurations
- Eliminated the `OPENHUMAN_WEB_SEARCH_ENABLED` environment variable and associated logic, as web search is now always enabled by default.
- Updated the configuration schema to reflect the removal of the enable flag from `WebSearchConfig`.
- Adjusted tool registration to ensure web search is always available, simplifying the configuration process.
These changes streamline the web search functionality, ensuring it is consistently available across all sessions.
* refactor(config): update http_request flag to always enabled
- Changed the `http_request` configuration to always be enabled, removing the dependency on the `config.http_request.enabled` flag.
- This adjustment simplifies the configuration process and ensures consistent behavior across the application.
These changes contribute to a more streamlined configuration and enhance the overall reliability of the onboarding process.
* refactor(debug-agent-prompts): transition to dump-all command for agent prompts
- Replaced the previous method of listing agent IDs and dumping prompts with a new `dump-all` command that consolidates the functionality into a single call.
- Updated the script to handle output directory and workspace options more efficiently, leveraging Rust's `dump_all_agent_prompts` for processing.
- Enhanced the handling of the `integrations_agent` to generate separate dumps for each connected toolkit, improving the clarity and organization of output files.
- Revised related logging and summary generation to reflect the new structure, ensuring a more streamlined user experience.
These changes modernize the prompt dumping process, aligning it with the latest architectural improvements and enhancing usability.
* feat(composio): implement dynamic fetching of toolkit actions for integrations
- Added a new `fetch_toolkit_actions` function to retrieve the current action catalogue for a specified Composio toolkit, enhancing the responsiveness of the integrations agent.
- Updated the `subagent_runner` to utilize the fresh action list at spawn time, ensuring that the toolkit's actions reflect the latest backend state.
- Modified the `render_integrations_agent` function to refresh the action catalogue during prompt generation, improving the accuracy of the displayed tools.
These changes enhance the integration experience by providing up-to-date action information, aligning with the ongoing improvements in agent functionality.
* refactor(integrations-agent): update tool visibility and configuration handling
- Modified the `agent.toml` to replace `wildcard` with `named` tools, enhancing control over tool visibility for the integrations agent.
- Updated the `subagent_runner` to ensure that tool visibility aligns with the new TOML configuration, preventing unnecessary stripping of tools.
- Revised the `render_integrations_agent` function to respect the updated tool scope, improving the accuracy of the tool list generated for subagents.
These changes streamline the tool management process, ensuring that only explicitly defined tools are available during agent execution.
* feat(composio): add composio_list_connections tool for dynamic integration detection
- Introduced the `composio_list_connections` tool in the orchestrator's configuration, allowing the agent to detect newly-authorized Composio integrations mid-session.
- Enhanced the `ComposioListConnectionsTool` to filter and return only currently-connected integrations with ACTIVE or CONNECTED status, improving the accuracy of integration management.
- Updated the tool's description to clarify its functionality and usage context.
These changes enhance the agent's ability to manage integrations dynamically, aligning with ongoing improvements in the integration experience.
* fix(prompt): update delegation guide to reference Skills page
- Modified the `render_delegation_guide` function to change the reference from **Settings → Integrations** to the **Skills** page for connecting integrations. This update clarifies the user instructions for integration management.
* feat(session): introduce session key management for sub-agents
- Added `session_key` and `session_parent_prefix` fields to `ParentExecutionContext` to facilitate hierarchical transcript naming for sub-agents.
- Updated `persist_subagent_transcript` to generate transcript filenames based on the parent's session key, ensuring a flat file structure that reflects the parent-child relationship.
- Enhanced `AgentBuilder` and `Agent` to support the new session key management, allowing for better organization of session transcripts.
- Adjusted related functions to ensure proper handling of session keys during agent execution and transcript persistence.
These changes improve the clarity and organization of session transcripts, aligning with the ongoing enhancements in agent functionality.
* refactor(tests): update integrations agent tests for tool scope and transcript handling
- Renamed the `integrations_agent_is_wildcard` test to `integrations_agent_tool_scope_honours_toml` to better reflect its purpose of validating tool scope based on TOML configuration.
- Enhanced the test to assert that the `integrations_agent` correctly recognizes named tools instead of a wildcard.
- Removed the `integrations_agent_has_extra_tools_for_export` test as it is no longer relevant to the current tool management strategy.
- Improved the `latest_in_dir` function to clarify the handling of transcript naming schemes, ensuring proper differentiation between legacy and keyed formats.
These changes streamline the testing process and improve the accuracy of tool scope validation, aligning with recent updates in agent functionality.
* refactor(subagent_runner): improve transcript persistence and streamline inner loop
- Removed the `persist_subagent_transcript` function, transitioning transcript persistence to occur per-iteration within the `run_inner_loop`, enhancing reliability by ensuring transcripts are written immediately after each provider response.
- Updated the handling of session keys to maintain consistent naming for transcripts, reflecting the parent-child relationship in the file structure.
- Simplified the code by eliminating redundant post-loop transcript writes, aligning with recent changes in agent functionality and improving overall clarity in transcript management.
* refactor(agent_cli, subagent_runner, session): improve code readability and formatting
- Enhanced formatting in `agent_cli.rs` for better readability by adjusting the structure of string formatting.
- Streamlined conditional checks in `subagent_runner.rs` to improve clarity and maintainability.
- Simplified the handling of agent IDs in `builder.rs` to reduce line length and improve code flow.
- Updated test cases in `tests.rs` for better alignment and readability of expected values.
- Improved formatting in `debug_dump.rs` to enhance the clarity of toolkit action fetching and logging.
These changes collectively enhance the overall readability and maintainability of the codebase, aligning with ongoing refactoring efforts.
* fix(tests): add missing session_key fields to ParentExecutionContext stub
* fix: address PR review feedback
* fix(prompts-v2): round 2 PR review — dispatcher instructions, workspace-file preservation, cached-tool fallback, transcript persistence
- subagent_runner: populate PromptContext.dispatcher_instructions for Dynamic prompts (was empty string, dropping the ## Tool Use Protocol block in render_tools for PFormat/Json/Native sub-agents)
- subagent_runner: add post-tool persist_transcript after tool results are appended so a mid-round crash doesn't lose tool outputs
- prompts::sync_workspace_file: preserve user-edited workspace files — only overwrite when the file doesn't exist OR its current hash matches the stored builtin hash
- context::debug_dump: mirror runner's cached-tool fallback — keep cached action catalogue on empty/error from fetch_toolkit_actions instead of blanking it
- core::agent_cli: add entry/exit debug logs around dump_all / dump_prompt calls and a trace log around each prompt file write; update module banner to note --toolkit is required when --agent is integrations_agent
|
||
|
Steven EnamakelandGitHub
|
99d61f93a7 |
feat(webui-messaging): multi-provider webview accounts, scanners, and chat runtime (#629)
* feat(accounts): implement accounts management with webview integration - Added a new Accounts page for managing user accounts, including the ability to add and remove accounts. - Introduced AddAccountModal for selecting account providers and initiating account setup. - Implemented WebviewHost to display third-party web applications (e.g., WhatsApp Web) within the app. - Enhanced routing to include a protected route for the Accounts page. - Updated the BottomTabBar to include an Accounts tab for easy navigation. - Integrated Redux for state management of accounts, messages, and logs, ensuring a seamless user experience. - Updated dependencies in Cargo.lock to version 0.52.9 for compatibility. This commit significantly enhances the application's functionality by allowing users to manage accounts directly within the app, improving overall user engagement and experience. * refactor(accounts): clean up Accounts component layout and improve readability - Removed unnecessary comments and simplified the structure of the Accounts component for better clarity. - Adjusted the rendering logic to enhance the layout of the active account section, improving user experience. - Reformatted text in the no accounts message for better readability. - Streamlined the import statements by consolidating related imports, enhancing code organization. * feat(accounts): enhance account management with new providers and routing updates - Introduced support for additional account providers: Telegram, LinkedIn, Gmail, and Slack, expanding user options for account management. - Updated routing to replace the old /conversations path with /chat, streamlining navigation and improving user experience. - Refactored the App component to include an AppShell for better layout management, ensuring the bottom tab bar visibility aligns with the selected account. - Enhanced the BottomTabBar component to reflect the new routing and account options, improving accessibility and usability. - Implemented fullscreen logic for accounts, allowing for a more immersive experience when interacting with selected accounts. - Added utility functions for managing fullscreen states and account provider icons, enhancing code organization and maintainability. This commit significantly improves the application's account management capabilities, providing users with a more flexible and engaging experience. * feat(cef): integrate Chromium Embedded Framework support and enhance webview functionality - Added support for the Chromium Embedded Framework (CEF) as an alternative runtime, allowing for improved webview capabilities. - Updated Cargo.toml to include new dependencies and features for CEF integration, ensuring compatibility with existing Tauri plugins. - Enhanced the WhatsApp recipe to include ghost-text autocomplete functionality, improving user experience during message composition. - Implemented WebSocket observation in the WhatsApp recipe to capture and forward relevant message frames, enhancing real-time interaction. - Introduced user agent spoofing for specific providers to bypass fingerprinting checks, ensuring better compatibility with services like Slack and LinkedIn. - Refactored various components to accommodate the new runtime and improve overall code organization and maintainability. This commit significantly enhances the application's webview capabilities and user interaction with messaging services, providing a more robust and flexible experience. * feat(cef): add development command for CEF and update configuration - Introduced a new development command `dev:cef` in both package.json files to streamline the development process for the Chromium Embedded Framework (CEF). - Updated Cargo.toml to include the `tauri/devtools` feature alongside `tauri/cef`, enhancing debugging capabilities. - Modified tauri.conf.json to adjust visibility settings for the application window and refined the Content Security Policy (CSP) for improved security. - Enhanced resource paths in tauri.conf.json to support recursive file inclusion for better resource management. - Updated the Rust code to bypass macOS Keychain prompts when using CEF, improving user experience during development. This commit enhances the development workflow for CEF integration, providing better tools and configurations for developers. * fix(cef): update development command for CEF to include signing identity - Modified the `dev:cef` command in package.json to include the `APPLE_SIGNING_IDENTITY` environment variable, enhancing the development process for CEF on macOS. - This change improves the build process by ensuring proper code signing during development, streamlining the workflow for developers working with CEF integration. * feat(cef): enhance development command for CEF with safe storage setup - Updated the `dev:cef` command in package.json to include a call to a new script, `setup-chromium-safe-storage.sh`, which pre-seeds the "Chromium Safe Storage" keychain entry with a permissive ACL. - Added the `setup-chromium-safe-storage.sh` script to ensure that CEF/Chromium can read the keychain entry without prompting, improving the development experience on macOS. - This change streamlines the setup process for developers working with CEF integration, ensuring a smoother workflow. * feat(whatsapp): enhance message ingestion and IndexedDB integration - Introduced a new `IngestMessage` interface to standardize message structure for WhatsApp. - Updated `IngestPayload` to include additional fields for better message handling, including `provider`, `chatId`, and `day`. - Implemented a new function `persistWhatsappChatDay` to handle the ingestion of chat messages by day, improving data organization and retrieval. - Enhanced the WhatsApp recipe to utilize IndexedDB for direct data access, eliminating the need for DOM scraping and improving performance. - Updated the Tauri configuration to enable development tools for easier debugging of webview accounts. This commit significantly improves the application's ability to manage and ingest WhatsApp messages, providing a more robust and efficient user experience. * feat(cdp): integrate IndexedDB scanner for WhatsApp via Chrome DevTools Protocol - Added a new module for scanning IndexedDB using the Chrome DevTools Protocol (CDP), enabling direct access to WhatsApp data without DOM scraping. - Implemented a scanner that communicates with the embedded CEF instance to read and decrypt messages stored in IndexedDB. - Updated the Tauri application to manage the new scanner, ensuring it operates seamlessly with existing webview accounts. - Enhanced the Cargo.toml and Cargo.lock files to include necessary dependencies such as `tokio-tungstenite` and `futures-util` for asynchronous operations. - Refactored the WhatsApp recipe to utilize the new scanning capabilities, improving performance and data handling. This commit significantly enhances the application's ability to interact with WhatsApp's IndexedDB, providing a more efficient and robust user experience. * feat(cdp): enhance message diagnostics in IndexedDB scanner - Updated the ScanSnapshot struct to include new fields for message diagnostics: `messageKeyUnion`, `messageTypeBreakdown`, and `sampleByType`, providing a comprehensive overview of message structures and types. - Modified the scanner logic to capture and log detailed information about message types and their shapes, improving debugging capabilities. - Refactored the JavaScript scanner to aggregate message key signatures and counts, enhancing the analysis of message records. This commit significantly improves the application's ability to analyze and log message data from WhatsApp's IndexedDB, facilitating better debugging and data handling. * feat(cdp): implement fast DOM scraping and crypto key extraction for WhatsApp - Introduced a new fast-tick DOM scraping mechanism to extract rendered WhatsApp message bodies, enabling near real-time message updates without relying on IndexedDB. - Added scripts for capturing and logging CryptoKey operations within WhatsApp's workers, allowing for better analysis of key derivations and decryptions. - Enhanced the CDP scanner to interleave fast DOM scans with full IndexedDB scans, optimizing data retrieval and reducing UI spamming during idle periods. - Updated the ScanSnapshot struct to include new fields for DOM-scraped messages and crypto operation statistics, improving the overall diagnostic capabilities of the application. This commit significantly enhances the application's ability to interact with WhatsApp's messaging system, providing a more efficient and responsive user experience. * feat(whatsapp): replace cdp_indexeddb with whatsapp_scanner for enhanced message handling - Replaced the `cdp_indexeddb` module with `whatsapp_scanner` to streamline the scanning process for WhatsApp messages. - Updated the application to manage the new `ScannerRegistry` for WhatsApp, improving the integration with the Chrome DevTools Protocol. - Introduced new scripts for fast DOM scraping and full IndexedDB scanning, optimizing data retrieval and enhancing real-time message updates. - Added a new `dom_scan.js` for efficient extraction of rendered message bodies directly from the DOM, reducing reliance on IndexedDB. - Enhanced the `ScanSnapshot` struct to accommodate new fields for DOM-scraped messages, improving diagnostic capabilities. This commit significantly improves the application's ability to interact with WhatsApp, providing a more efficient and responsive user experience. * refactor(whatsapp): replace JavaScript DOM scanning with Rust-based DOM snapshot - Removed the `dom_scan.js` script and replaced it with a new Rust module `dom_snapshot.rs` that captures DOM snapshots directly via the Chrome DevTools Protocol, enhancing performance and reliability. - Introduced a new `idb.rs` module for scanning WhatsApp's IndexedDB, streamlining data retrieval and improving integration with the Rust backend. - Updated the `ScanSnapshot` struct to accommodate changes in data handling, ensuring compatibility with the new scanning methods. - Enhanced overall message handling capabilities, providing a more efficient and responsive user experience. This commit significantly improves the application's ability to interact with WhatsApp, leveraging Rust for better performance and reducing reliance on JavaScript for DOM operations. * feat(docs): add webview integration playbook for third-party messaging - Introduced a comprehensive playbook detailing the process for integrating third-party webviews (e.g., Instagram, Messenger) into the application. - Documented architecture, workflow, and best practices for building and debugging new integrations, leveraging Rust and Chrome DevTools Protocol. - Included step-by-step instructions for setting up scanners, monitoring logs, and optimizing message handling, ensuring a streamlined development experience for future integrations. This addition enhances the documentation, providing developers with a clear guide to implement and maintain webview integrations effectively. * docs(webview): improve table formatting for clarity - Enhanced the formatting of tables in the webview integration playbook to improve readability and consistency. - Adjusted column headers and alignment for better presentation of job intervals and costs, ensuring clearer communication of scanning processes and common pitfalls. This update aims to provide a more user-friendly documentation experience for developers integrating third-party webviews. * feat(slack): integrate Slack scanner for message extraction and management - Updated the OpenHuman package version to 0.52.15 in both Cargo.lock files. - Introduced a new Slack scanner module to extract messages, users, and channels from Slack's IndexedDB using the Chrome DevTools Protocol. - Added functionality to manage Slack accounts within the application, allowing for automatic opening of Slack webviews based on environment variables. - Enhanced the existing webview account management to support Slack integration, ensuring seamless interaction with the Slack API. This commit significantly improves the application's ability to interact with Slack, providing a robust framework for message handling and account management. * fix(slack): one-doc-per-channel + omit CDP indexName param CEF 146's IndexedDB.requestData rejects `indexName: ""` with "Could not get index"; the CDP spec says empty string means the primary-key index but this backend only accepts the field unset. Omit it entirely so the Slack Redux-persist dump actually comes back. Also switch memory grouping from (channel, day) → channel. Each Slack channel is now one long-running memory doc keyed by channel name (e.g. `general`, `team-product`, `elvin516`), falling back to channel id for non-slug names. Every transcript line carries its own `YYYY-MM-DD HH:MM` stamp and the header records the full date range. `infer_team_id` updated to Slack's real DB naming pattern `objectStore-<TEAM>-<USER>` (not `ReduxPersistIDB:` as initially assumed). * fix(onboarding): update navigation and test descriptions in OnboardingOverlay tests - Changed the navigation path from '/conversations' to '/chat' in the OnboardingOverlay tests to reflect the updated routing logic. - Updated test descriptions for clarity, ensuring they accurately describe the functionality being tested. These changes enhance the accuracy and readability of the onboarding tests, aligning them with the current application flow. * fix(conversations): add return statement to Conversations component - Introduced a return statement in the Conversations component to ensure proper rendering of the sidebar or page variant. - This change enhances the component's functionality by ensuring it returns the expected JSX structure. These modifications improve the overall structure and behavior of the Conversations component. * feat(accounts): add Discord integration and enhance AddAccountModal - Introduced Discord as a new account provider, including its icon and service details. - Updated the AddAccountModal to filter out already connected providers, improving user experience. - Enhanced the UI to display a message when all providers are connected, ensuring clarity for users. - Implemented context menu functionality for account management, allowing users to log out directly from the accounts list. These changes expand the application's capabilities by integrating Discord and refining account management features. * feat(discord): integrate Discord scanner for HTTP and WebSocket monitoring - Added a new `discord_scanner` module to capture Discord API calls and WebSocket frames using the Chrome DevTools Protocol (CDP). - Updated the `lib.rs` to manage the new Discord scanner alongside existing WhatsApp and Slack scanners. - Enhanced the `webview_accounts` module to support Discord account management, including scanner registration and cleanup. These changes expand the application's capabilities by enabling real-time monitoring of Discord interactions, enhancing user experience and functionality. * feat(google-meet): integrate Google Meet as a new account provider - Added Google Meet as a supported account provider, including its icon and service details. - Updated the account management logic to handle Google Meet interactions, including recipe integration for call monitoring and notifications. - Enhanced the UI to accommodate the new provider, ensuring a seamless user experience when managing accounts. These changes expand the application's capabilities by integrating Google Meet, allowing users to join calls and receive notifications directly within the app. * refactor(runtime): remove notification handling and composer autocomplete - Eliminated the notification interception logic and associated functions, streamlining the runtime code. - Removed composer autocomplete features, transferring responsibility for ghost-text overlays to the UI host. - Updated comments to reflect the changes and clarify the remaining functionality. These modifications simplify the runtime script, focusing on core features while delegating UI responsibilities. * feat(google-meet): enhance Google Meet integration with lifecycle event handling - Implemented lifecycle event handling for Google Meet, including events for call start, captions, and call end. - Introduced in-memory storage for caption snapshots during meetings, allowing for the generation of markdown transcripts upon call completion. - Added interfaces for payload structures related to Google Meet events, improving type safety and clarity in the codebase. - Updated the webview account service to manage active meetings and flush transcripts to memory, ensuring a seamless user experience. These changes significantly enhance the Google Meet integration, enabling real-time caption handling and transcript generation, thereby improving the overall functionality of the application. * feat(cef): integrate CEF-based notification handling and update dependencies - Added support for native OS notifications through the `tauri-runtime-cef` crate, enabling interception of browser notifications in embedded webviews. - Introduced a new submodule for `tauri-cef` to manage CEF dependencies and facilitate notification handling. - Updated the `.gitignore` to exclude CEF-related build artifacts and lock files. - Removed the deprecated `notification_scanner` module, streamlining the codebase and focusing on the new CEF integration. - Enhanced the `webview_accounts` module to register and manage CEF browser notifications, improving user experience with real-time alerts. These changes significantly enhance the application's notification capabilities, leveraging CEF for a more integrated and responsive user experience. * feat(cef): update CEF integration and dependency management - Added `cef` and `tauri-runtime-cef` as dependencies to enhance CEF support. - Updated `Cargo.toml` to reference `tauri-runtime-cef` from a local path, ensuring proper integration with the vendored CEF submodule. - Removed direct Git references for Tauri packages, streamlining dependency management by using local paths. These changes improve the application's CEF capabilities and simplify the dependency structure, facilitating better integration and maintenance. * feat(browserscan): add BrowserScan as a new account provider with associated resources - Introduced BrowserScan as a development-only account provider, including its icon and service details. - Updated the account provider types and management logic to accommodate BrowserScan, enhancing the application's capabilities. - Added a new recipe and manifest for BrowserScan, ensuring it integrates seamlessly into the existing webview account lifecycle. - Enhanced the UI to display BrowserScan, providing users with a bot-detection sandbox for testing purposes. These changes expand the application's functionality by integrating BrowserScan, allowing for improved testing and development workflows. * feat(google-meet): enhance Google Meet integration with new transcript handling and session recovery - Introduced a new service to handle Google Meet transcripts, enabling structured note extraction and proactive follow-up actions. - Implemented session recovery logic to manage in-progress meetings when navigating away from the call. - Updated the webview account service to log call events and captions, improving monitoring and debugging capabilities. - Enhanced the Google Meet recipe to persist meeting state across navigations, ensuring seamless user experience. These changes significantly improve the Google Meet integration, allowing for better management of meeting transcripts and user interactions. * refactor(App): reorganize imports and clean up code structure - Removed unused imports from App.tsx to streamline the code. - Adjusted the import order for better readability and consistency. - Enhanced the BottomTabBar component by simplifying the button rendering logic. - Cleaned up the AddAccountModal component by consolidating prop destructuring. - Improved formatting in various components for better code clarity. These changes enhance code maintainability and readability across the application. * update tauri * feat(google-meet): improve caption handling and speaker identification logic - Enhanced the logic for extracting speaker names from Google Meet rows, adding checks to filter out icon ligatures and irrelevant text. - Updated the caption processing to better identify and score caption regions, ensuring more accurate transcript generation. - Introduced new utility functions to differentiate between real captions and icon names, improving the overall reliability of the captioning feature. These changes significantly enhance the accuracy and usability of the Google Meet integration, providing users with clearer and more relevant caption data. * chore(dependencies): update Cargo.lock with new package versions and remove obsolete entries - Bump OpenHuman version to 0.52.20. - Add new dependencies: cef, futures-util, tauri-plugin-notification, tauri-runtime-cef, tokio-tungstenite, bzip2, clap, console, cookie_store, data-encoding, dioxus-debug-cell, document-features, download-cef, encode_unicode, filetime. - Remove obsolete packages: alloc-no-stdlib, alloc-stdlib, brotli, brotli-decompressor, gdkx11, gdkx11-sys. - Update existing dependencies to their latest versions where applicable. * chore(dependencies): update submodule URLs and pin plugin versions - Changed the tauri-cef submodule URL from SSH to HTTPS for consistency. - Updated Cargo.toml to pin tauri plugins to a specific commit for reproducibility. - Adjusted Cargo.lock to reflect the new plugin source format with pinned revisions. - Modified the builder configuration in lib.rs to conditionally enable devtools in debug builds only, enhancing security in release builds. - Updated webview_accounts to restrict devtools access based on build type, ensuring better control over webview inspection. * feat(ui): enhance BottomTabBar and AddAccountModal interactions - Added focus and blur event handlers to BottomTabBar for improved visibility management. - Implemented keyboard accessibility in AddAccountModal by closing the modal on Escape key press and focusing the close button when opened. - Updated Content Security Policy in tauri.conf.json for enhanced security. - Improved Gmail recipe to use stable message IDs for better message tracking. - Introduced account ID sanitization in webview_accounts to prevent path traversal vulnerabilities. * fix(AddAccountModal): add missing import for improved functionality * chore(dependencies): bump OpenHuman version to 0.52.20 in Cargo.lock |
||
|
|
8d4934e634 |
feat(agent): progressive-disclosure handoff + token-based summarizer threshold (#574) (#586)
* feat(agent): summarizer sub-agent compresses oversized tool results Issue #574. Before this change, tool results larger than a few KB landed verbatim in orchestrator history, burning context budget on raw JSON/HTML/file dumps. The only guardrail was tool_result_budget_bytes which hard-truncated mid-payload, dropping everything past the cut. This PR introduces a dedicated `summarizer` sub-agent (model.hint = "summarization") that the runtime automatically dispatches whenever a tool result exceeds summarizer_payload_threshold_bytes (default 100 KB). The summarizer compresses the payload per an extraction contract that preserves identifiers and key facts, and the compressed summary replaces the raw payload before it enters agent history. Payloads above summarizer_max_payload_bytes (default 5 MB) skip summarization and fall through to the existing truncation path — paying for an LLM call on a 5 MB blob is counterproductive. Scoped to the orchestrator session only. Welcome, skills_agent, researcher, planner, and every other typed sub-agent get None and their tool results are untouched. Gated in build_session_agent_inner by checking agent_id == "orchestrator". Instrumented at both tool-loop paths: - run_tool_call_loop in tool_loop.rs (event-bus/channels path) - Agent::execute_tool_call in session/turn.rs (web channel path via run_single) Both paths call into the same `PayloadSummarizer::maybe_summarize` trait so the threshold check, circuit breaker (3 consecutive failures disables for the session), and sub-agent dispatch policy live in exactly one place (agent/harness/payload_summarizer.rs). The summarizer sub-agent is runtime-dispatched only — it is NOT exposed as a delegation tool to the orchestrator's LLM. It is listed in the orchestrator's `subagents = [...]` for explicit registration, but `collect_orchestrator_tools` filters out the `summarizer` id and never synthesises a `delegate_summarizer` tool. Files: * NEW: src/openhuman/agent/agents/summarizer/{agent.toml,prompt.md} * NEW: src/openhuman/agent/harness/payload_summarizer.rs (trait + SubagentPayloadSummarizer impl + circuit breaker + tests) * src/openhuman/agent/agents/mod.rs — register summarizer in BUILTINS * src/openhuman/agent/agents/orchestrator/agent.toml — list summarizer in subagents (runtime-only, no LLM delegation tool) * src/openhuman/agent/harness/mod.rs — declare module * src/openhuman/agent/harness/builtin_definitions.rs — expect summarizer in `expected_builtin_ids_are_present` * src/openhuman/agent/harness/tool_loop.rs — thread Option<&dyn PayloadSummarizer> into run_tool_call_loop + agent_turn, intercept at the tool-execution success site, plus a new integration test using a MockSummarizer * src/openhuman/agent/harness/tests.rs — thread None through the existing run_tool_call_loop callsites * src/openhuman/agent/harness/session/turn.rs — same interception in Agent::execute_tool_call * src/openhuman/agent/harness/session/types.rs — payload_summarizer field on Agent and AgentBuilder * src/openhuman/agent/harness/session/builder.rs — .payload_summarizer() setter + orchestrator-only construction in build_session_agent_inner * src/openhuman/agent/bus.rs — pass None for the bus path * src/openhuman/config/schema/context.rs — summarizer_payload_threshold_bytes + summarizer_max_payload_bytes on ContextConfig * src/openhuman/tools/orchestrator_tools.rs — filter out the summarizer id from delegation-tool synthesis Tests: unit tests in payload_summarizer pin the pass-through rules (below threshold, above max cap, breaker tripped) and the prompt construction. Integration test in tool_loop::tests uses a MockSummarizer to verify the interception wires through end-to-end. Closes tinyhumansai/openhuman#574. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(tools): csv_export tool + skills_agent oversized output handling When a Composio tool returns a large payload inside skills_agent, the agent now has two prompt-driven paths: Path A — user wants an answer derived from the data: skills_agent extracts the answer in its next iteration and returns a targeted response (no file I/O needed). Path B — user wants the actual raw dataset: skills_agent calls csv_export (tabular data) or file_write (non-tabular) to persist the output to workspace/exports/, then returns a summary + file path. Changes: * NEW: src/openhuman/tools/impl/filesystem/csv_export.rs — CsvExportTool: parses JSON array, formats as CSV, writes to workspace/exports/{filename}. Handles missing keys (empty cells), nested values (JSON-serialised), and optional column ordering. Sandboxed via SecurityPolicy. * src/openhuman/tools/impl/filesystem/mod.rs — wire module * src/openhuman/tools/ops.rs — register CsvExportTool * src/openhuman/agent/harness/definition.rs — new extra_tools field on AgentDefinition, allowing named system tools to bypass category_filter * src/openhuman/agent/harness/subagent_runner.rs — inject extra_tools into allowed_indices after category filtering * src/openhuman/agent/agents/skills_agent/agent.toml — add file_write + csv_export via extra_tools * src/openhuman/agent/agents/skills_agent/prompt.md — new "Handling Oversized Tool Results" section with Path A (extract answer) vs Path B (export file) decision tree, intent-detection heuristics, and examples Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(agent): progressive-disclosure handoff for skills_agent + token-based summarizer threshold (#574) Two layered fixes for #574 plus supporting changes. ## Summarizer thresholds in tokens, not bytes Renamed `ContextConfig::summarizer_payload_threshold_bytes` -> `summarizer_payload_threshold_tokens` (default 500 000, was 100 KB in bytes) and `summarizer_max_payload_bytes` -> `summarizer_max_payload_tokens` (default 2 000 000). Token count is estimated at ~4 chars/token via a local helper that mirrors `tree_summarizer::types::estimate_tokens`. Serde aliases on both fields keep existing config.toml files parsing. `payload_summarizer.rs` now compares token estimates for the threshold and cap, with both `tokens=` and `bytes=` surfaced in log events. ## Progressive-disclosure handoff for skills_agent `skills_agent` typed subagents (when spawned with a Composio toolkit) now receive a per-spawn `ResultHandoffCache` and a new built-in `extract_from_result` tool. When a per-action tool returns more than `HANDOFF_OVERSIZE_THRESHOLD_TOKENS` (20 000 tokens), the raw payload is stashed in the cache and a compact placeholder replaces it in history: [oversized tool output: 187432 bytes - stashed as result_id="res_1"] Preview (first 1500 chars): ... If the preview does not answer your task, call: extract_from_result(result_id="res_1", query="<specific question>") The subagent can answer from the preview, call `extract_from_result` with a targeted query, or re-call the original tool with tighter filters. `extract_from_result` dispatches the `summarizer` subagent against the cached payload with the query as the extraction contract. For payloads over `SUMMARIZER_CHUNK_CHAR_BUDGET` (60 000 chars) the extractor runs a chunked map-reduce: - split at blank-line / newline boundaries - map each chunk to a per-chunk partial via bounded concurrency (`buffer_unordered(3)`) to avoid storming the provider gateway - reduce partials in one more summarizer turn, or fall back to the concatenation if the reduce stage fails Falls back gracefully when the summarizer definition is missing from the registry (placeholder + preview still work; just no extractor). ## Text-mode dispatcher for skills_agent + tighter tool filter Provider-side grammar decoders (Fireworks etc.) cap tool-schema rules at 65 535 (uint16_t). Large Composio toolkits - Gmail, Notion, GitHub, Salesforce, HubSpot, Google Workspace - ship per-action JSON schemas dense enough that even a handful of them exceed the cap. - `subagent_runner::run_inner_loop` now detects `skills_agent` and omits `tools: [...]` from the API request, instead appending an `<tool_call>{...}</tool_call>` XML protocol block (with compact per-tool parameter summaries) into the system prompt. Tool calls are parsed out of the model's free-form response via the shared `parse_tool_calls` helper. Mirrors XmlToolDispatcher behaviour. - `top_k_for_toolkit()` picks 12 for HEAVY_SCHEMA_TOOLKITS (the six listed above plus googledocs, googlesheets, microsoftteams) vs the default 25 for lighter toolkits. ## Skills_agent agent.toml / prompt changes - Dropped `csv_export` from `skills_agent.extra_tools` (kept `file_write`) - the export tool was triggering superfluous file writes after extraction had already answered the query. - `builtin_definitions::skills_agent_has_extra_tools_for_export` test inverted to assert `csv_export` is NOT present. - `prompt.md` re-aligned: Path B now references only `file_write`. ## Wiring - `session/builder.rs`: threshold rename wiring (`_tokens`). - `subagent_runner.rs::run_inner_loop`: new `handoff_cache` parameter (threaded through both call sites; fork-mode passes `None`). ## Tests All green: - `payload_summarizer::tests`: 6/6 (thresholds in tokens) - `tool_loop::tests`: 10/10 (MockSummarizer integration) - `subagent_runner::tests`: 19/19 - `builtin_definitions::tests`: 5/5 (csv_export removal) - `csv_export::tests`: 7/7 (implementation kept, wiring removed) Closes #574. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * perf(agent): drop reduce LLM call and pre-clean tool outputs in chunked extraction Two optimisations to the oversize-handoff / extract_from_result pipeline: 1. Concat chunk summaries in order; drop reduce stage. The reduce-stage summarizer call was the slowest single turn of the pipeline and a single point of failure when the upstream provider hung — a hung reduce burned minutes before timing out. Map summaries are now sorted by original chunk index and concatenated with a plain-separator join. For listing/extraction queries this is equivalent; for top-N / global-ordering queries the caller can post-process. Observed: 8m24s → ~70s on a Notion SEARCH run. 2. Generic cleaner before the oversize check. Strips <script>/<style>/<svg> blocks, HTML comments, data:...;base64,... inline URIs, remaining HTML tags, and collapses whitespace. Applied on every tool output so results that are mostly markup drop below the 20k-token threshold and skip the extract pipeline entirely; the ones that remain oversized get chunked on clean bytes. Debug log emits before/after bytes + saved_pct per call. No changes to the extract_from_result tool surface or placeholder format. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(chat): rearm 2-min silence timer on inference progress events The client-side safety timer in Conversations.tsx was a flat 120s deadline from the send, not a silence window. Long agent turns (chained tool calls, chunked extraction fan-outs, subagent spawns) would trip the timeout even though the backend was actively making progress — showing "No response from the assistant after 2 minutes" while the server was still working. The effect now watches inferenceStatusByThread and rearms the timer on every status change for the sending thread. Any tool_call / tool_result / iteration_start / subagent_{spawned,done} event resets the 120s window. When status is cleared (chat_done / chat_error), the timer is dropped. A truly silent server still fails after 120s as before. Tracks the sending thread id in a ref (sendingThreadIdRef) so switching threads mid-turn doesn't move the timer's reference point. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
|
da5fe9260b |
fix(local_ai): hard-override to disabled until explicit opt-in (#573) (#637)
* refactor(local_ai): default to opt-in on all devices (#573) Local AI now defaults to disabled whenever the user has not explicitly picked a tier, regardless of device RAM. The onboarding flow and Settings panel remain the only ways to turn it on. Previously the bootstrap only disabled local AI on <8 GB devices and auto-applied a recommended preset on larger hosts; this flip completes the MVP goal of cloud-first defaults with a single, opt-in local model. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * style(local_ai): apply cargo fmt to bootstrap tests (#573) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(onboarding): present cloud AI as default on sufficient-RAM path (#573) Flips the LocalAIStep sufficient-RAM screen so the primary button is "Continue with Cloud" and local AI appears as an explicit opt-in ("Use local AI instead"). This aligns onboarding with the new opt-in bootstrap: every device now starts on cloud unless the user chooses local AI. The low-RAM cloud-fallback screen is unchanged. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(onboarding): cover opt-in local AI semantics in LocalAIStep (#573) Updates the sufficient-RAM path tests to match the cloud-primary UI: the default "Continue with Cloud" click advances without triggering local AI bootstrap, and the secondary "Use local AI instead" opt-in still starts the recommended-preset bootstrap and propagates errors. Low-RAM cloud-fallback tests are unchanged. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(local_ai): add opt_in_confirmed marker to local AI config (#573) Bootstrap will hard-override `enabled=false` unless this marker is true, ensuring existing installs with a stale `selected_tier` from the pre-MVP default-on era fall back to cloud until the user explicitly re-opts in. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(local_ai): hard-override local AI to disabled until explicit opt-in (#573) Every bootstrap path now returns `enabled=false` unless `opt_in_confirmed` is true, regardless of device RAM or `selected_tier`. This closes the regression where upgrading users with a persisted `selected_tier` bypassed the onboarding opt-in and started local AI without consent. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * refactor(local_ai): set opt_in_confirmed on apply_preset and surface MVP tier only (#573) `apply_preset` is the single source of truth for the opt-in marker: any non-disabled tier flips it true, `disabled` clears it. The preset RPC now returns `mvp_presets()` so the Settings UI exposes only the allowlisted `ram_2_4gb` tier, matching the MVP scope already enforced on the onboarding path. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * revert(local_ai): keep full preset catalog in presets RPC (#573) Revert the `mvp_presets()` swap in `handle_local_ai_presets`. PR #588 already renders all 5 tier cards in Settings with non-MVP tiers shown as "Coming soon" / non-selectable, and that roadmap visibility is the intended UX. Returning only the MVP tier from the RPC hid the other 4 cards entirely and broke that signal. The opt-in gate still holds: `apply_preset` remains the single writer of `opt_in_confirmed`, the RPC guard continues to reject non-MVP apply_preset calls, and the bootstrap hard-override still clamps stale configs. This commit only rolls back the UI catalog surface. Fixes failing `json_rpc_local_ai_device_profile_and_presets` integration test which expects 5 presets. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
|
5082f3e741 |
feat(overlay): activate main window on orb click (#605) (#611)
* feat(tauri): add activate_main_window command for overlay (#605) Exposes the existing show_main_window helper as a Tauri command so the overlay webview can bring the main window to front. The command is whitelisted in allow-core-process.toml so the overlay window capability can invoke it. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(overlay): activate main window on orb click (#605) The overlay orb had no click behavior. Now clicking it in idle mode invokes activate_main_window, bringing the main app window to the front (mirrors the tray icon flow). Since the overlay is an NSPanel NonactivatingPanel on macOS, React's synthesized onClick does not fire. Instead we record the press position on mousedown and emulate click on mouseup when the pointer stayed within a 4px slop. Dragging is deferred to mousemove past the slop so startDragging doesn't swallow the mouseup event. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(window): propagate show_main_window errors instead of swallowing them (#605) `show_main_window` silently logged failures and returned `()`, so the `activate_main_window` Tauri command could report success on a no-op. Thread `Result<(), String>` through so JS `invoke().catch()` sees real failures, and preserve the previous log-on-error behavior at the tray/Reopen call sites where no caller consumes the result. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(overlay): stabilize orb click vs drag vs double-click (#605) Two follow-ups to the deferred-drag pattern: 1. Drop stale pressRef when the primary button is no longer held during mousemove. Window-drag / focus changes can steal the mouseup, leaving the ref populated so the next idle hover would start a spurious drag. 2. Debounce the synthetic click by 250 ms so a follow-up dblclick can cancel it — the double-click-to-reset gesture was firing activate + reset together. Clear the timer on dblclick and on unmount. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Mega MindandGitHub
|
43d44acc1a | fix: wire proactive welcome into conversations (#635) | ||
|
|
8ab2c6b540 |
fix(overlay): restore status bubble visibility during voice dictation (#604) (#614)
Reverts to CSS-transition-based visibility so the bubble wrapper
stays mounted in the DOM. The prior conditional mount (`status === 'active' && bubble`)
raced with the async Tauri window resize — the bubble component mounted
before the overlay webview had grown from 50x50 to 224x208, and
`overflow: hidden` on the webview clipped the bubble above the visible
area.
With the wrapper always mounted and toggled via `max-w-0 opacity-0` ↔
`max-w-[184px] opacity-100`, the typewriter status text ("Listening…",
"Transcribing…") reappears reliably when the hotkey is held.
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
Steven EnamakelandGitHub
|
7db71408bd |
feat(local_ai): default to cloud fallback on <8GB RAM devices (#589)
* Enhance draft message handling in streaming edits
- Introduced a new `draft_sent` field in the `StreamingState` struct to track when a draft message has been posted, decoupling the existence of a draft from the ability to edit it.
- Updated the `flush_streaming_edit` function to set `draft_sent` upon posting a draft, ensuring that duplicate messages are not sent if the backend fails to return an ID.
- Modified the `finalize_channel_reply` function to prevent sending a fresh message if a draft was already posted without an ID, improving user experience by avoiding duplicate bubbles.
These changes enhance the robustness of message handling during streaming edits, ensuring a smoother interaction for users.
* feat(onboarding): enhance LocalAIStep for low-RAM device handling
- Added logic to determine if the device is below the RAM threshold, defaulting to a cloud AI fallback if necessary.
- Introduced a new UI for low-RAM devices, informing users about the cloud mode and providing options to continue with cloud AI or force-enable local AI.
- Updated the `ensureRecommendedLocalAiPresetIfNeeded` function to return a `recommend_disabled` flag based on device RAM.
- Enhanced tests to cover new cloud fallback UI and local AI consent handling.
These changes improve the onboarding experience by providing clearer options based on device capabilities.
* style: apply prettier formatting to LocalAIStep
* feat(local_ai): unlock model selection + add Disabled/cloud fallback option
- Remove MVP ceiling that clamped selection to the 2-4 GB tier, in bootstrap
and in the apply_preset RPC — users can now pick any tier.
- Add special "disabled" tier string to apply_preset that toggles
config.local_ai.enabled = false so the app uses the cloud summarizer.
- Presets RPC now returns local_ai_enabled so the UI can render the
currently-active state correctly.
- Rewrite DeviceCapabilitySection: tiers are now clickable buttons that
call apply_preset; adds a "Disabled — Cloud fallback" card at the top
marked "Recommended" on low-RAM devices and "Active" when enabled=false.
- Remove the MVP message copy from the model tier panel.
- Remove unread-count badge from the bottom tab bar.
* refactor(onboarding): streamline LocalAIStep and update local AI preset handling
- Removed the `ensureRecommendedLocalAiPresetIfNeeded` function call from `LocalAIStep`, replacing it with `openhumanLocalAiPresets` to directly fetch preset information.
- Updated the logic in `ensureRecommendedLocalAiPresetIfNeeded` to ensure the recommended tier is applied correctly based on user consent, improving clarity in the local AI setup process.
- Enhanced tests to mock the new `openhumanLocalAiPresets` function, ensuring coverage for low-RAM device scenarios and local AI consent handling.
- Updated documentation in the `schemas.rs` file to reflect changes in the data structure returned by the presets RPC.
These changes improve the onboarding experience by providing a more direct and efficient method for managing local AI presets based on device capabilities.
* test(LocalAIStep): improve mock implementation for local AI presets
- Refactored the mock for `openhumanLocalAiPresets` to enhance readability and maintainability.
- Ensured the mock returns consistent device information and preset details for testing scenarios.
- This change supports better test coverage and clarity in the LocalAIStep component's behavior during onboarding.
* fix(local_ai): preserve hadSelectedTier semantics in bootstrap return
hadSelectedTier / selectedTier represent the incoming state ("was a tier
already selected before this call"), not the post-apply state. Setting
both to the just-applied tier broke the existing localAiBootstrap
contract and the unit test that encodes it.
The Rust-side persistence fix is independent: removing the
recommend_disabled short-circuit ensures openhumanLocalAiApplyPreset
actually writes the selected tier to disk on the opt-in path, so
config_with_recommended_tier_if_unselected() honors the user's choice.
|
||
|
Steven EnamakelandGitHub
|
2bb20faa55 |
feat(threads): dedicated threads controller with per-thread session scoping (#590)
* feat(conversations): implement thread management features including creation and deletion - Added functionality to create new conversation threads with unique IDs and titles based on the current date and time. - Introduced a deleteThread action to remove existing threads, updating the selected thread accordingly. - Enhanced the UI to display threads in a sidebar, allowing users to select and delete threads easily. - Updated the Conversations component to handle thread loading and selection more efficiently, ensuring a smoother user experience. Also updated the OpenHuman package version to 0.52.15 in Cargo.lock files. * refactor(conversations): rename createThreadLocal to createNewThread and streamline thread creation logic - Updated the function name from `createThreadLocal` to `createNewThread` for clarity and consistency. - Simplified the thread creation process by removing the manual ID and title generation, leveraging the new API method for automatic thread creation. - Adjusted the Conversations component to utilize the new `handleCreateNewThread` function, enhancing readability and maintainability. - Removed unused thread ID and title generation functions to clean up the codebase. This refactor improves the overall structure and clarity of the thread management functionality. * refactor(memory): remove deprecated conversation thread management functions - Eliminated unused functions related to listing, creating, updating, appending, and deleting conversation threads in memory operations. - This cleanup enhances code maintainability and reduces complexity in the memory module. - The refactor focuses on streamlining the conversation management logic, aligning with recent changes in the thread handling API. * refactor(api): update thread API method names and remove deprecated functions - Renamed thread-related API methods to align with the new naming convention, improving clarity and consistency. - Removed deprecated functions related to thread creation, streamlining the API and enhancing maintainability. - Adjusted the implementation of existing methods to reflect the updated API structure, ensuring proper functionality. * refactor(thread): simplify thread state management and remove unused properties - Streamlined the thread state by removing the lastViewedAt property and related logic, enhancing clarity in unread message counting. - Updated the BottomTabBar component to reflect the new unread count logic, which now simply returns the length of threads. - Removed the setLastViewed action from the Conversations component, further simplifying the thread management logic. - Introduced a new deleteThread action to handle thread deletion, ensuring proper state updates and selection handling. - Overall, these changes improve maintainability and reduce complexity in the thread management system. * style(threads): apply cargo fmt * refactor(tabbar): remove conversations unread badge * update(Cargo.lock): bump OpenHuman package version to 0.52.15 * test(threadApi): update RPC method names to threads namespace * refactor(conversations): update model ID for chat functionality - Changed the model ID used in the chatSend function from `agentic-v1` to `reasoning-v1`, clarifying the purpose of each model. - Added comments to explain the distinction between the reasoning model and the agentic model, enhancing code readability and maintainability. |