mirror of
https://github.com/tinyhumansai/openhuman.git
synced 2026-07-28 13:32:23 +00:00
cb455e807d
* feat(imessage): add chat.db scanner that ingests into memory_doc_ingest
Adds a macOS-only Tauri-side scanner at
`app/src-tauri/src/imessage_scanner/` that reads
`~/Library/Messages/chat.db` read-only on a 60s tick, groups messages
by `(chat_identifier, day)`, and posts one
`openhuman.memory_doc_ingest` JSON-RPC call per group — matching the
convention documented in `docs/webview-integration-playbook.md` and
used by the WhatsApp scanner.
This is the first local-native integration to follow the memory-doc
convention (webview sources like WhatsApp already do). No CEF / CDP /
DOM / IDB needed — iMessage persists everything locally in SQLite, so
a single-tick scanner is sufficient.
Changes:
- New module `imessage_scanner/{mod,chatdb}.rs` with unit tests.
- `lib.rs`: register `ScannerRegistry` and spawn on setup (macOS only).
- `Cargo.toml`: add `anyhow`, `parking_lot`, `chrono`, and
macOS-only `rusqlite` (bundled).
Feature-gated to `target_os = "macos"`; non-macOS builds get a no-op
stub. Requires Full Disk Access to read chat.db — the read_since
helper surfaces a clear error pointing at System Settings on
permission failure.
* test(imessage): add real chat.db integration tests (ignored by default)
Two tests validating against the actual ~/Library/Messages/chat.db:
- real_chat_db_opens_and_returns_messages — confirms the SQL JOIN is
compatible with macOS's actual schema, rusqlite bindings deserialize
rows into our Message struct, and Full Disk Access errors surface
cleanly when denied.
- real_chat_db_empty_past_cursor — confirms cursor semantics (rows
past i64::MAX-1 return empty).
Both gated with #[ignore] so CI remains green without FDA. Run with:
cargo test --manifest-path app/src-tauri/Cargo.toml \
--lib imessage_scanner -- --ignored
Verified locally against a 463K-message chat.db: both pass.
* fix(imessage): address CodeRabbit review — data loss, upsert correctness, client reuse, local TZ
Four fixes stacked on top of the scanner from the previous commit:
1. Data-loss bug (major): newer macOS versions store the body in
attributedBody (NSKeyedArchiver/typedstream blob) with text = NULL.
chatdb now fetches attributedBody alongside text. format_transcript
falls back to a heuristic extractor (longest printable-ASCII run,
skipping known typedstream type markers) when text is absent. This
recovers the plain-text body on macOS >= 11 without pulling in a
full typedstream decoder. Emoji / non-Latin glyphs in attributedBody
are deferred (follow-up).
2. Correctness bug (major): per-tick groups were ingested as partial
deltas but keyed on (chat, day) — each tick\'s upsert replaced the
full-day transcript with just the minute\'s messages. Now on each
tick we collect unique (chat, day) keys touched by the new rows,
call chatdb::read_chat_day() to fetch the complete day slice, and
post the full transcript. Upsert is idempotent; re-runs after a
crash reproduce the same document.
3. Cursor persistence: last_rowid is now read from / written to a file
under the Tauri app-data dir (per account) so a restart doesn\'t
replay the whole 30-day backfill.
4. Nits: reqwest::Client is shared via OnceLock (one TLS/pool init),
and day grouping uses chrono::Local so the user-facing (chat, day)
keys line up with the user\'s calendar day instead of UTC.
Tests: 6 unit tests green (added extractor + message_body + local-tz
coverage); 2 real-db integration tests (ignored by default) still pass
against the 463k-message chat.db on my machine.
* fix(imessage): gate scanner startup on feature="cef" to match module decl
Module declaration is #[cfg(feature = "cef")], so the macOS setup block
that references imessage_scanner::ScannerRegistry must carry the same
feature gate or wry builds fail to resolve the symbol.
* feat(channels): register iMessage channel with local-only auth + config persistence
iMessage already has the AppleScript send/receive bridge in channels/providers/imessage.rs
and the startup wiring in channels/runtime/startup.rs, but was missing from the public
channel registry, so the UI can't list or connect it and `connect_channel` returns
"unknown channel: imessage".
This adds:
- imessage_definition() in the channel registry with ManagedDm auth mode and an
optional allowed_contacts field (comma-separated phone numbers / emails; * for any)
- Short-circuit branch in connect_channel: iMessage has no credentials to store, so
it persists channels_config.imessage (IMessageConfig { allowed_contacts }) and
returns connected without calling store_provider_credentials (which rejects
empty credential payloads)
- Matching branch in disconnect_channel: skips remove_provider_credentials and
clears channels_config.imessage
- Three unit tests: contacts persisted, empty contacts allowed, disconnect clears
Capabilities: SendText + ReceiveText. No SendRichText (AppleScript bridge is
plaintext only).
Test plan:
- cargo test -p openhuman --lib channels::controllers → 66 passed
- Still need GUI end-to-end with FDA granted; covered in follow-up.
* fix(imessage-scanner): address CodeRabbit review (PR #725)
- chatdb: filter queries to `service = 'iMessage'` so SMS/MMS rows
in chat.db are never ingested into the iMessage channel.
- scanner: gate every tick on `channels_config.imessage` via the
existing `openhuman.config_get` JSON-RPC — no ingestion before
the user connects, and scanning stops as soon as they disconnect.
- scanner: apply the configured `allowed_contacts` allowlist before
rebuilding/ingesting each chat (`*` or empty list means allow all).
- scanner: keep the cursor pinned when a full-day read or memory
write fails, so a transient core outage no longer permanently
skips messages.
- tests: add `chat_allowed` coverage for empty list, wildcard, and
case-insensitive exact match.
---------
Co-authored-by: Jwalin Shah <jshah1331@gmail.com>
Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai>