mirror of
https://github.com/tinyhumansai/openhuman.git
synced 2026-07-30 06:32:24 +00:00
b54c213aec
* feat(imessage): add chat.db scanner that ingests into memory_doc_ingest
Adds a macOS-only Tauri-side scanner at
`app/src-tauri/src/imessage_scanner/` that reads
`~/Library/Messages/chat.db` read-only on a 60s tick, groups messages
by `(chat_identifier, day)`, and posts one
`openhuman.memory_doc_ingest` JSON-RPC call per group — matching the
convention documented in `docs/webview-integration-playbook.md` and
used by the WhatsApp scanner.
This is the first local-native integration to follow the memory-doc
convention (webview sources like WhatsApp already do). No CEF / CDP /
DOM / IDB needed — iMessage persists everything locally in SQLite, so
a single-tick scanner is sufficient.
Changes:
- New module `imessage_scanner/{mod,chatdb}.rs` with unit tests.
- `lib.rs`: register `ScannerRegistry` and spawn on setup (macOS only).
- `Cargo.toml`: add `anyhow`, `parking_lot`, `chrono`, and
macOS-only `rusqlite` (bundled).
Feature-gated to `target_os = "macos"`; non-macOS builds get a no-op
stub. Requires Full Disk Access to read chat.db — the read_since
helper surfaces a clear error pointing at System Settings on
permission failure.
* test(imessage): add real chat.db integration tests (ignored by default)
Two tests validating against the actual ~/Library/Messages/chat.db:
- real_chat_db_opens_and_returns_messages — confirms the SQL JOIN is
compatible with macOS's actual schema, rusqlite bindings deserialize
rows into our Message struct, and Full Disk Access errors surface
cleanly when denied.
- real_chat_db_empty_past_cursor — confirms cursor semantics (rows
past i64::MAX-1 return empty).
Both gated with #[ignore] so CI remains green without FDA. Run with:
cargo test --manifest-path app/src-tauri/Cargo.toml \
--lib imessage_scanner -- --ignored
Verified locally against a 463K-message chat.db: both pass.
* fix(imessage): address CodeRabbit review — data loss, upsert correctness, client reuse, local TZ
Four fixes stacked on top of the scanner from the previous commit:
1. Data-loss bug (major): newer macOS versions store the body in
attributedBody (NSKeyedArchiver/typedstream blob) with text = NULL.
chatdb now fetches attributedBody alongside text. format_transcript
falls back to a heuristic extractor (longest printable-ASCII run,
skipping known typedstream type markers) when text is absent. This
recovers the plain-text body on macOS >= 11 without pulling in a
full typedstream decoder. Emoji / non-Latin glyphs in attributedBody
are deferred (follow-up).
2. Correctness bug (major): per-tick groups were ingested as partial
deltas but keyed on (chat, day) — each tick\'s upsert replaced the
full-day transcript with just the minute\'s messages. Now on each
tick we collect unique (chat, day) keys touched by the new rows,
call chatdb::read_chat_day() to fetch the complete day slice, and
post the full transcript. Upsert is idempotent; re-runs after a
crash reproduce the same document.
3. Cursor persistence: last_rowid is now read from / written to a file
under the Tauri app-data dir (per account) so a restart doesn\'t
replay the whole 30-day backfill.
4. Nits: reqwest::Client is shared via OnceLock (one TLS/pool init),
and day grouping uses chrono::Local so the user-facing (chat, day)
keys line up with the user\'s calendar day instead of UTC.
Tests: 6 unit tests green (added extractor + message_body + local-tz
coverage); 2 real-db integration tests (ignored by default) still pass
against the 463k-message chat.db on my machine.
* fix(imessage): gate scanner startup on feature="cef" to match module decl
Module declaration is #[cfg(feature = "cef")], so the macOS setup block
that references imessage_scanner::ScannerRegistry must carry the same
feature gate or wry builds fail to resolve the symbol.
---------
Co-authored-by: Jwalin Shah <jshah1331@gmail.com>
Tauri + React + Typescript
This template should help get you started developing with Tauri, React and Typescript in Vite.