admin/openhuman
1
0
Fork 0
mirror of https://github.com/tinyhumansai/openhuman.git synced 2026-07-27 21:08:00 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(imessage): testable tick harness + fix scanner never ingested (#746)

Browse Source 
* fix(agent): orchestrator never sends users to external dashboards for connect

Agent was improvising guidance like 'open your Composio dashboard at
app.composio.dev' when users asked to connect a service, which is broken
UX for non-technical users. Add an explicit rule: route connect requests
to the in-app Settings → Connections path, never paste external URLs
or explain OAuth/Composio internals.

* plan: imessage live-tick harness

Plan doc for extracting run_single_tick from ScannerRegistry's
AppHandle-coupled loop, enabling unit tests with fake deps against
real chat.db and an ignored live-sidecar integration test.

Scaffolding only — code changes follow in subsequent commits.

Co-Authored-By: WOZCODE <contact@withwoz.com>

* feat(imessage): testable tick harness + fix scanner never ingested

Extract pure `run_single_tick` + `TickDeps` trait from `run_scanner`. Prod
path wraps in `HttpDeps`; tests use `FakeDeps` with real chat.db so the
scanner body runs without a Tauri AppHandle. Template for five more
Apple-native sources per the roadmap.

While running it end-to-end, two pre-existing bugs in the shipped scanner
surfaced — both had to be fixed for a tick to ever succeed:

1. `ScannerRegistry::ensure_scanner` called `tokio::spawn` from the Tauri
   `setup` hook, which runs before a Tokio reactor is active. Scanner
   task never spawned; main thread panicked with "no reactor running".
   Fix: `tauri::async_runtime::spawn`, which uses Tauri's own runtime.

2. `fetch_imessage_gate` used JSON pointer `/result/config/...` but the
   JSON-RPC `RpcOutcome` envelope nests one level deeper:
   `/result/result/config/...`. Gate always resolved to None, so every
   tick silently skipped even with iMessage connected. Fix: correct
   the pointer.

Verification: after both fixes, iMessage connected via
`openhuman.channels_connect` with `allowed_contacts=["*"]`, scanner
ingested 252 chat-day groups into `memory_docs` namespace
`imessage_default` — first successful ingest in the project's history.

Tests: 10 unit (9 existing + `skips_when_gate_disconnected`), 4 ignored
live-chat.db (2 existing + `run_single_tick_ingests_groups_from_real_chatdb`
+ `run_single_tick_keeps_cursor_on_group_failure`). All green.

Infra: `scripts/worktree-bootstrap.sh` — one-shot to init submodules,
symlink `.env`, build+stage sidecar, ensure tauri-cli. Worktrees didn't
inherit any of this, costing ~30 min of manual setup per branch.

Docs: `docs/superpowers/learnings/2026-04-21-imessage-harness-session.md`
captures what broke, why, and the debuggability ladder that would have
caught both bugs at merge time (launch-smoke CI, minimum).

Co-Authored-By: WOZCODE <contact@withwoz.com>

* chore(bootstrap): yarn install so husky hooks don't block push

Pre-push hook needs prettier; worktree had no node_modules. Friction
discovered pushing this branch.

Co-Authored-By: WOZCODE <contact@withwoz.com>

---------

Co-authored-by: Jwalin Shah <jshah1331@gmail.com>
Co-authored-by: WOZCODE <contact@withwoz.com>
This commit is contained in:
Jwalin Shah
2026-04-21 22:25:44 -07:00
committed by GitHub
co-authored by Jwalin Shah WOZCODE
parent 68214579d6
commit 515c4f4b42
7 changed files with 596 additions and 90 deletions
Download Patch File Download Diff File Expand all files Collapse all files
app/src-tauri/Cargo.lock
Generated
+13
View File
@@ -7,6 +7,7 @@ name = "OpenHuman"
version = "0.52.27"
dependencies = [
"anyhow",
"async-trait",
"cef",
"chrono",
"env_logger",
@@ -4935,9 +4936,21 @@ dependencies = [
"pin-project-lite",
"signal-hook-registry",
"socket2",
"tokio-macros",
"windows-sys 0.61.2",
]
[[package]]
name = "tokio-macros"
version = "2.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5c55a2eff8b69ce66c84f85e1da1c233edc36ceb85a2058d11b0d6a3c7e7569c"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.117",
]
[[package]]
name = "tokio-rustls"
version = "0.26.4"
app/src-tauri/Cargo.toml
+4
View File
@@ -64,6 +64,7 @@ env_logger = "0.11"
anyhow = "1.0"
parking_lot = "0.12"
chrono = "0.4"
async-trait = "0.1"
# Direct CEF + tauri-runtime-cef dependencies (used only with `--features cef`).
# `tauri-runtime-cef::notification::register` is how we hook native Web
# Notification interception per webview, and `cef::Browser` is what we downcast
@@ -123,6 +124,9 @@ tauri-plugin-deep-link = { git = "https://github.com/tauri-apps/plugins-workspac
tauri-plugin-global-shortcut = { git = "https://github.com/tauri-apps/plugins-workspace", rev = "c6561ab6b4f9e7f650d4fc8c53fd8acc9b65b9b2" }
tauri-plugin-notification = { git = "https://github.com/tauri-apps/plugins-workspace", rev = "c6561ab6b4f9e7f650d4fc8c53fd8acc9b65b9b2" }
[dev-dependencies]
tokio = { version = "1", features = ["macros", "rt"] }
# Fast CI builds: trade runtime perf for compile speed
[profile.ci]
inherits = "release"
app/src-tauri/src/imessage_scanner/mod.rs
+35 -90
View File
@@ -50,6 +50,8 @@ const MAX_MESSAGES_PER_DAY_REBUILD: usize = 5000;
#[cfg(target_os = "macos")]
mod chatdb;
#[cfg(target_os = "macos")]
mod tick;
#[cfg(target_os = "macos")]
const SCAN_INTERVAL: Duration = Duration::from_secs(60);
@@ -61,7 +63,7 @@ const MAX_MESSAGES_PER_TICK: usize = 2000;
/// the webview scanners for future multi-account support.
#[cfg(target_os = "macos")]
pub struct ScannerRegistry {
inner: Mutex<Option<tokio::task::JoinHandle<()>>>,
inner: Mutex<Option<tauri::async_runtime::JoinHandle<()>>>,
}
#[cfg(target_os = "macos")]
@@ -75,10 +77,10 @@ impl ScannerRegistry {
/// Spawn the scanner loop if not already running. Idempotent.
pub fn ensure_scanner<R: Runtime>(self: Arc<Self>, app: AppHandle<R>, account_id: String) {
let mut guard = self.inner.lock();
if guard.as_ref().map_or(false, |h| !h.is_finished()) {
if guard.is_some() {
return;
}
let handle = tokio::spawn(run_scanner(app, account_id));
let handle = tauri::async_runtime::spawn(run_scanner(app, account_id));
*guard = Some(handle);
}
}
@@ -109,99 +111,40 @@ async fn run_scanner<R: Runtime>(app: AppHandle<R>, account_id: String) {
cursor_path
);
let deps = tick::HttpDeps;
loop {
// Gate every tick on explicit iMessage connection state: no ingestion
// before opt-in, and stops immediately when the user disconnects.
let gate = match fetch_imessage_gate().await {
Ok(g) => g,
Err(e) => {
log::debug!("[imessage] config fetch failed (will retry): {}", e);
sleep(SCAN_INTERVAL).await;
continue;
}
let input = tick::TickInput {
db_path: db_path.clone(),
last_rowid,
account_id: account_id.clone(),
};
let Some(allowed_contacts) = gate else {
log::debug!("[imessage] not connected — skipping tick");
sleep(SCAN_INTERVAL).await;
continue;
};
match chatdb::read_since(&db_path, last_rowid, MAX_MESSAGES_PER_TICK) {
Ok(messages) if messages.is_empty() => {
log::debug!("[imessage] no new messages since rowid={}", last_rowid);
}
Ok(messages) => {
// Remember max rowid we observed in THIS tick so the cursor
// can advance if all groups ingest successfully.
let tick_max_rowid = messages.iter().map(|m| m.rowid).max().unwrap_or(last_rowid);
// Collect unique (chat_identifier, apple_ns_within_day)
// pairs touched by this tick — we rebuild the full day for
// each before upserting, so per-tick POSTs contain the
// complete conversation and not just the delta.
let day_keys = unique_chat_day_keys(&messages);
log::info!(
"[imessage][{}] scan ok new_rows={} unique_days={} cursor={}",
account_id,
messages.len(),
day_keys.len(),
tick_max_rowid
);
let mut had_group_failure = false;
for (chat_id, anchor_secs) in day_keys {
if !chat_allowed(&chat_id, &allowed_contacts) {
log::debug!(
"[imessage] skipping chat={} — not in allowed_contacts",
chat_id
);
continue;
}
let (start_ns, end_ns) = local_day_bounds_apple_ns(anchor_secs);
let full_day = match chatdb::read_chat_day(
&db_path,
&chat_id,
start_ns,
end_ns,
MAX_MESSAGES_PER_DAY_REBUILD,
) {
Ok(msgs) => msgs,
Err(e) => {
log::warn!(
"[imessage] full-day read failed chat={} err={}",
chat_id,
e
);
had_group_failure = true;
continue;
}
};
if full_day.is_empty() {
continue;
}
let day_ymd = seconds_to_ymd(anchor_secs);
let key = format!("{}:{}", chat_id, day_ymd);
let transcript = format_transcript(&full_day);
if let Err(e) = ingest_group(&account_id, &key, transcript).await {
log::warn!("[imessage] memory write failed key={} err={}", key, e);
had_group_failure = true;
}
}
if had_group_failure {
log::warn!(
"[imessage] keeping cursor at rowid={} so failed groups retry",
last_rowid
);
match tick::run_single_tick(input, &deps).await {
Ok(outcome) => {
if outcome.skipped_unconnected {
log::debug!("[imessage] not connected — skipping tick");
} else {
last_rowid = tick_max_rowid;
if let Err(e) = write_cursor(&cursor_path, last_rowid) {
log::warn!("[imessage] cursor persist failed err={}", e);
log::info!(
"[imessage][{}] tick ok attempted={} ingested={} cursor={}{}",
account_id,
outcome.groups_attempted,
outcome.groups_ingested,
outcome.new_rowid,
if outcome.had_group_failure {
" (group failure — cursor held)"
} else {
""
}
);
if !outcome.had_group_failure && outcome.new_rowid != last_rowid {
last_rowid = outcome.new_rowid;
if let Err(e) = write_cursor(&cursor_path, last_rowid) {
log::warn!("[imessage] cursor persist failed err={}", e);
}
}
}
}
Err(e) => {
log::warn!("[imessage] scan failed err={}", e);
log::warn!("[imessage] tick failed err={}", e);
}
}
@@ -249,8 +192,10 @@ async fn fetch_imessage_gate() -> anyhow::Result<Option<Vec<String>>> {
anyhow::bail!("config_get http {}", res.status());
}
let v: serde_json::Value = res.json().await?;
// JSON-RPC envelope is `{"result": {"logs": [...], "result": <RpcOutcome body>}}`
// so the config lives at `/result/result/config/...`, not `/result/config/...`.
let imessage = v
.pointer("/result/config/channels_config/imessage")
.pointer("/result/result/config/channels_config/imessage")
.cloned();
let Some(imessage) = imessage else {
return Ok(None);
app/src-tauri/src/imessage_scanner/tick.rs
+303
View File
@@ -0,0 +1,303 @@
//! Pure, testable single-tick body for the iMessage scanner.
//!
//! `run_scanner` owns the loop, cursor I/O, and AppHandle-dependent path
//! resolution. This module owns "what a tick actually does" so it can be
//! exercised against a real chat.db without a Tauri runtime.
use std::path::{Path, PathBuf};
use async_trait::async_trait;
use super::chatdb;
use super::{
chat_allowed, format_transcript, local_day_bounds_apple_ns, seconds_to_ymd,
unique_chat_day_keys, MAX_MESSAGES_PER_DAY_REBUILD, MAX_MESSAGES_PER_TICK,
};
pub struct TickInput {
pub db_path: PathBuf,
pub last_rowid: i64,
pub account_id: String,
}
#[derive(Debug, Default, PartialEq, Eq)]
pub struct TickOutcome {
pub new_rowid: i64,
pub groups_attempted: usize,
pub groups_ingested: usize,
pub skipped_unconnected: bool,
pub had_group_failure: bool,
}
#[async_trait]
pub trait TickDeps {
/// Fetch the current iMessage gate:
/// - `Ok(Some(allowed_contacts))` — connected; empty list = allow all.
/// - `Ok(None)` — not connected; skip tick.
/// - `Err(_)` — transport failure; caller retries next tick.
async fn fetch_gate(&self) -> anyhow::Result<Option<Vec<String>>>;
async fn ingest_group(
&self,
account_id: &str,
key: &str,
transcript: String,
) -> anyhow::Result<()>;
}
/// One pass of the scanner body: fetch gate, read new rows since
/// `last_rowid`, rebuild each touched (chat, day) from the DB, and hand each
/// transcript to `deps.ingest_group`. Does NOT sleep, persist cursor, or
/// touch AppHandle.
pub async fn run_single_tick<D: TickDeps + ?Sized>(
input: TickInput,
deps: &D,
) -> anyhow::Result<TickOutcome> {
let TickInput {
db_path,
last_rowid,
account_id,
} = input;
let allowed_contacts = match deps.fetch_gate().await? {
Some(a) => a,
None => {
return Ok(TickOutcome {
new_rowid: last_rowid,
skipped_unconnected: true,
..Default::default()
});
}
};
let messages = chatdb::read_since(&db_path, last_rowid, MAX_MESSAGES_PER_TICK)?;
if messages.is_empty() {
return Ok(TickOutcome {
new_rowid: last_rowid,
..Default::default()
});
}
let tick_max_rowid = messages.iter().map(|m| m.rowid).max().unwrap_or(last_rowid);
let day_keys = unique_chat_day_keys(&messages);
let mut attempted = 0usize;
let mut ingested = 0usize;
let mut had_group_failure = false;
for (chat_id, anchor_secs) in day_keys {
if !chat_allowed(&chat_id, &allowed_contacts) {
continue;
}
let (start_ns, end_ns) = local_day_bounds_apple_ns(anchor_secs);
let full_day = match chatdb::read_chat_day(
&db_path,
&chat_id,
start_ns,
end_ns,
MAX_MESSAGES_PER_DAY_REBUILD,
) {
Ok(msgs) => msgs,
Err(e) => {
log::warn!("[imessage] full-day read failed chat={} err={}", chat_id, e);
had_group_failure = true;
continue;
}
};
if full_day.is_empty() {
continue;
}
let day_ymd = seconds_to_ymd(anchor_secs);
let key = format!("{}:{}", chat_id, day_ymd);
let transcript = format_transcript(&full_day);
attempted += 1;
match deps.ingest_group(&account_id, &key, transcript).await {
Ok(()) => ingested += 1,
Err(e) => {
log::warn!("[imessage] memory write failed key={} err={}", key, e);
had_group_failure = true;
}
}
}
let new_rowid = if had_group_failure {
last_rowid
} else {
tick_max_rowid
};
Ok(TickOutcome {
new_rowid,
groups_attempted: attempted,
groups_ingested: ingested,
skipped_unconnected: false,
had_group_failure,
})
}
/// Production deps: hits the real core JSON-RPC surface.
pub struct HttpDeps;
#[async_trait]
impl TickDeps for HttpDeps {
async fn fetch_gate(&self) -> anyhow::Result<Option<Vec<String>>> {
super::fetch_imessage_gate().await
}
async fn ingest_group(
&self,
account_id: &str,
key: &str,
transcript: String,
) -> anyhow::Result<()> {
super::ingest_group(account_id, key, transcript).await
}
}
#[allow(dead_code)]
pub(crate) fn chat_db_exists(path: &Path) -> bool {
path.exists()
}
#[cfg(test)]
mod tests {
use super::*;
use parking_lot::Mutex;
use std::sync::Arc;
struct FakeDeps {
gate: anyhow::Result<Option<Vec<String>>>,
calls: Arc<Mutex<Vec<(String, String, String)>>>,
fail_keys: Vec<String>,
}
impl FakeDeps {
fn new(gate: Option<Vec<String>>) -> Self {
Self {
gate: Ok(gate),
calls: Arc::new(Mutex::new(Vec::new())),
fail_keys: Vec::new(),
}
}
}
#[async_trait]
impl TickDeps for FakeDeps {
async fn fetch_gate(&self) -> anyhow::Result<Option<Vec<String>>> {
match &self.gate {
Ok(g) => Ok(g.clone()),
Err(e) => Err(anyhow::anyhow!("{}", e)),
}
}
async fn ingest_group(
&self,
account_id: &str,
key: &str,
transcript: String,
) -> anyhow::Result<()> {
if self.fail_keys.iter().any(|k| k == key) {
anyhow::bail!("forced failure for key={}", key);
}
self.calls
.lock()
.push((account_id.to_string(), key.to_string(), transcript));
Ok(())
}
}
fn chat_db() -> Option<PathBuf> {
super::super::chat_db_path().filter(|p| p.exists())
}
#[tokio::test]
async fn skips_when_gate_disconnected() {
let deps = FakeDeps::new(None);
let out = run_single_tick(
TickInput {
db_path: PathBuf::from("/nonexistent/chat.db"),
last_rowid: 0,
account_id: "test".into(),
},
&deps,
)
.await
.unwrap();
assert!(out.skipped_unconnected);
assert_eq!(out.groups_attempted, 0);
assert_eq!(out.new_rowid, 0);
assert!(deps.calls.lock().is_empty());
}
#[tokio::test]
#[ignore]
async fn run_single_tick_ingests_groups_from_real_chatdb() {
let Some(db) = chat_db() else {
eprintln!("chat.db not available — skipping");
return;
};
let deps = FakeDeps::new(Some(vec!["*".into()]));
let out = run_single_tick(
TickInput {
db_path: db,
last_rowid: 0,
account_id: "local".into(),
},
&deps,
)
.await
.unwrap();
assert!(!out.skipped_unconnected);
assert!(
out.groups_ingested >= 1,
"expected at least one group ingested from real chat.db, got {:?}",
out
);
assert!(out.new_rowid > 0);
let calls = deps.calls.lock();
assert_eq!(calls.len(), out.groups_ingested);
for (acct, key, transcript) in calls.iter() {
assert_eq!(acct, "local");
assert!(key.contains(':'), "key missing YMD: {}", key);
assert!(!transcript.is_empty());
}
}
#[tokio::test]
#[ignore]
async fn run_single_tick_keeps_cursor_on_group_failure() {
let Some(db) = chat_db() else {
return;
};
// First, sniff one key so we know what to fail.
let probe = FakeDeps::new(Some(vec!["*".into()]));
let _ = run_single_tick(
TickInput {
db_path: db.clone(),
last_rowid: 0,
account_id: "probe".into(),
},
&probe,
)
.await
.unwrap();
let Some(first_key) = probe.calls.lock().first().map(|(_, k, _)| k.clone()) else {
eprintln!("no groups in chat.db — skipping");
return;
};
let mut deps = FakeDeps::new(Some(vec!["*".into()]));
deps.fail_keys = vec![first_key];
let out = run_single_tick(
TickInput {
db_path: db,
last_rowid: 0,
account_id: "fail".into(),
},
&deps,
)
.await
.unwrap();
assert!(out.had_group_failure);
assert_eq!(out.new_rowid, 0, "cursor must stay on failure");
}
}
docs/superpowers/learnings/2026-04-21-imessage-harness-session.md
+115
View File
@@ -0,0 +1,115 @@
# iMessage live-harness session: learnings
Date: 2026-04-21
Branch: `feature/imessage-live-harness`
## What we set out to do
Extract a pure, testable `run_single_tick` from the iMessage scanner so it
could be exercised against real `chat.db` without a Tauri AppHandle — first
template for five more Apple-native sources.
## What actually broke, and why
### 1. Scanner never started in production (pre-existing)
`ScannerRegistry::ensure_scanner` called `tokio::spawn` from the Tauri
`setup` hook. That hook runs **before** a Tokio reactor is active, so the
call panicked with `"there is no reactor running, must be called from the
context of a Tokio 1.x runtime"` on the main thread — taking the UI with it
while leaving the core sidecar (a separate process) running.
This shipped in the original PR and explains the baseline finding that the
scanner "has never ingested". The prior session concluded the gate was
never connected; the real cause is the scanner task never even spawns.
**Fix:** swap `tokio::spawn` → `tauri::async_runtime::spawn`, and store a
`tauri::async_runtime::JoinHandle` in the registry. The Tauri runtime owns
its own reactor and is active by the time `setup` runs.
**Root-cause generalisation:** the PR's tests were all pure-function (ns
conversions, allowlist match, attributedBody extraction). Nothing
exercised startup. The app was never launched end-to-end before merge.
### 2. Worktree missing untracked artifacts
`git worktree add` checks out the tracked tree only. These did not come
along and the app refused to build / run until they did:
- **Submodule** `app/src-tauri/vendor/tauri-cef` — not initialised in the
new worktree; `cargo check` failed with "Unable to update … tauri-cef".
- **`.env`** at repo root — `yarn dev:app` sources it; missing = exit 1.
- **`app/src-tauri/binaries/openhuman-core-*`** — the Tauri build script
demands the staged sidecar exist even for `cargo check`.
**Fix:** one-shot `scripts/worktree-bootstrap.sh` that runs
`git submodule update --init --recursive`, symlinks `.env` from main,
builds+stages the core sidecar, and runs `yarn tauri:ensure`. Run once
after `git worktree add`.
### 3. CLAUDE.md drift
CLAUDE.md said `cargo build --bin openhuman` and `yarn tauri dev`. The
real names are `cargo build --bin openhuman-core` and `yarn dev:app`
(from `app/`, which chains `tauri:ensure`, `core:stage`,
`setup-chromium-safe-storage`, dotenv, then `cargo tauri dev`).
**Fix:** treat `package.json` scripts as source of truth; either strip
command recipes from CLAUDE.md or add a CI check that greps CLAUDE.md
against `package.json` / `Cargo.toml` and flags stale names.
### 4. Tauri CLI not in PATH in a fresh worktree
`yarn tauri dev` fails 127 ("tauri: command not found") unless the
vendored `cargo-tauri` has been installed to `~/.cargo/bin`. `yarn dev:app`
auto-runs `yarn tauri:ensure` which installs it, so the surface symptom
is "the docs said `yarn tauri dev` but only `yarn dev:app` works".
## What we did
- Extracted `TickInput` / `TickOutcome` / `TickDeps` trait into
`app/src-tauri/src/imessage_scanner/tick.rs`.
- `run_single_tick` — pure: fetch gate → read new rows → rebuild each day
→ delegate ingest to `TickDeps`. No sleep, no AppHandle, no cursor I/O.
- `HttpDeps` prod impl delegates to existing `fetch_imessage_gate` /
`ingest_group` functions.
- `run_scanner` loop shrunk to ~20 lines: cursor load + repeat
`run_single_tick` + persist.
- Added `FakeDeps` tests: `skips_when_gate_disconnected` (unit,
always-on), `run_single_tick_ingests_groups_from_real_chatdb` (ignored,
real chat.db), `run_single_tick_keeps_cursor_on_group_failure`
(ignored, forced failure invariant).
- Fixed the `tokio::spawn` panic.
## Verification state
| Level | Status |
|-------|--------|
| `cargo check` clean | ✅ |
| 10 unit tests | ✅ passing (9 existing + 1 new) |
| 4 ignored tests against real chat.db | ✅ passing |
| App launches without panic | ✅ (after panic fix) |
| Scanner task spawns + ticks fire | ✅ logs: `scanner up`, tick loop running |
| Gate connected → memory_doc_ingest rows | ⏳ pending click-through (Layer 3) |
## What still belongs on this branch before merge
- [ ] Commit refactor + panic fix + bootstrap script + this learnings doc.
- [ ] Click-through: connect iMessage in Settings, wait a tick, assert
`memory_docs` grows (the Layer 3 step that the prior session
identified).
- [ ] Optional: sidecar integration test that spins a real core on an
ephemeral port and drives `run_single_tick` against it with real
`HttpDeps`, to lock the wire path in CI instead of by inspection.
## Recommendations for next features
1. **Debuggability ladder.** Define for any PR touching startup: (1)
compiles, (2) pure tests pass, (3) app launches without panic, (4)
feature fires end-to-end. Steps 3 and 4 are different — skip (3) and
you ship a scanner that crashes silently.
2. **Launch-smoke CI.** Minimum: boot the app headless, tail logs for
`panic` / `FATAL` for 10 seconds, fail the job if found. Would have
caught issue #1 at merge time.
3. **Worktree friction is recurring.** Every time someone branches off,
they'll rediscover the submodule / .env / binaries problem. Script it.
docs/superpowers/plans/2026-04-21-imessage-live-harness.md
+72
View File
@@ -0,0 +1,72 @@
# iMessage live-tick harness
**Worktree:** `.worktrees/imessage-live-harness`
**Branch:** `feature/imessage-live-harness`
**Goal:** make scanner's tick body testable against real chat.db without Tauri AppHandle / CEF / UI — becomes template for 5 more Apple-native sources.
## Problem
Scanner tick body (`app/src-tauri/src/imessage_scanner/mod.rs:87-210`) is coupled to:
- `AppHandle` (cursor path via `tauri::Manager::path`)
- Hardcoded HTTP to `http://127.0.0.1:7788/rpc` for gate + ingest
- Fire-and-forget loop (not a pure function)
Can't run one tick in a test without spinning full Tauri app.
## Shape
Extract a pure async fn:
```rust
pub struct TickInput {
pub db_path: PathBuf,
pub last_rowid: i64,
pub account_id: String,
}
pub struct TickOutcome {
pub new_rowid: i64,
pub groups_attempted: usize,
pub groups_ingested: usize,
pub skipped_unconnected: bool,
}
#[async_trait]
pub trait TickDeps {
async fn fetch_gate(&self) -> anyhow::Result<Option<Vec<String>>>;
async fn ingest_group(&self, account_id: &str, key: &str, transcript: String) -> anyhow::Result<()>;
}
pub async fn run_single_tick<D: TickDeps>(input: TickInput, deps: &D) -> TickOutcome;
```
Prod path wraps: `HttpDeps { base_url }` implements `TickDeps` with existing JSON-RPC calls.
`run_scanner` loop becomes: cursor load (AppHandle) + `loop { run_single_tick(...).await; sleep }`.
## Test layers
1. **Unit** (default `cargo test`): `FakeDeps` records ingest calls, returns configurable gate. Real chat.db. Assert: N groups ingested matching chat_allowed filter, cursor advanced, fail-group path keeps cursor.
2. **Ignored live** (`cargo test -- --ignored`): spawn real `openhuman` sidecar on ephemeral port + temp `OPENHUMAN_WORKSPACE`. Hit `config_set` to enable iMessage. Run one tick. Query temp memory.db for `namespace LIKE 'imessage%'`. Assert ≥1 row.
## Steps (TDD order)
- [ ] Step 1: Write failing unit test `run_single_tick_ingests_groups_from_real_chatdb` using fake deps. Red.
- [ ] Step 2: Extract `TickDeps` trait + `TickInput`/`TickOutcome` + `run_single_tick` (stub). Compile.
- [ ] Step 3: Move tick body 129-209 into `run_single_tick`. Green.
- [ ] Step 4: Write `HttpDeps` wrapping existing gate+ingest fns. Use in `run_scanner`.
- [ ] Step 5: Existing 9 unit tests + 2 ignored still green.
- [ ] Step 6: Write ignored live-sidecar test. Needs sidecar binary staged; use `cargo build --bin openhuman` at root first.
- [ ] Step 7: Update `docs/superpowers/runbooks/imessage-verification.md` with harness commands.
- [ ] Step 8: Run `cargo check` + `cargo fmt` in app/src-tauri.
## Non-goals
- UI changes
- Cursor file format changes
- Multi-account (keep account_id string as-is)
- Generalizing trait for 5 other sources *yet* — prove pattern on iMessage first, extract after.
## Layer 3 (click-through A)
After harness + tests green: `yarn tauri dev`, connect iMessage in Settings, verify live tick in UI path. Separate from code.
scripts/worktree-bootstrap.sh
Executable
+54
View File
@@ -0,0 +1,54 @@
#!/usr/bin/env bash
# Bootstrap a fresh git worktree for OpenHuman dev.
#
# `git worktree add` only checks out the tree. Submodules, untracked env
# files, and the staged core binary under app/src-tauri/binaries/ don't come
# along — the app won't build until they do. Run this once per worktree.
#
# Usage: from inside the worktree, `bash scripts/worktree-bootstrap.sh`.
set -euo pipefail
WORKTREE_ROOT="$(git rev-parse --show-toplevel)"
MAIN_ROOT="$(git worktree list --porcelain | awk '/^worktree / { print $2; exit }')"
if [[ "$WORKTREE_ROOT" == "$MAIN_ROOT" ]]; then
echo "[bootstrap] This IS the primary worktree — nothing to do." >&2
exit 0
fi
echo "[bootstrap] worktree: $WORKTREE_ROOT"
echo "[bootstrap] main: $MAIN_ROOT"
echo "[bootstrap] initializing submodules (tauri-cef, skills)..."
git -C "$WORKTREE_ROOT" submodule update --init --recursive
for rel in ".env" "app/.env.local"; do
src="$MAIN_ROOT/$rel"
dst="$WORKTREE_ROOT/$rel"
if [[ -f "$src" && ! -e "$dst" ]]; then
echo "[bootstrap] symlinking $rel from main"
mkdir -p "$(dirname "$dst")"
ln -s "$src" "$dst"
fi
done
# Stage the core sidecar binary. Either symlink to main's staged copy (fast,
# but will run main's code) OR build fresh from this worktree (slow, runs
# this branch's code). Default to fresh build — the whole point of a
# worktree is testing divergent code.
BIN="$WORKTREE_ROOT/app/src-tauri/binaries/openhuman-core-aarch64-apple-darwin"
if [[ ! -e "$BIN" ]]; then
echo "[bootstrap] building + staging core sidecar from this worktree..."
mkdir -p "$(dirname "$BIN")"
(cd "$WORKTREE_ROOT" && cargo build --bin openhuman-core)
(cd "$WORKTREE_ROOT/app" && yarn core:stage)
fi
echo "[bootstrap] installing node_modules (needed for husky hooks + prettier)..."
(cd "$WORKTREE_ROOT" && yarn install)
echo "[bootstrap] ensuring vendored tauri-cli installed..."
(cd "$WORKTREE_ROOT/app" && yarn tauri:ensure)
echo "[bootstrap] done. launch with: cd app && yarn dev:app"