openhuman/gitbooks/developing/e2e-testing.md

description, icon

description	icon
End-to-end testing with WDIO + tauri-driver / Appium. CI and local setup.	vials

E2E Testing Guide

Overview

Desktop E2E tests use WebDriverIO (WDIO) to drive the Tauri app via two automation backends:

Platform	Driver	Port	App format	Selectors
Linux / CEF status	tauri-driver	4444	Debug binary	CSS / DOM
macOS / Appium	Appium Mac2	4723	.app bundle	XPath / accessibility

OpenHuman's desktop app currently uses the CEF runtime (tauri-runtime-cef). Linux tauri-driver talks to WebKitWebDriver / webkit2gtk and cannot drive a CEF-backed WebView, so Linux CEF E2E is disabled in CI until a CEF-compatible driver or replacement harness exists. The supported path today is macOS/Appium for local runs, with manual macOS/Appium workflow runs when that workflow is enabled.

---

Quick start

Linux / CEF status

# Install tauri-driver (one-time)
cargo install tauri-driver

# Build the E2E app
pnpm --filter openhuman-app test:e2e:build

# Run all flows
pnpm --filter openhuman-app test:e2e:all:flows

# Run a single spec
bash app/scripts/e2e-run-spec.sh test/e2e/specs/smoke.spec.ts smoke

On headless Linux, the harness runs under Xvfb for a virtual display. This path is currently useful only for non-CEF / WebKit-compatible debugging; the default CEF app cannot be automated by WebKitWebDriver.

macOS / Appium

# Install Appium + Mac2 driver (one-time, needs Node 24+)
npm install -g appium
appium driver install mac2

# Build the .app bundle
pnpm --filter openhuman-app test:e2e:build

# Run all flows
pnpm --filter openhuman-app test:e2e:all:flows

Docker on macOS (Linux harness locally)

Run the same Linux-based harness from macOS using Docker. The same CEF limitation applies: this is not a supported path for the default CEF runtime until a CEF-compatible driver exists.

# Build + run all E2E flows
docker compose -f e2e/docker-compose.yml run --rm e2e

# Build the app first (if needed)
docker compose -f e2e/docker-compose.yml run --rm e2e \
  pnpm --filter openhuman-app test:e2e:build

# Run a single spec
docker compose -f e2e/docker-compose.yml run --rm e2e \
  bash app/scripts/e2e-run-spec.sh test/e2e/specs/smoke.spec.ts smoke

Requires Docker Desktop or Colima. The repo is bind-mounted so builds persist between runs.

---

Architecture

Platform detection

app/test/e2e/helpers/platform.ts exports:

• isTauriDriver(), true on Linux (tauri-driver session)
• isMac2(), true on macOS (Appium Mac2 session)
• supportsExecuteScript(), true when browser.execute() works (tauri-driver only)

Element helpers

app/test/e2e/helpers/element-helpers.ts provides a unified API:

Helper	Mac2 (macOS)	tauri-driver (Linux)
waitForText(text)	XPath over @label/@value/@title	XPath over DOM text content
waitForButton(text)	XCUIElementTypeButton XPath	button / [role="button"] XPath
clickText(text)	W3C pointer actions	Standard el.click()
clickNativeButton(text)	W3C pointer actions on XCUIElementTypeButton	Standard el.click() on button
clickToggle()	XCUIElementTypeSwitch / XCUIElementTypeCheckBox	[role="switch"] / input[type="checkbox"]
waitForWindowVisible()	XCUIElementTypeWindow	Window handle check
waitForWebView()	XCUIElementTypeWebView	document.readyState check
hasAppChrome()	XCUIElementTypeMenuBar	Window handle check
dumpAccessibilityTree()	Accessibility XML	HTML page source

Stable test IDs

Prefer stable data-testid hooks for UI affordances that E2E specs click or poll. Use the taxonomy <surface>-<element>-<id?>, for example:

• cron-jobs-panel, cron-refresh
• cron-job-row-<jobId>, cron-job-toggle-<jobId>, cron-job-run-<jobId>, cron-job-view-runs-<jobId>, cron-job-remove-<jobId>
• settings-nav-<routeId>
• skill-row-<skillId>, skill-install-<skillId>, skill-uninstall-<skillId>
• thread-row-<threadId>, new-thread-button, send-message-button
• onboarding-next-button

Use waitForTestId(testId) and clickTestId(testId) from element-helpers.ts when a spec targets one of these hooks. Keep text selectors for user-visible copy assertions, not row/action discovery.

Deep link helpers

app/test/e2e/helpers/deep-link-helpers.ts handles auth deep links:

• tauri-driver: browser.execute(window.__simulateDeepLink(url)) (primary), xdg-open (fallback)
• Appium Mac2: macos: deepLink extension command (primary), open -a ... (fallback)

Writing cross-platform specs

1. Use helpers from element-helpers.ts, never use raw XCUIElementType* selectors in specs
2. Use clickNativeButton(text) instead of inline button-clicking code
3. Use hasAppChrome() instead of checking for XCUIElementTypeMenuBar
4. Use waitForWebView() instead of checking for XCUIElementTypeWebView
5. For macOS-only tests, use process.platform guards or separate spec files

---

Environment variables

Variable	Default	Description
TAURI_DRIVER_PORT	4444	tauri-driver WebDriver port
APPIUM_PORT	4723	Appium server port
E2E_MOCK_PORT	18473	Mock backend server port
OPENHUMAN_WORKSPACE	(temp dir)	App workspace directory
OPENHUMAN_SERVICE_MOCK	0	Enable service mock mode
OPENHUMAN_E2E_MODE	unset	Enables destructive test-support RPCs; the E2E runner sets this to 1
OPENHUMAN_E2E_AUTH_BYPASS	unset	Enable JWT bypass auth
DEBUG_E2E_DEEPLINK	(verbose)	Set to 0 to silence deep link logs
E2E_FORCE_CARGO_CLEAN	unset	Force cargo clean before E2E build

---

CI workflows

Push / PR checks

The default test.yml workflow runs frontend unit tests and Rust checks. Its Linux tauri-driver E2E job is commented out because WebKitWebDriver cannot drive the CEF-backed WebView.

The disabled Linux E2E job used to:

1. Installs system deps (webkit2gtk, Xvfb, dbus)
2. Installs tauri-driver via cargo
3. Builds the app with mock server URL baked in
4. Runs all E2E flows under Xvfb

macOS / Appium

macOS/Appium is the supported automation backend for the current CEF desktop app. Run it locally, or through a manually dispatched macOS workflow when that workflow is enabled:

1. Installs Appium + Mac2 driver
2. Builds the .app bundle
3. Runs all E2E flows

---

Troubleshooting

Linux: "WebView not ready" timeout

For the default CEF runtime, this usually means the unsupported Linux tauri-driver path is trying to drive a CEF-backed WebView through WebKitWebDriver. Use macOS/Appium, or wait for a CEF-compatible Linux driver.

Ensure DISPLAY is set and Xvfb is running:

export DISPLAY=:99
Xvfb :99 -screen 0 1280x1024x24 &

Also ensure dbus is started (required by webkit2gtk):

eval $(dbus-launch --sh-syntax)

Linux: tauri-driver not found

cargo install tauri-driver

macOS: Deep links not working in tauri dev

Deep links require a .app bundle. Use pnpm tauri build --debug --bundles app instead.

Docker: Build is slow on first run

The first Docker build compiles Rust + tauri-driver from source. Subsequent runs use cached layers. Cargo registry and git sources are cached via Docker volumes.

Spec: Notifications

File: app/test/e2e/specs/notifications.spec.ts

Tests notification RPC methods via the live core sidecar and the Notifications UI page:

• notification_ingest, creates a new notification via core RPC
• notification_list, verifies the ingested notification is returned
• notification_mark_read, marks a notification as read
• notification_stats, checks aggregate statistics shape
• UI: Notifications page renders the integration notifications section ([data-testid="integration-notifications-section"])
• UI: Notifications page shows the System Events section ([data-testid="system-events-section"])

Run:

bash app/scripts/e2e-run-spec.sh test/e2e/specs/notifications.spec.ts notifications

Platform note: RPC tests (notification_ingest, notification_list, notification_mark_read, notification_stats) are written for both Linux/tauri-driver and macOS/Appium Mac2, but Linux execution is disabled for the default CEF runtime until a CEF-compatible driver exists. UI assertions (Notifications page sections) require browser.execute() support, so they auto-skip on Mac2 when supportsExecuteScript() returns false.

---

Agent-observable artifact flow

For a canonical, inspectable run that drops screenshots, page-source dumps, and mock request logs on disk:

bash app/scripts/e2e-agent-review.sh

Artifacts land in app/test/e2e/artifacts/<timestamp>-agent-review/. Full details + helper API: AGENT-OBSERVABILITY.md. Any failing test triggers wdio.conf.ts's afterTest hook, which writes failure-*.png + failure-*.source.xml into the same run dir.

---

Rust inference provider E2E

These tests (tests/inference_provider_e2e.rs) use wiremock to mock HTTP upstreams and require no live LLM API calls. They cover OpenAI-compat chat, Anthropic auth style, per-model temperature suppression, Ollama local provider, and the /v1 HTTP endpoint auth layer.

# Local:
bash scripts/test-rust-inference-e2e.sh

# Via Docker (Linux, same image as CI):
docker compose -f e2e/docker-compose.yml run --rm inference-e2e