Files
openhuman/.github/workflows/test.yml
T
cf344facf9 feat(voice): dedicated voice assistance module for STT/TTS (#178)
* feat(voice): add dedicated voice assistance module for STT/TTS

Extracts speech-to-text (whisper.cpp) and text-to-speech (piper) into a
dedicated `src/openhuman/voice/` domain module with its own RPC namespace
(`openhuman.voice_*`). Adds proactive availability checking via
`voice_status` so the UI can show clear errors when binaries/models are
missing instead of failing silently at transcription time.

- New module: voice/types.rs, voice/ops.rs, voice/schemas.rs, voice/mod.rs
- 4 RPC endpoints: voice_status, voice_transcribe, voice_transcribe_bytes, voice_tts
- 21 unit tests + 1 integration test (json_rpc_e2e)
- Frontend updated to use voice_* endpoints with status check on mode switch

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: fix cargo fmt in voice/ops.rs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(e2e): add voice mode integration spec

Tests switching to voice input mode, verifying status check fires,
recording button renders, and switching back to text mode restores
text input. Also checks reply mode toggle visibility.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove unused waitForText import in voice-mode e2e spec

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(voice): in-process whisper engine and LLM post-processing

- Add whisper-rs (0.16) for in-process whisper.cpp inference, eliminating
  cold-start latency from subprocess-per-call (~1-3s) to warm inference
  (~50ms). Model is loaded once during bootstrap and reused across calls.
  Falls back to whisper-cli subprocess if in-process loading fails.

- Add LLM post-processing layer that passes raw transcription through
  Ollama to fix grammar, punctuation, and filler words. Accepts optional
  conversation context to disambiguate names and technical terms.
  Gracefully degrades to raw whisper output if Ollama is unavailable.

- Update voice RPC endpoints with new optional params (context,
  skip_cleanup) and return both cleaned text and raw_text.

- Update frontend to pass conversation history as context for voice
  transcription cleanup, and update TypeScript interfaces to match.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: apply cargo fmt formatting fixes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(build): make whisper-rs optional behind `whisper` feature flag

The whisper-rs crate requires cmake to compile whisper.cpp from source,
which is not available in the CI environment. Move it behind an optional
cargo feature so CI builds succeed without cmake.

The whisper_engine module now compiles as a no-op stub when the feature
is disabled, returning "whisper feature not compiled in" errors. Desktop
builds can opt in with `--features whisper`.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: cargo fmt whisper_engine.rs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): make whisper-rs mandatory and install cmake in CI

Revert whisper-rs from optional to mandatory dependency. Add cmake
installation to all CI workflows (build, typecheck, test, release) and
the CI Docker image so whisper-rs can compile whisper.cpp from source.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: cargo fmt whisper_engine.rs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address code review findings across voice module

- whisper_engine: validate WAV sample rate (must be 16kHz) and channel
  count (1 or 2) before feeding audio to whisper
- speech: offload load_engine and transcribe_in_process to
  tokio::task::spawn_blocking to avoid blocking the Tokio runtime
- ops: use RAII guard for WHISPER_BIN env var in test to prevent races
  and ensure restore on panic; log temp file cleanup failures instead
  of silently ignoring; sanitize paths in debug logs to basenames only
- postprocess: add test for disabled cleanup config returning raw text
- voice-mode.spec: assert failure when neither voice CTA nor
  unavailable message appears; make reply mode test runnable in
  isolation with auth/nav setup

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: apply cargo fmt

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 15:52:52 -07:00

256 lines
7.7 KiB
YAML

name: Test
on:
push:
branches: [main]
pull_request:
workflow_dispatch:
inputs:
run_macos_e2e:
description: 'Run macOS E2E tests (Appium Mac2)'
required: false
default: 'false'
type: choice
options:
- 'false'
- 'true'
permissions:
contents: read
pull-requests: read
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.head_ref || github.ref }}
cancel-in-progress: true
jobs:
unit-tests:
name: Frontend Unit Tests
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Setup Node.js 24.x
uses: actions/setup-node@v4
with:
node-version: 24.x
cache: "yarn"
- name: Cache node modules
id: yarn-cache
uses: actions/cache@v4
with:
path: node_modules
key: ${{ runner.os }}-test-${{ hashFiles('**/yarn.lock') }}
restore-keys: |
${{ runner.os }}-test-
${{ runner.os }}-build-
- name: Install dependencies
if: steps.yarn-cache.outputs.cache-hit != 'true'
run: yarn install --frozen-lockfile
- name: Run tests with coverage
run: yarn test:coverage
env:
NODE_ENV: test
- name: Upload coverage reports
if: always()
uses: actions/upload-artifact@v4
with:
name: coverage-report
path: coverage
retention-days: 7
rust-tests:
name: Rust Tests + Quality
runs-on: ubuntu-22.04
container:
image: ghcr.io/tinyhumansai/openhuman_ci:rust-1.93.0
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Cache Rust build artifacts
uses: Swatinem/rust-cache@v2
with:
workspaces: |
. -> target
app/src-tauri -> target
cache-on-failure: true
- name: Install cmake (for whisper-rs)
run: apt-get update && apt-get install -y --no-install-recommends cmake && rm -rf /var/lib/apt/lists/*
- name: Check formatting (cargo fmt)
run: cargo fmt --all -- --check
- name: Run clippy (core crate)
run: cargo clippy -p openhuman
- name: Test core crate (openhuman)
run: cargo test -p openhuman
- name: Build sidecar core binary
run: cargo build --profile ci --target x86_64-unknown-linux-gnu --bin openhuman-core
- name: Stage sidecar for Tauri shell tests
run: |
mkdir -p app/src-tauri/binaries
cp target/x86_64-unknown-linux-gnu/ci/openhuman-core app/src-tauri/binaries/openhuman-core-x86_64-unknown-linux-gnu
chmod +x app/src-tauri/binaries/openhuman-core-x86_64-unknown-linux-gnu
- name: Test Tauri shell (OpenHuman)
run: cargo test --manifest-path app/src-tauri/Cargo.toml
e2e-linux:
name: E2E (Linux / tauri-driver)
runs-on: ubuntu-22.04
timeout-minutes: 60
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 1
submodules: recursive
- name: Setup Node.js 24.x
uses: actions/setup-node@v4
with:
node-version: 24.x
cache: "yarn"
- name: Install Rust (rust-toolchain.toml)
uses: dtolnay/rust-toolchain@1.93.0
- name: Install system dependencies
run: |
sudo apt-get update
sudo apt-get install -y \
libgtk-3-dev libwebkit2gtk-4.1-dev libappindicator3-dev \
librsvg2-dev patchelf \
xvfb at-spi2-core dbus-x11 \
webkit2gtk-driver
- name: Cargo.lock fingerprint (deps only)
id: cargo-lock-fingerprint
shell: bash
run: |
echo "hash=$(tail -n +8 Cargo.lock | openssl dgst -sha256 | awk '{print $2}')" >> "$GITHUB_OUTPUT"
- name: Cache Cargo registry and build
uses: actions/cache@v4
with:
path: |
~/.cargo/registry
~/.cargo/git
target
key: ${{ runner.os }}-e2e-cargo-${{ steps.cargo-lock-fingerprint.outputs.hash }}
restore-keys: |
${{ runner.os }}-e2e-cargo-
- name: Install tauri-driver
run: cargo install tauri-driver
- name: Install JS dependencies
run: yarn install --frozen-lockfile
- name: Ensure .env exists for E2E build
run: |
touch .env
touch app/.env
- name: Build E2E app
run: yarn workspace openhuman-app test:e2e:build
- name: Stage sidecar next to app binary
run: |
# Tauri resolves externalBin relative to the running binary's directory.
# Copy the sidecar from binaries/ to target/debug/ so the app can find it.
cp app/src-tauri/binaries/openhuman-core-x86_64-unknown-linux-gnu \
app/src-tauri/target/debug/openhuman-core-x86_64-unknown-linux-gnu
chmod +x app/src-tauri/target/debug/openhuman-core-x86_64-unknown-linux-gnu
echo "Sidecar staged next to app binary:"
ls -la app/src-tauri/target/debug/openhuman-core-* app/src-tauri/target/debug/OpenHuman
- name: Run E2E tests under Xvfb
run: |
export DISPLAY=:99
Xvfb :99 -screen 0 1280x1024x24 &
sleep 2
# dbus session is required by webkit2gtk
eval "$(dbus-launch --sh-syntax)"
# Ensure XDG dirs exist for deep-link URL scheme registration on Linux
mkdir -p ~/.local/share/applications
export RUST_BACKTRACE=1
cd app
# Core specs — must pass on Linux CI
FAILED=0
for spec in \
test/e2e/specs/login-flow.spec.ts \
test/e2e/specs/smoke.spec.ts \
test/e2e/specs/navigation.spec.ts \
test/e2e/specs/telegram-flow.spec.ts; do
SPEC_NAME=$(basename "$spec" .spec.ts)
echo "=== Running $SPEC_NAME ==="
bash scripts/e2e-run-spec.sh "$spec" "$SPEC_NAME" || {
echo "FAILED: $SPEC_NAME"
cat /tmp/tauri-driver-e2e-${SPEC_NAME}.log 2>/dev/null || true
FAILED=1
}
done
# Extended specs (auth, billing, gmail, notion, payments) are skipped
# on Linux CI — webkit2gtk text matching differences cause Settings
# page navigation timeouts. Full suite runs on macOS locally.
if [ "$FAILED" -eq 1 ]; then
echo "Core E2E specs failed"
exit 1
fi
echo "Core E2E specs passed"
# e2e-macos:
# name: E2E (macOS / Appium)
# if: github.event_name == 'workflow_dispatch' && github.event.inputs.run_macos_e2e == 'true'
# runs-on: macos-latest
# timeout-minutes: 90
# steps:
# - name: Checkout code
# uses: actions/checkout@v4
# with:
# fetch-depth: 1
# submodules: recursive
# - name: Setup Node.js 24.x
# uses: actions/setup-node@v4
# with:
# node-version: 24.x
# cache: "yarn"
# - name: Install Rust (rust-toolchain.toml)
# uses: dtolnay/rust-toolchain@1.93.0
# - name: Install dependencies
# run: yarn install --frozen-lockfile
# - name: Ensure .env exists for E2E build
# run: |
# touch .env
# touch app/.env
# - name: Install Appium and mac2 driver
# run: |
# npm install -g appium
# appium driver install mac2
# - name: Build E2E app bundle
# run: yarn workspace openhuman-app test:e2e:build
# - name: Run all E2E flows
# run: yarn workspace openhuman-app test:e2e:all:flows