Files
openhuman/.github/workflows/test.yml
T
11f718d8bc feat(voice): standalone voice dictation server with hotkey support (#368)
* feat: add standalone voice dictation server with hotkey support

- Introduced a new `voice` subcommand to the CLI for running a standalone voice dictation server that listens for a hotkey, records audio, transcribes it using Whisper, and inserts the result into the active text field.
- Implemented configuration options for the voice server, including hotkey combination, activation mode (tap or push), and an option to skip LLM post-processing.
- Added audio capture functionality using the `cpal` crate and integrated hotkey listening with the `rdev` crate for global key event handling.
- Enhanced the configuration schema to include voice server settings and updated the main configuration structure accordingly.
- Updated relevant modules and tests to ensure consistent behavior and functionality across the application.

This feature enhances user interaction by allowing voice dictation directly into any active text field, improving accessibility and usability.

* feat: add voice dictation server with hotkey support

- Introduced a standalone voice dictation server that listens for a configurable hotkey to start recording audio, transcribes it using whisper, and inserts the transcribed text into the active text field.
- Added CLI support for the `voice` command, allowing users to manage the voice server's configuration, including hotkey and activation mode settings.
- Implemented configuration structures for the voice server, including options for automatic start, hotkey combination, activation mode, and cleanup behavior.
- Enhanced audio capture functionality using the `cpal` library for microphone input and integrated text insertion using the `enigo` library for simulating keyboard input.
- Updated relevant modules and schemas to support the new voice server features, ensuring a cohesive integration within the OpenHuman platform.

* refactor: streamline voice server command and enhance audio capture functionality

- Updated the `run_voice_server_command` function to initialize the configuration with environment overrides instead of loading from a file, improving performance and flexibility.
- Refactored the audio capture logic in `start_recording` to enhance thread management and error handling, ensuring a more robust audio stream setup.
- Improved the handling of audio stream creation and playback, ensuring that all cpal objects are managed on the same thread as required, enhancing stability during recording operations.

* fix: remove unused import in voice server module

- Eliminated the `HotkeyListenerHandle` import from the `server.rs` file, streamlining the code and improving clarity by removing unnecessary dependencies.

* feat(voice): auto-enable LLM cleanup when local model is ready

The postprocessor now checks the local LLM state and automatically
enables transcription cleanup when the model is downloaded and ready,
even if not explicitly configured. Falls back gracefully to raw text
when the LLM is unavailable.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: apply cargo fmt + prettier formatting

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(voice): dictation config, hotkey lifecycle, and WebSocket streaming (#332)

Add the foundational infrastructure for voice dictation (EPIC #332):

**Rust core:**
- New `DictationConfig` schema with serde defaults and env var overrides
  (enabled, hotkey, activation_mode, llm_refinement, streaming, interval)
- RPC controllers: `config_get_dictation_settings` / `config_update_dictation_settings`
- WebSocket endpoint `/ws/dictation` for streaming PCM16 transcription
  with periodic partial inference and final LLM refinement
- Microphone permission declaration (`NSMicrophoneUsageDescription`) in
  Tauri macOS bundle config

**Frontend:**
- `useDictationHotkey` hook: fetches config from core RPC, auto-registers
  global hotkey, listens for `dictation://toggle` events
- `DictationHotkeyManager` headless component mounted in App.tsx
- Fix voice RPC response type mismatch: voice handlers return flat results
  (no `{result, logs}` wrapper), so remove incorrect `CommandResponse<T>`
  wrapping from `openhumanVoiceStatus`, `openhumanVoiceTranscribe`,
  `openhumanVoiceTranscribeBytes`, and `openhumanVoiceTts`

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(tauri): remove invalid infoPlist config that breaks tauri dev

The `infoPlist` field in tauri.conf.json expects a string path, not an
inline object. Remove it for now — microphone permission will be added
via a proper Info.plist supplement in the production build pipeline.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style: apply Prettier formatting to dictation files

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* format files

* feat(dictation): integrate dictation listener and event broadcasting

- Added a global dictation hotkey listener that activates based on configuration.
- Implemented a web channel bridge to handle dictation events and broadcast them to connected clients.
- Updated the voice module to include the new dictation listener functionality.

This enhances the voice dictation capabilities by ensuring real-time event handling and client communication.

* update code

* format

* feat(voice): enhance voice server configuration and functionality

- Updated `Cargo.toml` to mark voice-related dependencies as optional.
- Introduced `VoiceActivationMode` enum for better control over voice server activation.
- Refactored voice server command handling and dictation event broadcasting to support new features.
- Added conditional compilation for voice features across various modules, ensuring they are only included when enabled.

This commit improves the modularity and configurability of the voice server, allowing for more flexible integration and usage.

* refactor: clean up whitespace and formatting in core and voice modules

- Removed unnecessary blank lines in `cli.rs`, `jsonrpc.rs`, `schemas.rs`, and `socketio.rs` to improve code readability.
- Adjusted import order in `mod.rs` for better organization.

This commit enhances the overall code quality by ensuring consistent formatting across multiple files.

* chore: update Dockerfile and test workflow to install additional system dependencies

- Added installation of system dependencies (cmake, ALSA, X11) in the Dockerfile for improved build support.
- Updated the GitHub Actions workflow to reflect the new dependencies, ensuring consistent environment setup for testing.

This commit enhances the build environment by including necessary libraries for audio and GUI support.

* format

* fix claude

* format

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: oxoxDev <nikhil@tinyhumans.ai>
2026-04-06 16:08:52 -07:00

257 lines
7.9 KiB
YAML

name: Test
on:
push:
branches: [main]
pull_request:
workflow_dispatch:
inputs:
run_macos_e2e:
description: 'Run macOS E2E tests (Appium Mac2)'
required: false
default: 'false'
type: choice
options:
- 'false'
- 'true'
permissions:
contents: read
pull-requests: read
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.head_ref || github.ref }}
cancel-in-progress: true
jobs:
unit-tests:
name: Frontend Unit Tests
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Setup Node.js 24.x
uses: actions/setup-node@v4
with:
node-version: 24.x
cache: "yarn"
- name: Cache node modules
id: yarn-cache
uses: actions/cache@v4
with:
path: node_modules
key: ${{ runner.os }}-test-${{ hashFiles('**/yarn.lock') }}
restore-keys: |
${{ runner.os }}-test-
${{ runner.os }}-build-
- name: Install dependencies
if: steps.yarn-cache.outputs.cache-hit != 'true'
run: yarn install --frozen-lockfile
- name: Run tests with coverage
run: yarn test:coverage
env:
NODE_ENV: test
- name: Upload coverage reports
if: always()
uses: actions/upload-artifact@v4
with:
name: coverage-report
path: coverage
retention-days: 7
rust-tests:
name: Rust Tests + Quality
runs-on: ubuntu-22.04
container:
image: ghcr.io/tinyhumansai/openhuman_ci:rust-1.93.0
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Cache Rust build artifacts
uses: Swatinem/rust-cache@v2
with:
workspaces: |
. -> target
app/src-tauri -> target
cache-on-failure: true
- name: Install system dependencies (cmake, ALSA, X11)
run: apt-get update && apt-get install -y --no-install-recommends cmake libasound2-dev libxdo-dev libxtst-dev libx11-dev libevdev-dev && rm -rf /var/lib/apt/lists/*
- name: Check formatting (cargo fmt)
run: cargo fmt --all -- --check
- name: Run clippy (core crate)
run: cargo clippy -p openhuman
- name: Test core crate (openhuman)
run: cargo test -p openhuman
- name: Build sidecar core binary
run: cargo build --profile ci --target x86_64-unknown-linux-gnu --bin openhuman-core
- name: Stage sidecar for Tauri shell tests
run: |
mkdir -p app/src-tauri/binaries
cp target/x86_64-unknown-linux-gnu/ci/openhuman-core app/src-tauri/binaries/openhuman-core-x86_64-unknown-linux-gnu
chmod +x app/src-tauri/binaries/openhuman-core-x86_64-unknown-linux-gnu
- name: Test Tauri shell (OpenHuman)
run: cargo test --manifest-path app/src-tauri/Cargo.toml
e2e-linux:
name: E2E (Linux / tauri-driver)
runs-on: ubuntu-22.04
timeout-minutes: 60
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 1
submodules: recursive
- name: Setup Node.js 24.x
uses: actions/setup-node@v4
with:
node-version: 24.x
cache: "yarn"
- name: Install Rust (rust-toolchain.toml)
uses: dtolnay/rust-toolchain@1.93.0
- name: Install system dependencies
run: |
sudo apt-get update
sudo apt-get install -y \
libgtk-3-dev libwebkit2gtk-4.1-dev libappindicator3-dev \
librsvg2-dev patchelf \
xvfb at-spi2-core dbus-x11 \
webkit2gtk-driver \
libasound2-dev libxdo-dev libxtst-dev libx11-dev libevdev-dev
- name: Cargo.lock fingerprint (deps only)
id: cargo-lock-fingerprint
shell: bash
run: |
echo "hash=$(tail -n +8 Cargo.lock | openssl dgst -sha256 | awk '{print $2}')" >> "$GITHUB_OUTPUT"
- name: Cache Cargo registry and build
uses: actions/cache@v4
with:
path: |
~/.cargo/registry
~/.cargo/git
target
key: ${{ runner.os }}-e2e-cargo-${{ steps.cargo-lock-fingerprint.outputs.hash }}
restore-keys: |
${{ runner.os }}-e2e-cargo-
- name: Install tauri-driver
run: cargo install tauri-driver --version 2.0.5
- name: Install JS dependencies
run: yarn install --frozen-lockfile
- name: Ensure .env exists for E2E build
run: |
touch .env
touch app/.env
- name: Build E2E app
run: yarn workspace openhuman-app test:e2e:build
- name: Stage sidecar next to app binary
run: |
# Tauri resolves externalBin relative to the running binary's directory.
# Copy the sidecar from binaries/ to target/debug/ so the app can find it.
cp app/src-tauri/binaries/openhuman-core-x86_64-unknown-linux-gnu \
app/src-tauri/target/debug/openhuman-core-x86_64-unknown-linux-gnu
chmod +x app/src-tauri/target/debug/openhuman-core-x86_64-unknown-linux-gnu
echo "Sidecar staged next to app binary:"
ls -la app/src-tauri/target/debug/openhuman-core-* app/src-tauri/target/debug/OpenHuman
- name: Run E2E tests under Xvfb
run: |
export DISPLAY=:99
Xvfb :99 -screen 0 1280x1024x24 &
sleep 2
# dbus session is required by webkit2gtk
eval "$(dbus-launch --sh-syntax)"
# Ensure XDG dirs exist for deep-link URL scheme registration on Linux
mkdir -p ~/.local/share/applications
export RUST_BACKTRACE=1
cd app
# Core specs — must pass on Linux CI
FAILED=0
for spec in \
test/e2e/specs/login-flow.spec.ts \
test/e2e/specs/smoke.spec.ts \
test/e2e/specs/navigation.spec.ts \
test/e2e/specs/telegram-flow.spec.ts; do
SPEC_NAME=$(basename "$spec" .spec.ts)
echo "=== Running $SPEC_NAME ==="
bash scripts/e2e-run-spec.sh "$spec" "$SPEC_NAME" || {
echo "FAILED: $SPEC_NAME"
cat /tmp/tauri-driver-e2e-${SPEC_NAME}.log 2>/dev/null || true
FAILED=1
}
done
# Extended specs (auth, billing, gmail, notion, payments) are skipped
# on Linux CI — webkit2gtk text matching differences cause Settings
# page navigation timeouts. Full suite runs on macOS locally.
if [ "$FAILED" -eq 1 ]; then
echo "Core E2E specs failed"
exit 1
fi
echo "Core E2E specs passed"
# e2e-macos:
# name: E2E (macOS / Appium)
# if: github.event_name == 'workflow_dispatch' && github.event.inputs.run_macos_e2e == 'true'
# runs-on: macos-latest
# timeout-minutes: 90
# steps:
# - name: Checkout code
# uses: actions/checkout@v4
# with:
# fetch-depth: 1
# submodules: recursive
# - name: Setup Node.js 24.x
# uses: actions/setup-node@v4
# with:
# node-version: 24.x
# cache: "yarn"
# - name: Install Rust (rust-toolchain.toml)
# uses: dtolnay/rust-toolchain@1.93.0
# - name: Install dependencies
# run: yarn install --frozen-lockfile
# - name: Ensure .env exists for E2E build
# run: |
# touch .env
# touch app/.env
# - name: Install Appium and mac2 driver
# run: |
# npm install -g appium
# appium driver install mac2
# - name: Build E2E app bundle
# run: yarn workspace openhuman-app test:e2e:build
# - name: Run all E2E flows
# run: yarn workspace openhuman-app test:e2e:all:flows