mirror of
https://github.com/tinyhumansai/openhuman.git
synced 2026-07-30 23:14:37 +00:00
ci: speed up GitHub Actions builds (~14m → ~3-5m warm) (#136)
* chore: add CI profile for faster compilation in Cargo.toml files - Introduced a new `[profile.ci]` section in both root and Tauri Cargo.toml files to optimize build settings for continuous integration. - Adjusted compilation parameters to prioritize speed over runtime performance, including reduced optimization level and enabled code generation units. * refactor(tests): update agent test setup to return temporary directory - Modified the `build_agent_with` function calls in the agent tests to return a temporary directory alongside the agent instance, improving resource management during tests. - Ensured consistency in test setup across multiple test functions. * chore: update .gitignore to include fastembed_cache - Added 'workflow' and '.fastembed_cache' to the .gitignore file to prevent unnecessary files from being tracked in the repository. * test: enhance dispatch routing tests with panic handling Updated the `dispatch_routes_memory_doc_ingest` test to use `AssertUnwindSafe` and `catch_unwind` for better handling of potential panics during execution. This ensures that the test verifies route existence even if the handler encounters a panic, improving robustness against shared state issues in concurrent tests. * ci: speed up builds with rust-cache, sccache, mold linker, and CI profile - Replace manual Cargo registry cache with Swatinem/rust-cache@v2 (caches target/ directories for both core and Tauri crates) - Add mozilla-actions/sccache for cross-branch compilation caching - Install mold linker on Linux for faster linking - Use --profile ci for sidecar build (opt-level 1, codegen-units 16) - Override release profile env vars for Tauri build with CI-tuned settings - Add --bundles none to CI build (skip unused deb/appimage packaging) - Restrict push triggers to main branch only (PRs already cover feature branches, preventing duplicate runs) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(ci): unset RUSTC_WRAPPER for cargo fmt to avoid sccache errors The sccache-action sets RUSTC_WRAPPER globally for the job. cargo fmt invokes rustc through sccache which fails if the GHA cache service is unavailable. Clear RUSTC_WRAPPER for fmt steps and remove redundant per-step env overrides (the action already sets them job-wide). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: implement Default trait for various structs and enums - Added Default implementations for ConnectionStatus, AgentBuilder, NativeRuntime, AutocompleteEngine, CliChannel, ActionTracker, SkillStatus, and ExtractionMode to streamline object initialization. - Simplified condition checks in several places by replacing map_or with is_some_and and is_none_or for better readability and performance. - Updated various instances of string handling in message sending to remove unnecessary conversions. This refactor enhances code clarity and consistency across the codebase. * style: clean up whitespace and improve code formatting - Removed unnecessary blank lines in several files to enhance code readability. - Simplified condition checks by consolidating method calls into single lines for better clarity. - Improved formatting in various functions to maintain consistency across the codebase. * fix(ci): use --bundles deb instead of unsupported none value Tauri CLI on this version doesn't support 'none' as a bundle type. Use 'deb' as the lightest single bundle to minimize packaging time. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(ci): add Dockerfile and CI workflows for building and pushing Docker images - Introduced a Dockerfile to set up a CI environment with necessary dependencies for Tauri, Rust, Node.js, and sccache. - Created a new workflow to build and push the CI Docker image to GitHub Container Registry on main branch pushes. - Updated existing workflows to utilize the new Docker image for building and testing, enhancing consistency and efficiency in CI processes. * chore(ci): update container image references in CI workflows - Removed unnecessary permissions for packages in build, test, and typecheck workflows. - Updated container image references to use a specific digest instead of the latest tag for improved stability and reproducibility in CI processes. * fix(ci): use correct amd64 Docker image digest Previous digest was from arm64 build (Mac). Rebuilt with --platform linux/amd64 for GitHub Actions runners. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(ci): use tag instead of digest for Docker image reference GHCR doesn't support pulling OCI index manifests by digest reliably. Use the rust-1.93.0 tag which is pinned to a specific Rust version and resolves correctly on amd64 CI runners. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(ci): rename Docker package to openhuman_ci Move from nested ghcr.io/tinyhumansai/openhuman/ci-runner to ghcr.io/tinyhumansai/openhuman_ci to avoid GHCR nested package manifest resolution issues. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(ci): remove sccache env vars from container jobs sccache can't access the GHA cache API from inside a Docker container (missing ACTIONS_CACHE_URL/ACTIONS_RUNTIME_TOKEN). Swatinem/rust-cache already caches target/ which provides the main build speedup. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update installer smoke workflow to trigger on main branch pushes - Added a trigger for the installer smoke workflow to run on pushes to the main branch, enhancing CI coverage for mainline changes. * fix: enhance Sentry DSN retrieval logic - Updated the Sentry DSN retrieval process to include an additional fallback option using `option_env!`, ensuring that the DSN can be sourced from both environment variables and optional configuration, improving robustness in observability setup. * chore: add OPENHUMAN_SENTRY_DSN to release workflow and example secrets - Included the OPENHUMAN_SENTRY_DSN variable in the release workflow configuration to enhance observability setup. - Updated the ci-secrets.example.json file to include a placeholder for OPENHUMAN_SENTRY_DSN, providing clarity for developers on required environment variables. --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.6
parent
8486a3b02e
commit
c5e5ae170c
@@ -0,0 +1,43 @@
|
||||
FROM ubuntu:22.04
|
||||
|
||||
ENV DEBIAN_FRONTEND=noninteractive
|
||||
|
||||
# System deps for Tauri + mold linker + clang (for mold)
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
build-essential \
|
||||
curl \
|
||||
ca-certificates \
|
||||
git \
|
||||
pkg-config \
|
||||
libgtk-3-dev \
|
||||
libwebkit2gtk-4.1-dev \
|
||||
libappindicator3-dev \
|
||||
librsvg2-dev \
|
||||
patchelf \
|
||||
mold \
|
||||
clang \
|
||||
libssl-dev \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Rust 1.93.0 with minimal profile + fmt/clippy
|
||||
ENV RUSTUP_HOME=/usr/local/rustup \
|
||||
CARGO_HOME=/usr/local/cargo \
|
||||
PATH="/usr/local/cargo/bin:$PATH"
|
||||
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- \
|
||||
-y --default-toolchain 1.93.0 --profile minimal \
|
||||
-c rustfmt -c clippy \
|
||||
-t x86_64-unknown-linux-gnu
|
||||
|
||||
# Node.js 24.x + yarn
|
||||
RUN curl -fsSL https://deb.nodesource.com/setup_24.x | bash - \
|
||||
&& apt-get install -y --no-install-recommends nodejs \
|
||||
&& npm install -g yarn \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# sccache (pre-installed so the action only needs to configure it)
|
||||
RUN curl -fsSL https://github.com/mozilla/sccache/releases/download/v0.10.0/sccache-v0.10.0-x86_64-unknown-linux-musl.tar.gz \
|
||||
| tar xz -C /usr/local/bin --strip-components=1 sccache-v0.10.0-x86_64-unknown-linux-musl/sccache \
|
||||
&& chmod +x /usr/local/bin/sccache
|
||||
|
||||
# Verify installs
|
||||
RUN rustc --version && cargo --version && node --version && yarn --version && mold --version && sccache --version
|
||||
+17
-34
@@ -2,6 +2,7 @@ name: Build
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
|
||||
permissions:
|
||||
@@ -16,6 +17,8 @@ jobs:
|
||||
build:
|
||||
name: Build Tauri App
|
||||
runs-on: ubuntu-22.04
|
||||
container:
|
||||
image: ghcr.io/tinyhumansai/openhuman_ci:rust-1.93.0
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
@@ -23,38 +26,13 @@ jobs:
|
||||
fetch-depth: 1
|
||||
submodules: true
|
||||
|
||||
- name: Setup Node.js 24.x
|
||||
uses: actions/setup-node@v4
|
||||
- name: Cache Rust build artifacts
|
||||
uses: Swatinem/rust-cache@v2
|
||||
with:
|
||||
node-version: 24.x
|
||||
cache: "yarn"
|
||||
|
||||
- name: Install Rust (rust-toolchain.toml)
|
||||
uses: dtolnay/rust-toolchain@1.93.0
|
||||
with:
|
||||
targets: x86_64-unknown-linux-gnu
|
||||
|
||||
- name: Install Tauri dependencies
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y libgtk-3-dev libwebkit2gtk-4.1-dev libappindicator3-dev librsvg2-dev patchelf
|
||||
|
||||
# Skip first 7 lines of Cargo.lock (workspace package version bumps) so the key tracks dependency changes only
|
||||
- name: Cargo.lock fingerprint (deps only)
|
||||
id: cargo-lock-fingerprint
|
||||
shell: bash
|
||||
run: |
|
||||
echo "hash=$(tail -n +8 Cargo.lock | openssl dgst -sha256 | awk '{print $2}')" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Cache Cargo registry and git sources
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: |
|
||||
~/.cargo/registry
|
||||
~/.cargo/git
|
||||
key: ${{ runner.os }}-cargo-registry-${{ steps.cargo-lock-fingerprint.outputs.hash }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-cargo-registry-
|
||||
workspaces: |
|
||||
. -> target
|
||||
app/src-tauri -> target
|
||||
cache-on-failure: true
|
||||
|
||||
- name: Cache node modules
|
||||
id: yarn-cache
|
||||
@@ -70,18 +48,23 @@ jobs:
|
||||
run: yarn install --frozen-lockfile
|
||||
|
||||
- name: Build sidecar core binary
|
||||
run: cargo build --manifest-path Cargo.toml --release --target x86_64-unknown-linux-gnu --bin openhuman-core
|
||||
run: cargo build --profile ci --target x86_64-unknown-linux-gnu --bin openhuman-core
|
||||
|
||||
- name: Stage sidecar for Tauri bundler
|
||||
run: |
|
||||
mkdir -p app/src-tauri/binaries
|
||||
cp target/x86_64-unknown-linux-gnu/release/openhuman-core app/src-tauri/binaries/openhuman-core-x86_64-unknown-linux-gnu
|
||||
cp target/x86_64-unknown-linux-gnu/ci/openhuman-core app/src-tauri/binaries/openhuman-core-x86_64-unknown-linux-gnu
|
||||
chmod +x app/src-tauri/binaries/openhuman-core-x86_64-unknown-linux-gnu
|
||||
|
||||
- name: Build Tauri app
|
||||
working-directory: app
|
||||
run: |
|
||||
TAURI_CONFIG_OVERRIDE='{"plugins":{"updater":{"active":false}}}'
|
||||
yarn tauri build -c "$TAURI_CONFIG_OVERRIDE"
|
||||
yarn tauri build -c "$TAURI_CONFIG_OVERRIDE" --bundles deb
|
||||
env:
|
||||
NODE_ENV: production
|
||||
CARGO_PROFILE_RELEASE_OPT_LEVEL: "1"
|
||||
CARGO_PROFILE_RELEASE_CODEGEN_UNITS: "16"
|
||||
CARGO_PROFILE_RELEASE_LTO: "false"
|
||||
CARGO_PROFILE_RELEASE_STRIP: "true"
|
||||
CARGO_PROFILE_RELEASE_DEBUG: "false"
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
name: Build CI Docker Image
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- ".github/Dockerfile"
|
||||
- ".github/workflows/docker-ci-image.yml"
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
packages: write
|
||||
|
||||
jobs:
|
||||
build-image:
|
||||
name: Build and push CI image
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Log in to GHCR
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
registry: ghcr.io
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Build and push
|
||||
uses: docker/build-push-action@v6
|
||||
with:
|
||||
context: .github
|
||||
file: .github/Dockerfile
|
||||
push: true
|
||||
tags: |
|
||||
ghcr.io/tinyhumansai/openhuman_ci:latest
|
||||
ghcr.io/tinyhumansai/openhuman_ci:rust-1.93.0
|
||||
@@ -2,6 +2,7 @@ name: Installer Smoke
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
workflow_dispatch:
|
||||
|
||||
|
||||
@@ -375,6 +375,7 @@ jobs:
|
||||
MATRIX_TARGET: ${{ matrix.settings.target }}
|
||||
CORE_MANIFEST: ${{ steps.core-paths.outputs.core_manifest }}
|
||||
CORE_BIN_NAME: ${{ steps.core-paths.outputs.core_bin_name }}
|
||||
OPENHUMAN_SENTRY_DSN: ${{ vars.OPENHUMAN_SENTRY_DSN }}
|
||||
|
||||
- name: Stage sidecar for Tauri bundler
|
||||
shell: bash
|
||||
|
||||
+11
-67
@@ -2,6 +2,7 @@ name: Test
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
|
||||
permissions:
|
||||
@@ -58,39 +59,21 @@ jobs:
|
||||
rust-tests:
|
||||
name: Rust Tests + Quality
|
||||
runs-on: ubuntu-22.04
|
||||
container:
|
||||
image: ghcr.io/tinyhumansai/openhuman_ci:rust-1.93.0
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 1
|
||||
|
||||
- name: Install Rust (rust-toolchain.toml)
|
||||
uses: dtolnay/rust-toolchain@1.93.0
|
||||
- name: Cache Rust build artifacts
|
||||
uses: Swatinem/rust-cache@v2
|
||||
with:
|
||||
components: rustfmt, clippy
|
||||
targets: x86_64-unknown-linux-gnu
|
||||
|
||||
- name: Install Tauri build dependencies
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y libgtk-3-dev libwebkit2gtk-4.1-dev libappindicator3-dev librsvg2-dev patchelf
|
||||
|
||||
# Match build.yml: lockfile body only (skip workspace version header noise)
|
||||
- name: Cargo.lock fingerprint (deps only)
|
||||
id: cargo-lock-fingerprint
|
||||
shell: bash
|
||||
run: |
|
||||
echo "hash=$(tail -n +8 Cargo.lock | openssl dgst -sha256 | awk '{print $2}')" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Cache Cargo registry and git sources
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: |
|
||||
~/.cargo/registry
|
||||
~/.cargo/git
|
||||
key: ${{ runner.os }}-rust-test-cargo-${{ steps.cargo-lock-fingerprint.outputs.hash }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-rust-test-cargo-
|
||||
workspaces: |
|
||||
. -> target
|
||||
app/src-tauri -> target
|
||||
cache-on-failure: true
|
||||
|
||||
- name: Check formatting (cargo fmt)
|
||||
run: cargo fmt --all -- --check
|
||||
@@ -102,52 +85,13 @@ jobs:
|
||||
run: cargo test -p openhuman
|
||||
|
||||
- name: Build sidecar core binary
|
||||
run: cargo build --manifest-path Cargo.toml --release --target x86_64-unknown-linux-gnu --bin openhuman-core
|
||||
run: cargo build --profile ci --target x86_64-unknown-linux-gnu --bin openhuman-core
|
||||
|
||||
- name: Stage sidecar for Tauri shell tests
|
||||
run: |
|
||||
mkdir -p app/src-tauri/binaries
|
||||
cp target/x86_64-unknown-linux-gnu/release/openhuman-core app/src-tauri/binaries/openhuman-core-x86_64-unknown-linux-gnu
|
||||
cp target/x86_64-unknown-linux-gnu/ci/openhuman-core app/src-tauri/binaries/openhuman-core-x86_64-unknown-linux-gnu
|
||||
chmod +x app/src-tauri/binaries/openhuman-core-x86_64-unknown-linux-gnu
|
||||
|
||||
- name: Test Tauri shell (OpenHuman)
|
||||
run: cargo test --manifest-path app/src-tauri/Cargo.toml
|
||||
|
||||
# e2e-macos:
|
||||
# name: E2E (macOS / Appium)
|
||||
# runs-on: macos-latest
|
||||
# timeout-minutes: 90
|
||||
# steps:
|
||||
# - name: Checkout code
|
||||
# uses: actions/checkout@v4
|
||||
# with:
|
||||
# fetch-depth: 1
|
||||
# submodules: recursive
|
||||
|
||||
# - name: Setup Node.js 24.x
|
||||
# uses: actions/setup-node@v4
|
||||
# with:
|
||||
# node-version: 24.x
|
||||
# cache: "yarn"
|
||||
|
||||
# - name: Install Rust (rust-toolchain.toml)
|
||||
# uses: dtolnay/rust-toolchain@1.93.0
|
||||
|
||||
# - name: Install dependencies
|
||||
# run: yarn install --frozen-lockfile
|
||||
|
||||
# - name: Ensure .env exists for E2E build
|
||||
# run: |
|
||||
# touch .env
|
||||
# touch app/.env
|
||||
|
||||
# - name: Install Appium and mac2 driver
|
||||
# run: |
|
||||
# npm install -g appium
|
||||
# appium driver install mac2
|
||||
|
||||
# - name: Build E2E app bundle
|
||||
# run: yarn workspace openhuman-app test:e2e:build
|
||||
|
||||
# - name: Run all E2E flows
|
||||
# run: yarn workspace openhuman-app test:e2e:all:flows
|
||||
|
||||
@@ -2,6 +2,7 @@ name: Type Check
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
|
||||
permissions:
|
||||
@@ -61,37 +62,20 @@ jobs:
|
||||
rust-quality:
|
||||
name: Rust Quality (fmt + clippy)
|
||||
runs-on: ubuntu-22.04
|
||||
container:
|
||||
image: ghcr.io/tinyhumansai/openhuman_ci:rust-1.93.0
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 1
|
||||
|
||||
- name: Install Rust (rust-toolchain.toml)
|
||||
uses: dtolnay/rust-toolchain@1.93.0
|
||||
- name: Cache Rust build artifacts
|
||||
uses: Swatinem/rust-cache@v2
|
||||
with:
|
||||
components: rustfmt, clippy
|
||||
|
||||
- name: Install Tauri build dependencies
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y libgtk-3-dev libwebkit2gtk-4.1-dev libappindicator3-dev librsvg2-dev patchelf
|
||||
|
||||
- name: Cargo.lock fingerprint (deps only)
|
||||
id: cargo-lock-fingerprint
|
||||
shell: bash
|
||||
run: |
|
||||
echo "hash=$(tail -n +8 Cargo.lock | openssl dgst -sha256 | awk '{print $2}')" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Cache Cargo registry and git sources
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: |
|
||||
~/.cargo/registry
|
||||
~/.cargo/git
|
||||
key: ${{ runner.os }}-cargo-registry-${{ steps.cargo-lock-fingerprint.outputs.hash }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-cargo-registry-
|
||||
workspaces: |
|
||||
. -> target
|
||||
cache-on-failure: true
|
||||
|
||||
- name: Check formatting (cargo fmt)
|
||||
run: cargo fmt --all -- --check
|
||||
|
||||
Reference in New Issue
Block a user