From d58312047dad364134df288dbdd656565bacf866 Mon Sep 17 00:00:00 2001 From: Steven Enamakel <31011319+senamakel@users.noreply.github.com> Date: Sat, 9 May 2026 20:20:14 -0700 Subject: [PATCH] Document building the Rust core from scratch (#1454) --- AGENTS.md | 12 +- gitbooks/SUMMARY.md | 1 + gitbooks/developing/README.md | 7 +- gitbooks/developing/architecture.md | 6 +- .../developing/architecture/tauri-shell.md | 16 +- gitbooks/developing/building-rust-core.md | 166 ++++++++++++++++++ gitbooks/developing/getting-set-up.md | 6 +- 7 files changed, 192 insertions(+), 22 deletions(-) create mode 100644 gitbooks/developing/building-rust-core.md diff --git a/AGENTS.md b/AGENTS.md index 61a7df57b..f8bd1d5fc 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -11,9 +11,9 @@ This file orients contributors and coding agents. Authoritative narrative archit | Path | Role | | ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **`app/`** | Yarn workspace **`openhuman-app`**: Vite + React (`app/src/`), Tauri desktop host (`app/src-tauri/`), Vitest tests | -| **Repo root `src/`** | Rust library **`openhuman_core`** and **`openhuman`** CLI binary entrypoint (`src/main.rs`) — `core_server`, `openhuman::*` domains, skills runtime (QuickJS / `rquickjs`), MCP routing in the core process | +| **Repo root `src/`** | Rust library **`openhuman_core`** and **`openhuman-core`** CLI binary entrypoint (`src/main.rs`) — `core_server`, `openhuman::*` domains, skills runtime (QuickJS / `rquickjs`), MCP routing in the core process | | **Skills registry** | **[`tinyhumansai/openhuman-skills`](https://github.com/tinyhumansai/openhuman-skills)** on GitHub — canonical skill packages and TS build; not vendored in this tree (see blurb below). | -| **`Cargo.toml`** (root) | Core crate; `cargo build --bin openhuman` produces the sidecar the UI stages via `app`’s `core:stage` | +| **`Cargo.toml`** (root) | Core crate; `cargo build --bin openhuman-core` produces the sidecar the UI stages via `app`’s `core:stage` | | **`docs/`** | Architecture and deep-internal references | | **`gitbooks/developing/`** | Public contributor docs — frontend, Tauri shell, testing, release, skills | @@ -27,7 +27,7 @@ Commands in documentation assume the **repo root** unless noted: `pnpm dev` runs - **Shipped product**: desktop — Windows, macOS, Linux (see [`gitbooks/developing/architecture.md`](gitbooks/developing/architecture.md) “Platform reach”). - **Tauri host** (`app/src-tauri`): **desktop-only** (`compile_error!` for non-desktop targets). Do not add Android/iOS branches inside `app/src-tauri`. -- **Core binary** (`openhuman`): spawned/staged as a **sidecar**; the Web UI talks to it over HTTP (`core_rpc_relay` + `core_rpc` client), not by re-implementing domain logic in the shell. +- **Core binary** (`openhuman-core`): spawned/staged as a **sidecar**; the Web UI talks to it over HTTP (`core_rpc_relay` + `core_rpc` client), not by re-implementing domain logic in the shell. **Where logic lives** @@ -64,7 +64,7 @@ pnpm workspace openhuman-app skills:watch # Rust — core library + CLI (repo root) cargo check --manifest-path Cargo.toml -cargo build --manifest-path Cargo.toml --bin openhuman +cargo build --manifest-path Cargo.toml --bin openhuman-core # Rust — Tauri shell only cargo check --manifest-path app/src-tauri/Cargo.toml @@ -290,7 +290,7 @@ Bundled prompts live under **`src/openhuman/agent/prompts/`** at the **repositor ## Tauri shell (`app/src-tauri/`) -Thin desktop host: window management, daemon health bridging, **core process lifecycle** (`core_process`, `CoreProcessHandle`), and **JSON-RPC relay** to the **`openhuman`** sidecar (`core_rpc_relay`, `core_rpc`). +Thin desktop host: window management, daemon health bridging, **core process lifecycle** (`core_process`, `CoreProcessHandle`), and **JSON-RPC relay** to the **`openhuman-core`** sidecar (`core_rpc_relay`, `core_rpc`). Registered IPC commands (see [`gitbooks/developing/tauri-shell.md`](gitbooks/developing/tauri-shell.md)) include **`greet`**, **`write_ai_config_file`**, **`ai_get_config`**, **`ai_refresh_config`**, **`core_rpc_relay`**, **window** commands, and **OpenHuman service / daemon host** helpers (`openhuman_*`). @@ -524,7 +524,7 @@ Follow this order so behavior is **specified**, **proven in Rust**, **proven ove - **macOS deep links**: Often require a built **`.app`** bundle; not only `tauri dev`. See [`docs/telegram-login-desktop.md`](docs/telegram-login-desktop.md) if applicable. - **`window.__TAURI__`**: Not assumed at module load; guard Tauri usage accordingly. -- **Core sidecar**: Must be staged/built so `core_rpc` can reach the `openhuman` binary (see `scripts/stage-core-sidecar.mjs`). +- **Core sidecar**: Must be staged/built so `core_rpc` can reach the `openhuman-core` binary (see `scripts/stage-core-sidecar.mjs`). --- diff --git a/gitbooks/SUMMARY.md b/gitbooks/SUMMARY.md index f9b171b36..2c66d0ae2 100644 --- a/gitbooks/SUMMARY.md +++ b/gitbooks/SUMMARY.md @@ -37,6 +37,7 @@ * [Overview](developing/README.md) * [Getting Set Up](developing/getting-set-up.md) +* [Building the Rust Core](developing/building-rust-core.md) * [Testing Strategy](developing/testing-strategy.md) * [E2E Testing](developing/e2e-testing.md) * [Release Policy](developing/release-policy.md) diff --git a/gitbooks/developing/README.md b/gitbooks/developing/README.md index 4f1faa465..8907154e5 100644 --- a/gitbooks/developing/README.md +++ b/gitbooks/developing/README.md @@ -16,7 +16,7 @@ If you just want to use the app, head to [Getting Started](../overview/getting-s | Path | What's there | | ----------- | ----------------------------------------------------------------------------------------------------------------- | | `app/` | pnpm workspace `openhuman-app`. Vite + React frontend (`app/src/`) and the Tauri desktop host (`app/src-tauri/`). | -| `src/` | Rust crate `openhuman_core` and the `openhuman` CLI binary. Domains, JSON-RPC, MCP routing. | +| `src/` | Rust crate `openhuman_core` and the `openhuman-core` CLI binary. Domains, JSON-RPC, MCP routing. | | `gitbooks/` | This site (the public-facing docs). | | `docs/` | Older deep references not yet migrated to GitBook (memory pipeline diagrams, agent flows, etc.). | @@ -29,8 +29,9 @@ If you just want to use the app, head to [Getting Started](../overview/getting-s If it's your first time pulling the repo: 1. [**Getting Set Up**](getting-set-up.md). Toolchain, dependencies, the vendored Tauri CLI, sidecar staging - everything `pnpm dev` needs to actually start. -2. [**Architecture**](architecture.md). How the desktop app, the Rust core sidecar, the JSON-RPC bridge, and the dual sockets fit together. Read this before you make non-trivial changes. -3. [**Frontend**](architecture/frontend.md) and [**Tauri Shell**](architecture/tauri-shell.md). The React app and the desktop host that wraps it. +2. [**Building the Rust Core**](building-rust-core.md). Fresh-machine setup for the repo-root Rust crate only: pinned toolchain, OS packages, and exact `cargo` commands. +3. [**Architecture**](architecture.md). How the desktop app, the Rust core sidecar, the JSON-RPC bridge, and the dual sockets fit together. Read this before you make non-trivial changes. +4. [**Frontend**](architecture/frontend.md) and [**Tauri Shell**](architecture/tauri-shell.md). The React app and the desktop host that wraps it. *** diff --git a/gitbooks/developing/architecture.md b/gitbooks/developing/architecture.md index f30aa77f0..5997b5150 100644 --- a/gitbooks/developing/architecture.md +++ b/gitbooks/developing/architecture.md @@ -16,12 +16,12 @@ OpenHuman is a cross-platform communication and automation platform purpose-buil | Path | Contents | | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | **`app/`** | Yarn workspace **`openhuman-app`**: Vite/React UI (`app/src/`), Tauri shell (`app/src-tauri/`), Vitest tests | -| **Repo root `src/`** | Rust **`openhuman_core`** library + **`openhuman`** CLI binary - `core_server`, JSON-RPC, QuickJS skills runtime (`src/openhuman/skills/`), channels, memory, etc. | -| **`Cargo.toml`** (root) | Builds `openhuman` binary (`cargo build --bin openhuman`) staged into `app/src-tauri/binaries/` for the desktop bundle | +| **Repo root `src/`** | Rust **`openhuman_core`** library + **`openhuman-core`** CLI binary - `core_server`, JSON-RPC, QuickJS skills runtime (`src/openhuman/skills/`), channels, memory, etc. | +| **`Cargo.toml`** (root) | Builds the `openhuman-core` binary (`cargo build --bin openhuman-core`) staged into `app/src-tauri/binaries/` for the desktop bundle | | **`skills/`** | Skill packages consumed by the runtime | | **`docs/`** | This book + per-tree guides (`docs/src/`, `docs/src-tauri/`) | -The desktop app **WebView** loads the UI from `app/`; heavy RPC and skills run in the **`openhuman`** process, reachable over HTTP from the Tauri host (`core_rpc_relay`). +The desktop app **WebView** loads the UI from `app/`; heavy RPC and skills run in the **`openhuman-core`** process, reachable over HTTP from the Tauri host (`core_rpc_relay`). --- diff --git a/gitbooks/developing/architecture/tauri-shell.md b/gitbooks/developing/architecture/tauri-shell.md index 48429dffd..c0d2ddd3d 100644 --- a/gitbooks/developing/architecture/tauri-shell.md +++ b/gitbooks/developing/architecture/tauri-shell.md @@ -5,19 +5,19 @@ icon: desktop # Tauri shell (`app/src-tauri/`) -The desktop host for OpenHuman: Tauri v2 + WebView, IPC commands, window management, and bridging to the `openhuman` Rust sidecar (core JSON-RPC). It does **not** duplicate the full domain stack, that lives in the repo-root Rust crate (`openhuman_core`, `src/bin/openhuman.rs`). +The desktop host for OpenHuman: Tauri v2 + WebView, IPC commands, window management, and bridging to the `openhuman-core` Rust sidecar (core JSON-RPC). It does **not** duplicate the full domain stack; that lives in the repo-root Rust crate (`openhuman_core`, `src/main.rs`). ## Responsibilities 1. **Web UI**. Load the Vite build from `app/dist` (or dev server on port 1420). 2. **IPC**. Expose a small, explicit set of Tauri commands (see [Commands](#commands)). -3. **Core lifecycle**. Ensure the `openhuman` binary is running (child process and/or service) and proxy JSON-RPC via `core_rpc_relay`. +3. **Core lifecycle**. Ensure the `openhuman-core` binary is running (child process and/or service) and proxy JSON-RPC via `core_rpc_relay`. 4. **AI prompts on disk**. Resolve bundled `src/openhuman/agent/prompts` from resources / dev cwd for `ai_get_config` / `write_ai_config_file`. 5. **Window + tray**. Desktop window behavior and system tray (see `lib.rs`). ## Building the sidecar -`app/package.json` `core:stage` runs `scripts/stage-core-sidecar.mjs`, which `cargo build --bin openhuman` at the repo root and copies the binary into `app/src-tauri/binaries/` for Tauri `externalBin`. +`app/package.json` `core:stage` runs `scripts/stage-core-sidecar.mjs`, which runs `cargo build --bin openhuman-core` at the repo root and copies the binary into `app/src-tauri/binaries/` for Tauri `externalBin`. ## Stuck process recovery @@ -32,7 +32,7 @@ Startup recovery skips when `OPENHUMAN_CORE_REUSE_EXISTING=1` is set (so manual ### Overview -The **`app/src-tauri`** crate (Rust package **`OpenHuman`**, binary **`OpenHuman`**) is a **desktop-only** host. It embeds the React UI, registers plugins (deep link, opener, OS, notifications, autostart, updater), manages the main window and tray, and **relays JSON-RPC** to the separately built **`openhuman`** core binary. +The **`app/src-tauri`** crate (Rust package **`OpenHuman`**, binary **`OpenHuman`**) is a **desktop-only** host. It embeds the React UI, registers plugins (deep link, opener, OS, notifications, autostart, updater), manages the main window and tray, and **relays JSON-RPC** to the separately built **`openhuman-core`** binary. Non-desktop targets fail at compile time (`compile_error!` in `lib.rs`). @@ -107,7 +107,7 @@ All commands are registered in **`app/src-tauri/src/lib.rs`** inside `tauri::gen | Command | Purpose | | ---------------- | -------------------------------------------------------------------------------------------------------------- | -| `core_rpc_relay` | Body: `{ method, params?, serviceManaged? }` → forwards to local **`openhuman`** HTTP JSON-RPC (`core_rpc.rs`) | +| `core_rpc_relay` | Body: `{ method, params?, serviceManaged? }` → forwards to local **`openhuman-core`** HTTP JSON-RPC (`core_rpc.rs`) | Use **`app/src/services/coreRpcClient.ts`** (`callCoreRpc`) from the frontend. @@ -175,11 +175,11 @@ _See `app/src-tauri/src/lib.rs` for the authoritative list._ ## Core bridge & helpers (`app/src-tauri`) -This document replaces the old “SessionService / SocketService” split. The Tauri crate **does not** embed a duplicate Socket.io server or Telegram client; instead it focuses on **process management** and **HTTP JSON-RPC** to the **`openhuman`** binary. +This document replaces the old “SessionService / SocketService” split. The Tauri crate **does not** embed a duplicate Socket.io server or Telegram client; instead it focuses on **process management** and **HTTP JSON-RPC** to the **`openhuman-core`** binary. ### `CoreProcessHandle` (`core_process.rs`) -- Resolves the **`openhuman`** executable (staged under `binaries/` or `PATH` / dev layout). +- Resolves the **`openhuman-core`** executable (staged under `binaries/` or `PATH` / dev layout). - Starts or attaches to the core process and exposes its RPC URL (`OPENHUMAN_CORE_RPC_URL`). - Used during app setup in `lib.rs` (`app.manage(core_handle)`). @@ -207,5 +207,3 @@ This document replaces the old “SessionService / SocketService” split. The T Not in `src-tauri`, but **pairs** with the shell: the React app listens for Tauri events that mirror socket activity when using the Rust-side client. See `app/src/utils/tauriSocket.ts` and the [Frontend Services](frontend.md#services-layer) chapter. --- - - diff --git a/gitbooks/developing/building-rust-core.md b/gitbooks/developing/building-rust-core.md new file mode 100644 index 000000000..3787c0c7f --- /dev/null +++ b/gitbooks/developing/building-rust-core.md @@ -0,0 +1,166 @@ +--- +description: Build the Rust core from scratch on a fresh machine. +icon: terminal +--- + +# Building the Rust Core + +This page is the contributor-facing reference for compiling the Rust core on a fresh machine. + +It covers the **repo-root crate only**: + +- Cargo package: `openhuman` +- Binary: `openhuman-core` +- Library: `openhuman_core` + +If you want the full desktop app (`pnpm dev`, Tauri, CEF, frontend tooling), use [Getting Set Up](getting-set-up.md). That path has extra JavaScript, submodule, and desktop-runtime requirements that are **not** needed for a core-only `cargo` workflow. + +## 1. Install the pinned Rust toolchain + +The repository pins Rust in [`rust-toolchain.toml`](../../rust-toolchain.toml): + +- Channel: `1.93.0` +- Components: `rustfmt`, `clippy` + +Recommended install: + +```bash +rustup toolchain install 1.93.0 --component rustfmt --component clippy +rustup default 1.93.0 +``` + +You can also let `cargo` auto-install from `rust-toolchain.toml` after `rustup` itself is installed. + +## 2. Clone the repo + +Core-only work: + +```bash +git clone https://github.com/tinyhumansai/openhuman.git +cd openhuman +``` + +That is enough for the root crate. + +Desktop/Tauri work is different: + +- `app/src-tauri/vendor/` submodules are only needed when building the desktop shell or CEF-aware Tauri tooling. +- For that flow, follow [Getting Set Up](getting-set-up.md) and run `git submodule update --init --recursive`. + +## 3. Build commands + +From the repository root: + +```bash +# Fast dependency + type check +cargo check --manifest-path Cargo.toml + +# Debug build of the actual CLI / RPC binary +cargo build --manifest-path Cargo.toml --bin openhuman-core + +# Release build +cargo build --manifest-path Cargo.toml --release --bin openhuman-core + +# Rust tests +cargo test --manifest-path Cargo.toml +``` + +Notes: + +- The **package** name is `openhuman`, but the runnable binary is **`openhuman-core`**. +- If you prefer package-oriented cargo commands for packager scripts, use `-p openhuman`. +- The built binary lands at `target/debug/openhuman-core` or `target/release/openhuman-core`. + +## 4. macOS prerequisites + +Install: + +- Xcode Command Line Tools: `xcode-select --install` + +Why: + +- `whisper-rs` compiles native code during the build. +- On macOS this crate is built with the `metal` feature enabled in [`Cargo.toml`](../../Cargo.toml), so Apple toolchains and SDK headers need to be present. + +After Xcode CLT is installed, the core should build with the cargo commands above. + +## 5. Linux prerequisites + +### Core-only package set + +Install these packages before running `cargo` on a fresh Ubuntu/Debian machine: + +```bash +sudo apt-get update +sudo apt-get install -y \ + build-essential cmake pkg-config clang libssl-dev libclang-dev \ + libasound2-dev libxi-dev libxtst-dev libxdo-dev libudev-dev \ + libstdc++-14-dev +``` + +Why these matter: + +- `build-essential`, `cmake`, `pkg-config`: native builds used by transitive Rust dependencies. +- `clang`, `libclang-dev`: bindgen / C and C++ compilation paths used by native crates. +- `libssl-dev`: OpenSSL headers needed by some networking dependencies. +- `libasound2-dev`, `libxi-dev`, `libxtst-dev`, `libxdo-dev`, `libudev-dev`: required by audio/input/device crates pulled into the core build. + +### `whisper-rs` + `clang` note + +`whisper-rs-sys` can fail under `clang` with: + +```text +fatal error: 'array' file not found +``` + +This is why the docs call out `libstdc++-14-dev`: `clang` may pick GCC 14 C++ headers on Ubuntu runners. + +If your distro layout still leaves `libstdc++.so` unresolved for the build, use the same workaround documented in [`AGENTS.md`](../../AGENTS.md): + +```bash +sudo ln -sf /usr/lib/gcc/x86_64-linux-gnu/13/libstdc++.so /usr/lib/x86_64-linux-gnu/libstdc++.so +``` + +Adjust the GCC version in that path if your machine installs a different one. + +### Linux desktop/Tauri package set + +If you are building the desktop shell instead of the core-only crate, install the broader Ubuntu dependency set mirrored from [`.github/workflows/build-desktop.yml`](../../.github/workflows/build-desktop.yml): + +```bash +sudo apt-get update +sudo apt-get install -y \ + libgtk-3-dev libwebkit2gtk-4.1-dev libayatana-appindicator3-dev librsvg2-dev \ + patchelf cmake libasound2-dev libxdo-dev libxtst-dev libx11-dev libxi-dev \ + libevdev-dev libssl-dev libclang-dev \ + libnss3 libnspr4 libatk1.0-0 libatk-bridge2.0-0 libcups2 libdrm2 \ + libxkbcommon0 libxcomposite1 libxdamage1 libxfixes3 libxrandr2 \ + libgbm1 libpango-1.0-0 libcairo2 libatspi2.0-0 libxshmfence1 libu2f-udev +``` + +Use that desktop list only when you need `app/src-tauri/`; for root-crate work, the smaller core-only list above is the relevant baseline. + +## 6. Windows prerequisites + +Install: + +- Rust via `rustup` +- Visual Studio Build Tools 2022 or Visual Studio with the **Desktop development with C++** workload +- The MSVC target used by CI and release builds: `x86_64-pc-windows-msvc` + +Recommended commands after the Microsoft toolchain is installed: + +```powershell +rustup toolchain install 1.93.0 --component rustfmt --component clippy +rustup target add x86_64-pc-windows-msvc +cargo build --manifest-path Cargo.toml --bin openhuman-core +``` + +Windows note: + +- The repo patches `whisper-rs-sys` to force the static MSVC CRT and avoid the `LNK2038` / `LNK1169` mismatch called out in [`Cargo.toml`](../../Cargo.toml). Use the MSVC toolchain, not MinGW. + +## 7. Related paths + +- [Getting Set Up](getting-set-up.md): full desktop contributor setup with `pnpm`, Tauri, submodules, and sidecar staging. +- [OpenHuman Architecture](architecture/README.md): where the core fits into the desktop app and RPC flow. diff --git a/gitbooks/developing/getting-set-up.md b/gitbooks/developing/getting-set-up.md index 8fdd576b2..76f547823 100644 --- a/gitbooks/developing/getting-set-up.md +++ b/gitbooks/developing/getting-set-up.md @@ -5,6 +5,10 @@ icon: wrench # Building & Installing OpenHuman +This guide covers the full desktop/source install path and release installers. + +If you only need the repo-root Rust crate on a fresh machine, use [Building the Rust Core](building-rust-core.md). That page documents the pinned Rust toolchain, OS package prerequisites, and the exact `cargo` commands for `openhuman-core`. + This guide covers two paths: 1. Build and compile OpenHuman from source @@ -29,7 +33,7 @@ cd openhuman pnpm install # 3) Build Rust core binary -cargo build --manifest-path Cargo.toml --bin openhuman +cargo build --manifest-path Cargo.toml --bin openhuman-core # 4) Stage core sidecar for the desktop app cd app