openhuman/scripts/profile

scripts/profile/

Reproducible benchmarking scripts for the OpenHuman Rust core as an embedded library (no RPC server), built around the library-profile and rss-bench binaries (see src/bin/library_profile/main.rs). Full write-up: docs/library-benchmarking.md. Prior findings: docs/resource-profiling-session-2026-07-21.md.

Five driver scripts: library-bench.sh (per-scenario RSS/duration), library-cpu.sh (samply), library-heap.sh (dhat), library-fleet.sh (fleet-scale sweep + budget gate), and library-instances.sh (multi-process instance sweep).

Scripts

library-bench.sh — RSS/duration benchmark

Builds library-profile + rss-bench, runs each scenario N times as a fresh process, and aggregates median/min/max duration, settled RSS, retained delta, and peak delta into summary.json + summary.md.

./scripts/profile/library-bench.sh                              # all 7 scenarios, default build, 5 repeats
./scripts/profile/library-bench.sh --slim --repeat 7             # slim (no-default-features) build
./scripts/profile/library-bench.sh --scenarios "long-agent,subagents" --turns 50 --warm

Results land in target/profile/rust-library/bench-<timestamp>/ (or --out DIR).

library-cpu.sh — CPU profile via samply

Wraps samply record around one scenario, isolated from persistence/timezone noise by default (matching the documented cold-path CPU recipe).

./scripts/profile/library-cpu.sh subagents
./scripts/profile/library-cpu.sh long-agent -- OPENHUMAN_PROFILE_TURNS=50
samply load target/profile/rust-library/subagents-cpu.json.gz

library-heap.sh — live heap attribution via dhat

Builds the rss-bench-dhat variant and runs one scenario under dhat. RSS and timing numbers from this build are perturbed by instrumentation; use it only for allocation-site/retained-bytes attribution, not for RSS comparisons.

./scripts/profile/library-heap.sh memory-ingest
# open https://nnethercote.github.io/dh_view/dh_view.html and load
# target/profile/rust-library/dhat-memory-ingest.json

library-fleet.sh — fleet sweep + 2 GB / 2 vCPU budget gate

Builds library-profile + rss-bench, sweeps the fleet scenario (N concurrent live agents with latency-realistic mock inference) across a list of agent counts, aggregates medians per N, and gates on whether the projected footprint at the target agent count fits the RAM budget.

./scripts/profile/library-fleet.sh --agents 100 --latency-ms 200
./scripts/profile/library-fleet.sh --agents "50,100,500" --target 1000 --budget-mib 2048

Results land in target/profile/rust-library/fleet-<timestamp>/ (or --out DIR). Exits nonzero if any swept N reports fits: false (use --no-gate to report only). See docs/library-benchmarking.md for the budget math.

library-instances.sh — multi-instance (many-processes) sweep

Spawns N independent library-profile processes (each a live instance held alive via OPENHUMAN_PROFILE_HOLD_SECS), staggered on startup, and measures per-process cost and box survivability — the opencompany "N independent processes/containers" deployment model, as opposed to library-fleet.sh's "N agents in one process" model. Samples aggregate sum-RSS + live count every 2s while instances hold, captures a vm_stat snapshot at peak (and a best-effort footprint sample if that macOS tool is present), then aggregates per swept N: launched/ok counts, median settled RSS per instance, mean and peak aggregate sum-RSS, and — on Linux, where it's meaningful — summed PSS.

./scripts/profile/library-instances.sh --instances "10,50" --hold-secs 30
./scripts/profile/library-instances.sh --instances "100,500" --max-instances 500 --gate

Results land in target/profile/rust-library/instances-<timestamp>/ (or --out DIR). Refuses to spawn more than --max-instances (default 200) without an explicit raise — see the script's --help for the RAM math. Exits nonzero with --gate if any instance failed to complete cleanly (nonzero exit or missing/invalid JSON); default is report-only. See docs/library-benchmarking.md for the fleet-vs-instances framing.

library-pool-gate.sh — runtime pool regression gate (#5106)

Drives the skill-run scenario with K parallel skill runs and asserts the DoD: with the shared runtime pool ON, the process tree grows by ~one pooled worker, not K interpreters. The scenario hard-asserts tree.child_count <= max_workers for K > 1 (nonzero exit); this script runs it and reports a pooled-vs-unpooled comparison. A regression that reintroduces per-run forking fails the gate.

./scripts/profile/library-pool-gate.sh                     # K=8, max_workers=1
./scripts/profile/library-pool-gate.sh --concurrency 16 --skip-build

Exits 0 = pass, 1 = regression. SKIPs (exit 0) when no system node is on PATH (the scenario must never download an interpreter), so node-less CI runners don't false-fail.

Quick start

# 1. Baseline RSS/duration across all scenarios
./scripts/profile/library-bench.sh

# 2. CPU attribution for the slowest/most interesting scenario
./scripts/profile/library-cpu.sh subagents

# 3. If a scenario's RSS looks off, drill into live heap
./scripts/profile/library-heap.sh subagents

All scripts require jq for JSON parsing/aggregation; library-cpu.sh also requires samply (cargo install samply). Build commands use GGML_NATIVE=OFF to work around the Apple Silicon whisper-rs/llama.cpp NEON fp16 build issue.