scripts/profile/
Reproducible benchmarking scripts for the OpenHuman Rust core as an embedded
library (no RPC server), built around the library-profile and rss-bench
binaries (see src/bin/library_profile/main.rs). Full write-up:
docs/library-benchmarking.md. Prior
findings: docs/resource-profiling-session-2026-07-21.md.
Five driver scripts: library-bench.sh (per-scenario RSS/duration),
library-cpu.sh (samply), library-heap.sh (dhat), library-fleet.sh
(fleet-scale sweep + budget gate), and library-instances.sh (multi-process
instance sweep).
Scripts
library-bench.sh — RSS/duration benchmark
Builds library-profile + rss-bench, runs each scenario N times as a fresh
process, and aggregates median/min/max duration, settled RSS, retained delta,
and peak delta into summary.json + summary.md.
./scripts/profile/library-bench.sh # all 7 scenarios, default build, 5 repeats
./scripts/profile/library-bench.sh --slim --repeat 7 # slim (no-default-features) build
./scripts/profile/library-bench.sh --scenarios "long-agent,subagents" --turns 50 --warm
Results land in target/profile/rust-library/bench-<timestamp>/ (or --out DIR).
library-cpu.sh — CPU profile via samply
Wraps samply record around one scenario, isolated from persistence/timezone
noise by default (matching the documented cold-path CPU recipe).
./scripts/profile/library-cpu.sh subagents
./scripts/profile/library-cpu.sh long-agent -- OPENHUMAN_PROFILE_TURNS=50
samply load target/profile/rust-library/subagents-cpu.json.gz
library-heap.sh — live heap attribution via dhat
Builds the rss-bench-dhat variant and runs one scenario under dhat. RSS and
timing numbers from this build are perturbed by instrumentation; use it only
for allocation-site/retained-bytes attribution, not for RSS comparisons.
./scripts/profile/library-heap.sh memory-ingest
# open https://nnethercote.github.io/dh_view/dh_view.html and load
# target/profile/rust-library/dhat-memory-ingest.json
library-fleet.sh — fleet sweep + 2 GB / 2 vCPU budget gate
Builds library-profile + rss-bench, sweeps the fleet scenario (N
concurrent live agents with latency-realistic mock inference) across a list
of agent counts, aggregates medians per N, and gates on whether the
projected footprint at the target agent count fits the RAM budget.
./scripts/profile/library-fleet.sh --agents 100 --latency-ms 200
./scripts/profile/library-fleet.sh --agents "50,100,500" --target 1000 --budget-mib 2048
Results land in target/profile/rust-library/fleet-<timestamp>/ (or
--out DIR). Exits nonzero if any swept N reports fits: false (use
--no-gate to report only). See
docs/library-benchmarking.md
for the budget math.
library-instances.sh — multi-instance (many-processes) sweep
Spawns N independent library-profile processes (each a live instance held
alive via OPENHUMAN_PROFILE_HOLD_SECS), staggered on startup, and measures
per-process cost and box survivability — the opencompany "N independent
processes/containers" deployment model, as opposed to library-fleet.sh's
"N agents in one process" model. Samples aggregate sum-RSS + live count every
2s while instances hold, captures a vm_stat snapshot at peak (and a
best-effort footprint sample if that macOS tool is present), then
aggregates per swept N: launched/ok counts, median settled RSS per instance,
mean and peak aggregate sum-RSS, and — on Linux, where it's meaningful —
summed PSS.
./scripts/profile/library-instances.sh --instances "10,50" --hold-secs 30
./scripts/profile/library-instances.sh --instances "100,500" --max-instances 500 --gate
Results land in target/profile/rust-library/instances-<timestamp>/ (or
--out DIR). Refuses to spawn more than --max-instances (default 200)
without an explicit raise — see the script's --help for the RAM math. Exits
nonzero with --gate if any instance failed to complete cleanly (nonzero
exit or missing/invalid JSON); default is report-only. See
docs/library-benchmarking.md
for the fleet-vs-instances framing.
Quick start
# 1. Baseline RSS/duration across all scenarios
./scripts/profile/library-bench.sh
# 2. CPU attribution for the slowest/most interesting scenario
./scripts/profile/library-cpu.sh subagents
# 3. If a scenario's RSS looks off, drill into live heap
./scripts/profile/library-heap.sh subagents
All scripts require jq for JSON parsing/aggregation; library-cpu.sh also
requires samply (cargo install samply). Build commands use
GGML_NATIVE=OFF to work around the Apple Silicon whisper-rs/llama.cpp NEON
fp16 build issue.