* build(skills): add serde_yaml for SKILL.md frontmatter (#681)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* feat(skills): parse SKILL.md frontmatter and add multi-path discovery (#681)
Extends the skills module with agentskills.io-style discovery:
* Parses YAML frontmatter in SKILL.md (name, description, version,
author, tags, allowed-tools, license) via serde_yaml, with a
catch-all extras map for forward-compatible keys.
* Adds SkillScope (User / Project / Legacy) and discover_skills(),
which scans ~/.openhuman/skills, ~/.agents/skills, <ws>/.openhuman/
skills, <ws>/.agents/skills and the legacy <ws>/skills directory.
* Gates project-scope loading behind an explicit
<ws>/.openhuman/trust marker (is_workspace_trusted helper).
* Resolves name collisions with project > user > legacy precedence
and surfaces shadowing as per-skill warnings.
* Inventories bundled resources under scripts/, references/, assets/.
* Keeps the existing load_skills(workspace_dir) API as a
backwards-compatible wrapper for existing callers.
* Preserves legacy skill.json fallback (marked legacy = true).
* Lenient validation: missing / mismatched / oversized name or
description produce warnings instead of errors.
* Adds 12 unit tests covering frontmatter parsing, trust gating,
collision shadowing, lenient fallbacks, legacy JSON and resource
inventory. One existing prompt test updated to use
..Default::default() for the expanded struct.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(skills): align frontmatter with agentskills.io spec (#681)
Per the agentskills.io SKILL.md spec, only name, description, license,
compatibility, metadata, and allowed-tools are valid top-level keys.
Our SkillFrontmatter had version, author, and tags as top-level fields,
which drifted from the spec and would silently swallow mis-shaped data
for skills authored against the canonical schema.
Demote version/author/tags into the metadata map. Non-spec top-level
keys still parse via #[serde(flatten)] into extra, and when present
there we emit a migration warning and still populate the derived Skill
fields so existing skills keep working. Add compatibility as an
optional string alongside license.
New tests cover the spec-compliant metadata shape, the deprecated
top-level fallback path, and verify that the full spec frontmatter
parses without leaking into extras.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* build(runtime): add tar+xz2+zip for node runtime extraction (#681)
Node.js distributions ship as .tar.xz on Unix and .zip on Windows. The
upcoming Node runtime bootstrap extracts these in pure Rust; xz2 is
built with the `static` feature so liblzma is bundled and not a system
dependency. zip is pinned to default-features = false + deflate to
avoid pulling in bzip2/zstd/aes which we don't need for Node archives.
Deps only — no behavior change.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* feat(config): add NodeConfig with env overrides (#681)
Introduce managed Node.js runtime configuration so skills that require
`node`/`npm` (agentskills.io packages with build steps) can be wired in
behind a single config switch. Fields:
- `node.enabled` — master switch (default true)
- `node.version` — pinned release (default `v22.11.0` LTS)
- `node.cache_dir` — absolute path for managed distributions; empty = use
workspace default
- `node.prefer_system` — reuse matching system `node` when found (default
true); set false for reproducible CI / airgapped deploys
Env overrides land in load.rs alongside the existing LOCAL_AI_TIER block:
- OPENHUMAN_NODE_ENABLED
- OPENHUMAN_NODE_VERSION
- OPENHUMAN_NODE_CACHE_DIR
- OPENHUMAN_NODE_PREFER_SYSTEM
No resolver / downloader yet — subsequent commits in the #681 series
consume this config to bootstrap the runtime.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(node_runtime): detect compatible system node on PATH (#681)
Introduce the `openhuman::node_runtime` module and its first piece: a
synchronous resolver that walks `PATH`, probes `node --version`, and
returns a `SystemNode` when the host toolchain major-version matches the
configured target.
Why resolve first: a successful probe lets the bootstrap skip a
~60 MB download per managed install. Matching is intentionally loose on
the patch level (Node LTS lines are ABI-stable and skills pin their own
deps via package-lock.json); operators needing strict pinning can set
`node.prefer_system = false`.
Tracing is verbose by design — resolver decisions gate the download
path, so operators need a clear breadcrumb trail at `debug`/`info`.
Includes unit tests for `parse_node_version` covering `v` prefix,
whitespace, major-only, and malformed inputs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(node_runtime): SHASUMS256-verified distribution downloader (#681)
Add the second piece of the managed Node.js runtime: streaming download
of OS/arch-specific prebuilt archives from nodejs.org, gated by a
SHA-256 match against the release's signed `SHASUMS256.txt`.
Key contracts:
- `NodeDistribution::for_host(version)` picks the right archive for the
current OS/arch tuple. Supported matrix:
* darwin-{arm64,x64}.tar.xz
* linux-{arm64,x64,armv7l}.tar.xz
* win-{arm64,x64}.zip
Unsupported hosts surface a clear error so the caller can flip
`node.enabled = false` or point at a pre-installed toolchain.
- `fetch_shasums(client, version)` parses `SHASUMS256.txt` into a
filename -> hex digest map. Tolerant of trailing / signature blocks.
- `download_distribution(...)` streams chunks into the target path,
computes SHA-256 on the fly, and wipes the partial file if the
digest does not match. Integrity check is mandatory — skills will
execute untrusted code inside the resolved runtime.
Verbose tracing at `info` and `debug` follows the repo debug-logging
rule; operators can grep `[node_runtime::downloader]` to trace every
GET, chunk boundary (via total-bytes log), and hash decision.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(node_runtime): archive extraction + atomic install (#681)
Extract downloaded Node.js distributions and move them into the final
cache path without leaving the reader observing a half-populated
directory.
- `extract_distribution(archive, extract_root, is_zip)` — wraps the
synchronous `tar`/`xz2`/`zip` crates in `spawn_blocking` and returns
the single top-level folder produced by the archive (`node-vX.Y.Z-
<os>-<arch>/`). `set_preserve_permissions(true)` + `set_overwrite(
true)` on the tar side keeps the `node` binary's `+x` bit. The zip
path restores Unix mode bits via `unix_mode()` so cross-OS builds
still land correct permissions.
- `atomic_install(staged, final_dest)` — renames a staged directory
into place via a single `rename(2)`, moving any pre-existing install
to a `.old-<pid>` sibling first and cleaning it up on success. This
gives concurrent readers a "before" or "after" view, never a
partial one.
- Unsafe zip paths (`enclosed_name()` rejection) are logged and
skipped rather than traversed — defence against malicious archives,
even though the digest check in the downloader already rules out
tampered official releases.
No new tests here: the logic leans on upstream tar/zip crates, and
meaningful end-to-end coverage requires a real archive on disk — that
comes in the integration-test commit later in this series.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(node_runtime): bootstrap mutex + cache + bin path helper (#681)
Stitch resolver, downloader, and extractor into a single idempotent
entry point callers use at startup (or lazily before the first
`node_exec`/`npm_exec` call).
`NodeBootstrap::resolve()` contract:
1. Return cached `ResolvedNode` if a previous call already succeeded.
2. Bail when `node.enabled = false`.
3. If `node.prefer_system`, probe the host PATH. Matching major
version wins — return a `System`-sourced `ResolvedNode`.
4. Otherwise compute the install path
(`{cache_dir}/node-v{version}-{os}-{arch}/`). If it already contains
valid bins, reuse it. This makes the bootstrap ~free across
restarts once a managed install lands.
5. Else fetch `SHASUMS256.txt`, locate the expected digest for our
archive, stream the download, extract into a `.stage-<pid>` scratch
dir, `atomic_install` the top-level folder into the final path, and
remove scratch + archive to reclaim disk.
Concurrency is handled by a `tokio::sync::Mutex<Option<ResolvedNode>>`
— parallel callers queue behind the first one; the winner memoises,
the losers pick up the cached result. No race can produce two
concurrent downloads of the same archive.
Platform-specific bin layout lives in `managed_bin_dir` / `build_
resolved`:
- Unix: `<install>/bin/{node,npm}`
- Windows: `<install>/{node.exe,npm.cmd}`
This closes the resolver/downloader/cache layer promised in the #681
checkpoint. Tools (`node_exec`, `npm_exec`) and shell-PATH injection
layer on top in the next batch of commits.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(tools): node_exec runs JS via managed node runtime (#681)
Executes `inline_code` (via `node -e`) or a workspace-relative
`script_path` through the NodeBootstrap-resolved `node` binary. POSIX
single-quote quoting keeps user input inert; `env_clear` + allow-list
mirrors the shell tool so secrets never leak. 300s default timeout
(capped at 1800s), 1MB stdout/stderr caps. PATH is prepended with the
resolved bin dir so child processes see managed `node`/`npm`.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* feat(tools): npm_exec runs npm via managed node runtime (#681)
Thin npm CLI wrapper paired with node_exec. Subcommands go through
`is_sane_subcommand` (alphanumerics + `._-:` only) and a deny-list
(publish/adduser/login/token/…) blocks registry-mutation and auth
flows. `cwd` is resolved under the workspace — absolute paths and
`..` components are rejected. 600s default timeout (1800s ceiling),
1MB stdout/stderr caps. PATH prepended with managed bin dir so
npm's own node/corepack lookups hit the managed toolchain.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* feat(tools): shell prepends managed node bin to PATH when cached (#681)
Adds a non-blocking `NodeBootstrap::try_cached()` primitive that peeks the
memoised `ResolvedNode` without holding the async lock or triggering a
download. ShellTool gains an optional `node_bootstrap` field wired through
a new `with_node_bootstrap` constructor; when set, each shell invocation
consults `try_cached()` and, on a hit, prepends the managed bin dir to
the child PATH using the platform separator. Unrelated shell commands
stay byte-identical — no download is ever forced from the shell path.
Consequence: once `node_exec`/`npm_exec` have resolved the toolchain
once, skills that shell out to `node`/`npm`/`npx`/`corepack` transparently
pick up the managed install without any per-command coordination.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* feat(tools): register node_exec + npm_exec behind node.enabled (#681)
Wires the skills-oriented Node.js toolchain into the tool registry:
* Constructs one session-scoped `NodeBootstrap` in `all_tools_with_runtime`
when `root_config.node.enabled` is true — all three consumers (ShellTool,
NodeExecTool, NpmExecTool) share the same `Arc<NodeBootstrap>` so the
download/extract/install pipeline runs at most once per session and the
memoised `ResolvedNode` is reused across every shelling-out call.
* Swaps ShellTool to `with_node_bootstrap(...)` when the bootstrap exists
so shell PATH injection fires automatically once any node/npm tool has
resolved the runtime.
* Registers `node_exec` and `npm_exec` only when the flag is on — flipping
`node.enabled = false` cleanly removes both tools and falls back to the
legacy ShellTool construction.
* Adds two regression tests (`all_tools_registers_node_exec_when_node_enabled`
and `all_tools_excludes_node_exec_when_node_disabled`) so future refactors
can't silently drop the wiring.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* feat(agents): grant node_exec + npm_exec to code_executor + tool_maker (#681)
code_executor is the sandboxed developer sub-agent — it writes, runs, and
debugs code — and tool_maker is the narrow self-healing agent that polyfills
missing commands. Both now see the managed Node.js tools so skills and
polyfills can call into the resolved runtime directly rather than relying on
whatever `node`/`npm` happens to be on the host PATH.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* style(core): apply cargo fmt auto-fixes (#681)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* chore(cargo): sync Cargo.lock to OpenHuman v0.52.26 after rebase (#681)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(node_runtime): strip leading v/V defensively in resolve_from_system (#681)
CodeRabbit R1 flagged that build_resolved receives a raw version string from
SystemNode. detect_system_node already trims the prefix, but trim defensively
at the resolve_from_system boundary too so any future code path constructing
SystemNode with an un-normalised version won't emit "vvX.Y.Z".
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(node_runtime): propagate flush errors and clean up partial archives (#681)
CodeRabbit R2 flagged that download_distribution silently swallowed flush
errors via `.ok()` and left partial files on disk when any chunk/write/flush
step errored. Wrap the streaming loop in an async block returning Result<()>,
propagate flush errors, and on any failure delete the partial archive so a
retry starts clean and callers never see a half-written file.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(node_runtime): restore previous install on staged rename failure (#681)
CodeRabbit R3 flagged that atomic_install left the user with no Node runtime
on disk if the staged->final rename failed after a backup had been taken.
Wrap the rename in `if let Err`, restore the backup if present, log restore
failures separately (warning), and always return the original error so the
caller sees the true failure cause.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(node_runtime): enforce real 5s timeout on node --version probe (#681)
CodeRabbit R4 flagged that probe_node_version used Command::output() with no
real timeout — a broken shim or FUSE-backed binary could hang the bootstrap
forever. Add wait-timeout crate dep and rewrite the probe to spawn with
piped stdio, wait_timeout(5s), kill on timeout, and read stdout/stderr from
the piped handles after exit. Logs a warning when a probe times out so the
install can be diagnosed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(tools): reject script_path escapes in node_exec (#681)
CodeRabbit R5 flagged that node_exec joined user-supplied script_path onto
the workspace without rejecting `..` segments or absolute paths, letting a
prompt-injected `../../../etc/passwd` read arbitrary files via node. Add a
resolve_script_path helper mirroring npm_exec::resolve_cwd — reject empty,
absolute, parent-dir, and Windows-prefix components. Extra positional args
stay opaque (shell-quoted, not path-checked) and get an explanatory comment
at the call site. Unit tests cover each rejection case plus the relative-
subdir happy path.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor(skills): drop dead SKILL.md branch in legacy manifest loader (#681)
CodeRabbit N1 flagged dead code: load_from_legacy_manifest only runs when
load_skill_dir already determined SKILL.md is absent, so the fallback that
re-checked for SKILL.md and its helper read_skill_md_description could
never execute. Simplify to a direct description/location decision and
delete the dead helper.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor(config): extract parse_env_bool helper with warn on unknown values (#681)
CodeRabbit N2 flagged two near-identical boolean env-override match blocks
for OPENHUMAN_NODE_ENABLED and OPENHUMAN_NODE_PREFER_SYSTEM. Extract a
module-level parse_env_bool helper that accepts 1/true/yes/on and
0/false/no/off (case-insensitive) and emits a tracing::warn when the value
is unrecognised so silent mis-spellings don't invisibly leave the config
unchanged.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(node_runtime): detect corrupted cache missing npm (#723)
`probe_managed_install` now verifies the npm launcher exists before
reusing an extracted install. A download interrupted after `node` was
extracted but before `npm` would otherwise be cached forever and
`npm_exec` could never self-heal — now the corrupted cache forces a
fresh download via the normal resolve path.
Addresses CodeRabbit review feedback on PR #723.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(node_runtime): skip non-executable PATH shims when probing node (#723)
`which_node` previously returned the first matching filename on `PATH`
regardless of its execute bit. A non-executable `node` placeholder
earlier in `PATH` (e.g. an unprivileged shim left by a failed install)
would mask a valid later install and force the managed runtime download.
Now checks `file && (mode & 0o111 != 0)` on Unix to mirror shell `which`
behaviour. Windows remains unchanged — `.exe` suffix already encodes
executability for the loader.
Addresses CodeRabbit review feedback on PR #723.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(skills): deterministic entry order in scan_root (#723)
`read_dir` order is unspecified by the OS. When two sibling skill
directories declare the same logical `frontmatter.name` (which can
differ from the folder name), cross-scope/same-scope deduplication
downstream would pick a non-deterministic winner across runs.
Sort entries by on-disk directory name for a stable, reproducible
order so collision resolution is deterministic.
Addresses CodeRabbit review feedback on PR #723.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(skills): surface YAML parse errors as skill warnings (#723)
`parse_skill_md` previously swallowed the `serde_yaml` error via `log::warn!`
and fell back to an empty `SkillFrontmatter`, then the catalog reported a
generic "could not parse — exposing directory as placeholder". Skill
authors had no way to see the real cause without scraping logs.
Return parse-level diagnostics as a third tuple element. `load_from_skill_md`
merges them into the skill's user-visible `warnings`, so the catalog now
surfaces the actual YAML error (e.g. "frontmatter parse error: mapping
values are not allowed here at line 3 column 12").
Addresses CodeRabbit review feedback on PR #723.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(skills): skip symlinks when walking skill resources (#723)
`walk_files` used `is_dir()` / `is_file()` which transparently follow
symlinks. Two failure modes:
1. **Unbounded recursion** — a skill resource symlink that points back
at an ancestor (e.g. `resources/self -> resources/`) would cause
infinite traversal, eventually blowing the stack.
2. **Silent out-of-tree leakage** — a symlink pointing at `/`, `/etc`,
or another skill's directory would enumerate its contents into the
current skill's resource listing.
Switch to `entry.file_type()` and skip symlinks before descending.
Directories and regular files behave as before.
Addresses CodeRabbit review feedback on PR #723.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(node_runtime): default managed cache to user-owned OS cache (#723)
Default `cache_root()` to `dirs::cache_dir()/openhuman/node-runtime/` so a
repo cannot ship a checked-in `./node-runtime/` and have the bootstrap
reuse it as a trusted managed install. Explicit `config.cache_dir` still
wins; workspace-local falls back only when `dirs::cache_dir()` is
unavailable, and we emit a warning on that fallback.
As a second line of defence, `probe_managed_install()` now canonicalises
both the install dir and the cache root and refuses any install that
escapes the cache tree.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(node_runtime): require npm when accepting system node (#723)
On distros that package `node` and `npm` separately (Debian/Ubuntu,
Alpine `nodejs-current`, some NixOS setups) the host `node` can be
present without `npm`. `npm_exec` then fails every call because the
resolved `SystemNode` has no usable npm launcher.
Before returning `Some(SystemNode)`, locate `npm` on `PATH` and probe
`npm --version` through the same `wait_timeout` path as the node probe.
Either missing gate falls back to the managed download flow.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(skills): surface user-scope skills via load_skills (#723)
Existing production callers (`agent::harness::session::builder`,
`channels::runtime::startup`) reach the skill catalog via
`load_skills(workspace_dir)`. Previously this shim passed `None` for
the home directory, so skills installed under `~/.openhuman/skills/`
and `~/.agents/skills/` were silently dropped even after the
multi-scope discovery landed in `discover_skills`. Delegate to
`discover_skills_inner` with `dirs::home_dir()` so user-scope skills
reach the runtime; project-scope still wins on name collision.
Tests stay hermetic via a `load_skills_ws` helper that preserves the
old workspace-only semantics. A new `load_skills_surfaces_user_scope`
test drives a tempdir-as-home through `discover_skills` and asserts
the user-scope skill is returned with `SkillScope::User`.
Addresses CodeRabbit review comment 3116332244.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(skills): skip symlinked skill dirs during scan (#723)
`scan_root` previously accepted any entry where `path.is_dir()`
returned true. `is_dir()` dereferences symlinks, so a link from
`<skills-root>/foo -> /some/external/tree` would load as a
legitimate skill even though `walk_files` already rejects
symlinks deeper in the resource walker. Attacker-authored
symlinks in a skills root would therefore have one remaining
escape hatch.
Switch to `entry.file_type()` (non-dereferencing) and reject
both symlinked and non-directory entries at the top level.
Treat a failed `file_type()` probe as "not safe to traverse"
and skip.
A new `symlinked_skill_dirs_are_skipped` test (Unix-only, since
symlinks are the platform guarantee being tested) creates an
external skill tempdir, links it into the workspace skills root,
and asserts `load_skills_ws` returns empty.
Addresses CodeRabbit review comment 3116332252.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(skills): reject symlinked resource roots (#723)
`inventory_resources` gated each resource sub-root (`scripts/`,
`references/`, `assets/`) on `root.is_dir()`. `is_dir()` follows
symlinks, so a `scripts -> /etc` link inside a skill directory
would pass the check and `walk_files` would inventory the
external tree. Deeper symlinks inside the walk were already
rejected by `walk_files`, but the root-level check was the
missing layer.
Use `std::fs::symlink_metadata` for a non-dereferencing probe
and reject roots whose own `file_type().is_symlink()` returns
true. Treat any `symlink_metadata` error as a non-existent root
and skip.
A new `symlinked_resource_roots_are_rejected` test (Unix-only)
links a skill's `assets/` sub-root to an external tempdir with a
file inside and asserts `inventory_resources` returns empty.
Addresses CodeRabbit review comment 3116332260.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* feat(tokenjuice): implement core functionality for terminal-output compaction engine
- Introduced the `tokenjuice` module, which includes the classification and reduction of tool outputs based on JSON-configured rules.
- Added new dependencies for Unicode handling: `unicode-segmentation` and `unicode-width`.
- Implemented the `classify` module to match tool execution inputs against predefined rules, enhancing the ability to process and summarize terminal outputs.
- Created a comprehensive set of types and utilities for managing tool execution inputs and classification results.
- Established a built-in rule set for common tools, improving the initial setup and usability of the `tokenjuice` engine.
- Enhanced testing framework with integration tests to ensure the accuracy of output compaction and classification.
These changes lay the groundwork for a robust terminal-output management system, facilitating better interaction with various tools and improving overall user experience.
* feat(tokenjuice): implement tokenjuice module for terminal output compaction
- Introduced the `tokenjuice` module, which includes functionality for classifying and reducing terminal output based on JSON-configured rules.
- Added new dependencies: `unicode-segmentation` and `unicode-width` to support text processing.
- Created a new `classify.rs` file for rule classification logic, including matching helpers and scoring functions.
- Implemented a `reduce.rs` file to handle the main reduction pipeline and text normalization.
- Established a structured approach for loading and compiling rules from multiple sources, including built-in and user-defined rules.
- Added integration tests to ensure the correctness of the output reduction process.
These changes enhance the application's ability to manage and compact verbose tool outputs, improving overall efficiency and user experience.
* test(tokenjuice): enhance test coverage for classification and reduction logic
- Added a series of unit tests to `classify.rs` to validate the behavior of tool name filters and argument matching, ensuring correct classification of tool executions.
- Introduced tests for edge cases in `reduce.rs`, including command tokenization and normalization of execution inputs, to improve robustness against various input formats.
- Expanded tests in `builtin.rs` to cover duplicate ID reporting and compile issues, enhancing error handling and reporting mechanisms.
- Implemented additional tests in `compiler.rs` to verify regex handling in rule definitions, ensuring invalid patterns are correctly ignored.
These enhancements improve the overall test coverage and reliability of the tokenjuice module, facilitating better maintenance and future development.
* test(tokenjuice): add edge-case tests for gh table and reduction pipeline
* style(tokenjuice): apply cargo fmt
* feat(tokenjuice): wire into agent tool loop output compaction
Add `tokenjuice::compact_tool_output` helper and call it in the agent
tool loop after credential scrubbing (and on error paths with exit=1),
before any optional payload_summarizer. Derives argv/command
heuristically from JSON tool arguments (command / args / argv / cmd
shapes) so shell-wrapping tools still match upstream family rules
(git/*, package/*, tests/*, etc.). Pass-through safe: outputs under
512 bytes or where compaction saves <5% are returned untouched.
* fix(tokenjuice): address coderabbit review comments
- classify: derive command from argv join when input.command is unset,
so commandIncludes* rules still match argv-only callers
- rules/loader: log read_dir / file_type / read_to_string failures at
debug level so permission or filesystem issues are observable rather
than silently skipped
- text/ansi: add trace log at strip_ansi entry/exit with lengths (no
text content) per the project debug-logging rules
- tests: remove orphan src/openhuman/tokenjuice/tests/integration.rs
which was never wired into any module declaration; the real fixture-
parity runner lives at tests/tokenjuice_integration.rs and asserts
hard when the fixtures directory is missing
Vendored-rule issues (docker-ps / kubectl-describe / git/branch /
grep casing / counter-pattern overbreadth / etc.) come from upstream
and are left as-is; this module is a straight port of the upstream
rule set and should not fork from it in v1.
* feat(composio): add user scope management for toolkits
- Introduced `get_user_scopes` and `set_user_scopes` functions to manage per-toolkit user scope preferences, allowing for read, write, and admin classifications.
- Updated `all_controller_schemas` and `all_registered_controllers` to include new schemas for user scope management.
- Implemented `evaluate_tool_visibility` to determine tool visibility based on user-defined scopes, enhancing security and control over tool actions.
- Added `UserScopePref` struct to store user preferences and integrated it with memory storage for persistence.
- Enhanced existing tools to respect user scope preferences during execution, ensuring actions align with user-defined permissions.
These changes improve the flexibility and security of toolkit interactions, allowing users to customize their access levels for different actions.
* feat(gmail): expand GMAIL_CURATED tools for enhanced email management
- Updated the GMAIL_CURATED constant to include additional tools for reading and writing emails, such as GMAIL_LIST_MESSAGES, GMAIL_LIST_THREADS, GMAIL_GET_ATTACHMENT, and GMAIL_FORWARD_MESSAGE.
- Improved organization of tools by categorizing them under distinct sections for reading messages, managing drafts, and handling labels.
- Enhanced admin tools with new functionalities like GMAIL_BATCH_DELETE_MESSAGES and GMAIL_UNTRASH_THREAD, improving overall email management capabilities.
These changes provide a more comprehensive toolkit for interacting with Gmail, enhancing user experience and functionality.
* refactor(tool_scope, gmail): improve code formatting and organization
- Reformatted the `ADMIN` and `GMAIL_CURATED` constants for better readability by aligning the entries vertically.
- Enhanced the clarity of the `classify_unknown` function by improving the structure of the constants, making it easier to maintain and understand.
- Updated test assertions in `toolkit_from_slug` for consistency in formatting, ensuring clearer test outputs.
These changes improve the overall readability and maintainability of the codebase, aligning with ongoing refactoring efforts.
* feat(github): add GitHub toolkit and curated tools for enhanced integration
- Introduced a new GitHub provider module, including a curated catalog of GitHub actions tailored for common tasks such as repository management, issue tracking, and pull request handling.
- Implemented the `catalog_for_toolkit` function to allow fallback to a static curated list for toolkits without a native provider, ensuring consistent tool visibility and access.
- Updated the `evaluate_tool_visibility` function to prioritize curated tools from registered providers, enhancing the overall user experience and security by enforcing whitelist checks.
These changes expand the capabilities of the Composio toolkit, providing users with a comprehensive set of tools for interacting with GitHub, while maintaining a focus on user-defined permissions and visibility.
* refactor(tools): improve formatting and organization of curated tools
- Reformatted the `GITHUB_CURATED`, `NOTION_CURATED`, and other tool constants for better readability by aligning entries vertically.
- Enhanced the clarity of the code structure, making it easier to maintain and understand.
- Updated related documentation to reflect the changes in formatting and organization.
These changes improve the overall readability and maintainability of the codebase, aligning with ongoing refactoring efforts.
* feat(catalogs): introduce curated catalogs for various toolkits
- Added a new module `catalogs.rs` containing curated tool lists for Slack, Discord, Google Calendar, Google Drive, Google Docs, and more, enhancing the Composio toolkit's integration capabilities.
- Updated the `mod.rs` file to include the new `catalogs` module and modified the `catalog_for_toolkit` function to support these curated lists, allowing for better organization and access to toolkit actions.
- This addition improves the overall functionality and user experience by providing a comprehensive set of tools for interacting with popular platforms.
* feat(post-process): implement HTML to markdown conversion for Gmail responses
- Introduced a new `post_process` module to handle per-toolkit response modifications, specifically for converting HTML content to markdown format.
- Enhanced the `ComposioExecuteTool` to apply post-processing on successful responses, improving the clarity and usability of data returned from Gmail.
- Added tests to ensure the correct functionality of HTML detection and conversion, validating the integrity of the post-processing logic.
These changes enhance the user experience by streamlining the handling of HTML content in responses, making it more suitable for further processing and display.
* refactor(post_process): reorganize HTML detection constants for improved readability
- Moved HTML detection markers in the `looks_like_html` function to a more structured format, enhancing clarity and maintainability.
- This change aligns with ongoing efforts to improve code organization and readability within the post-processing module.
* refactor(catalogs): enhance formatting and organization of curated tool constants
- Reformatted the `SLACK_CURATED`, `DISCORD_CURATED`, and `GOOGLECALENDAR_CURATED` constants for improved readability by aligning entries vertically.
- This change enhances the clarity and maintainability of the code, aligning with ongoing refactoring efforts to improve code organization.
* feat(connect-modal): wire read/write/admin scope toggles to composio prefs
Adds the three scope toggles to the connected-state of the
ComposioConnectModal so users can gate which Composio actions the
agent may invoke per integration. Loads the stored pref via
`composio_get_user_scopes` once the modal lands in the connected
phase and persists changes through `composio_set_user_scopes` with
optimistic updates and rollback on error.
- Adds `getUserScopes` / `setUserScopes` to composioApi.
- Adds `ComposioUserScopePref` type mirror.
- Renders accessible role="switch" toggles with hint text per row.
* refactor(ComposioConnectModal): simplify SCOPE_ROWS definition
- Streamlined the definition of SCOPE_ROWS in ComposioConnectModal by removing unnecessary line breaks, enhancing code readability and maintainability.
- Updated the agent.toml configuration to include "composio_execute" in the tools list, expanding the capabilities of the integrations agent.
* fix(composio): apply curated whitelist + scope pref to integrations_agent prompt
The agent prompt for integrations_agent was rendering every action
returned by the backend's `composio_list_tools` for each connected
toolkit, bypassing the curation/scope filter that the meta-tool layer
applies. Concretely the GitHub integrations_agent prompt was showing
~500 actions including non-curated entries like
GITHUB_ADD_OR_UPDATE_TEAM_REPOSITORY_PERMISSIONS.
Adds `is_action_visible_with_pref(slug, pref)` — a sync helper that
mirrors the meta-tool layer's decision logic — and applies it in:
- `fetch_connected_integrations_uncached` (bulk session-cached path)
- `fetch_toolkit_actions` (per-toolkit spawn-time path)
One pref load per toolkit (not per action) keeps the cost minimal.
* refactor(fetch_connected_integrations): streamline action visibility filter
Simplified the action visibility filter in `fetch_connected_integrations_uncached` by consolidating the filter logic into a single line. This change enhances code readability while maintaining the existing functionality of applying user preferences to the displayed actions.
* fix(prompt): drop duplicate `### Available Tools` listing in text-mode preamble
Text-mode subagent prompts were rendering the tool catalog twice: once
in the prompt template's `## Tools` section (with the richer
`Call as: NAME[arg|arg]` signatures from `prompts::ToolsSection::build`)
and once in `### Available Tools` under `## Tool Use Protocol`
(`Parameters: name:type, ...` format).
For an integrations_agent toolkit spawn (~50 actions) this doubled the
tool listing bytes for no informational gain. Keep only the protocol
preamble (essential for text mode); the catalog stays in `## Tools`.
Removes `summarise_parameters` and `first_line_truncated` which were
the sole consumers, plus the now-unused `std::fmt::Write` import.
* review: address PR feedback (UTF-8 boundary, structured logs, doc, debug)
Real fixes:
- post_process: walk back to UTF-8 char boundary before truncating to
4096 bytes; previous `&s[..4096]` could panic mid-codepoint. Adds
regression test that places a 3-byte char straddling the cutoff.
- composioApi: move misplaced `execute` docstring back above `execute`
(it had drifted above the new `getUserScopes`).
- schemas: include `get_user_scopes` / `set_user_scopes` in the
`every_known_schema_key_resolves` test.
Diagnosability:
- schemas: structured `[composio:scopes]` debug/error logs at entry,
exit, and every early-return in `handle_get_user_scopes` /
`handle_set_user_scopes` (method + toolkit + pref fields). The
memory-not-ready branch now logs an error before returning.
- composioApi + ConnectModal: grep-friendly `[composio][scopes]` debug
logs around getUserScopes / setUserScopes RPC round-trips and the
toggle handler (old → new state, persisted result, errors).
- user_scopes: load_or_default now logs the same normalized `key` that
`load()` does, so traces correlate across both code paths.
Cleanups:
- tools: replace external `idx` + `Vec::retain` in
`filter_list_tools_response` with a drain + zip + filter_map pattern.
- tool_scope: document the no-underscore-in-toolkit-name assumption of
`toolkit_from_slug`, and call out the `microsoft_teams` alias.
- Cargo.toml: comment on the html2md vs htmd choice.
Pushback (no change):
- Reviewer asked to split the type-only import in ConnectModal into a
separate `import type` line; ESLint's `no-duplicate-imports` rejects
that, so the inline `type` form (functionally identical) stays.
github-actions[bot]