admin/openhuman
1
0
Fork 0
mirror of https://github.com/tinyhumansai/openhuman.git synced 2026-07-30 23:14:37 +00:00
Code Issues Packages Projects Releases Wiki Activity

[codex] Port TokenJuice engine to vendored TinyJuice (#4523)

Browse Source
This commit is contained in:
Steven Enamakel
2026-07-04 21:02:48 -07:00
committed by GitHub
parent 14ce6361c0
commit 1d7f0cc62d
156 changed files with 133 additions and 14391 deletions
Download Patch File Download Diff File Expand all files Collapse all files
gitbooks/developing/architecture/agent-harness.md
+1 -1
View File
@@ -171,7 +171,7 @@ When a tool result exceeds the summarizer's threshold, it gets routed through a
### TokenJuice - content-aware tool-output compaction (Stage 1a)
Before a fresh tool result enters history (and ahead of the byte-budget backstop), it passes through the **TokenJuice content router** (`src/openhuman/tokenjuice/`). Inspired by Headroom, the router *detects the content kind* (JSON, code, log, search, diff, HTML, plain text) from the bytes and/or a hint derived from the tool name and arguments, then dispatches to a specialised compressor:
Before a fresh tool result enters history (and ahead of the byte-budget backstop), it passes through the **TokenJuice content router** in the vendored TinyJuice crate (`vendor/tinyjuice`), with OpenHuman adapters in `src/openhuman/tokenjuice/`. Inspired by Headroom, the router *detects the content kind* (JSON, code, log, search, diff, HTML, plain text) from the bytes and/or a hint derived from the tool name and arguments, then dispatches to a specialised compressor:
* **JSON** → SmartCrusher: array-of-objects → table (each key once), preserving rows that carry errors or numeric outliers.
* **Code** → tree-sitter (Rust/TS/JS/Python) signature keeper that collapses function bodies; brace-depth heuristic fallback.