5fff5568d3 feat(memory): Phase 2 memory tree - preprocessing, scoring, admission gate (#708) (#733)
* feat(memory): phase 1 memory tree - multi-source ingestion & canonical chunks (#707)

Adds an isolated memory tree layer under src/openhuman/memory/tree/
implementing Phase 1 of the new memory architecture (umbrella #711).
Zero edits to existing memory/*.rs files - the new layer coexists
with the legacy TinyHumans-backed client.

- Source adapters: chat / email / document -> canonical Markdown
- Token-bounded chunker with deterministic SHA-256 chunk IDs
- SQLite persistence at <workspace>/memory_tree/chunks.db with full
  provenance metadata (source_kind, source_id, owner, timestamps,
  tags, time_range) and back-pointer to raw source
- Unified JSON-RPC ingest (dispatches on source_kind + JSON payload):
  openhuman.memory_tree_ingest, _list_chunks, _get_chunk
- DataSource enum covering the 8 providers from m.excalidraw step 1
  (Discord/Telegram/Whatsapp/Gmail/OtherEmail/Notion/MeetingNotes/DriveDocs)
- ~40 unit tests (chunk ID stability, UTF-8-safe splitting,
  canonicalisation idempotence, store round-trip, filter behavior)

Additive only: new tables in a new DB file, new JSON-RPC namespace,
no existing behavior changes. Feeds #708 (scoring), #709 (summary
trees), #710 (query tools).

Closes #707. Parent: #711.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(memory): phase 2 memory tree - preprocessing, scoring, admission gate (#708)

Adds the scoring / admission layer between Phase 1's chunker and store.
Stacked on feat/707-memory-ingestion (PR #732) - depends on Phase 1's
chunk substrate.

- Pluggable EntityExtractor trait + CompositeExtractor chain
- RegexEntityExtractor: mechanical entities (emails, URLs, @handles, #hashtags)
  - Always on, deterministic, zero deps, UTF-8-safe char spans
- Five weighted signals: token count, unique-word ratio, metadata weight,
  source weight (per-DataSource), interaction (reply/sent/mention/dm tags),
  entity density
- Exact-match entity canonicalisation (email lowercased, @ and # stripped)
- Admission gate drops chunks below configurable threshold (default 0.3)
  - Score rationale persists for EVERY chunk (kept or dropped) for debugging
  - Entities indexed for KEPT chunks only
- Two new SQLite tables added to the memory_tree DB:
  - mem_tree_score: per-chunk score rationale with all signal values
  - mem_tree_entity_index: inverted index entity_id -> node_id
- Idempotent ALTER TABLE migration adds embedding BLOB column to
  mem_tree_chunks (used in Phase 3 retrieval, wired but not populated here)
- Ingest pipeline converted to async to accommodate the extractor trait;
  blocking SQLite work isolated on spawn_blocking; JSON-RPC surface
  unchanged (same memory_tree_ingest / list / get methods)
- Phase 2 deliberately ships without GLiNER/semantic NER - per-chunk
  semantic entities land later behind a cargo feature flag; the composite
  extractor interface keeps that drop-in trivial

Additive only: new tables, new columns, new module. Existing Phase 1
behavior unchanged except that low-signal chunks are now dropped before
reaching mem_tree_chunks. Raise score_drop_threshold to 0 to disable
the gate and restore Phase-1-identical behavior.

Closes #708. Parent: #711. Depends on: #707 (#732).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Fix memory tree scoring persistence issues

* Fix memory tree scoring robustness issues from PR review

- ingest: fail fast if scorer returns fewer/more results than chunks
  (silent zip truncation would drop chunks or their score rationale)
- score::persist_score{,_tx}: clear stale entity-index rows before
  re-indexing a re-scored chunk, since INSERT OR REPLACE never deletes
  rows whose entity_id is no longer in the new extraction
- score::store::lookup_entity: clamp limit to i64::MAX before casting
  to prevent a large usize wrapping into a negative LIMIT

Adds clear_entity_index_drops_stale_rows regression test.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai>
2026-04-21 16:17:27 -07:00
2026-04-21 14:14:15 +00:00
2026-02-20 13:03:15 +04:00
2026-02-20 13:03:15 +04:00

OpenHuman

The age of super intelligence is here. OpenHuman is your Personal AI super intelligence. Private, Simple and extremely powerful.

DiscordRedditX/TwitterDocs

Early Beta Platforms: desktop only Latest Release

The Tet

"The Tet. What a brilliant machine" — Morgan Freeman as he reminisces about alien superintelligence in the movie Oblivion

Early Beta — Under active development. Expect rough edges.

To install or get started, either download from the website over at tinyhumans.ai/openhuman or run

# For MacOS/Linux
curl -fsSL https://raw.githubusercontent.com/tinyhumansai/openhuman/main/scripts/install.sh | bash

# For Windows
irm https://raw.githubusercontent.com/tinyhumansai/openhuman/main/scripts/install.ps1 | iex

What is OpenHuman?

OpenHuman is an open-source agentic assistant that is designed to integrate with you in your daily life. Here's what makes OpenHuman special:

  • Simple, UI-first — A clean desktop experience and short onboarding paths so you can go from install to a working agent in a few clicks, without a config-first setup. You don't need a terminal to run OpenHuman.

  • One subscription, many providers — You only need one account to get access to many agentic APIs (AI Models, Search, Webhooks/Tunnels and other 3rd party APIs etc..), simplifying the experience to get a powerful agent going.

  • Rich Skills — Plug into Gmail, Slack, Notion, and the rest of your stack via rich, feature-backed skills. Connections are typically one click through setup wizards instead of wiring APIs by hand. Workflow data is kept on device, encrypted locally, and treated as yours: encryption and sensitive context stay on your machine. Webhooks give instant feedback into the agent when external systems or skills emit events, so the loop stays tight without constant polling.

  • Local knowledge base — Built from your data and your activity. How you work across tools, sessions, and connected services—so the agent gets rich, workflow-aware context, not a one-off chat transcript. Everything is stored on your machine and compounding over time without becoming a cloud dossier. Channels, skills and ongoing conversations feed the same loop so day-to-day context does not reset every session.

  • Local AI model — The Rust core exposes local AI paths (and the desktop bundle can ship local/bundled runners where applicable) for the workloads above—vision snippets, speech helpers, summarization, tooling—so sensitive steps can stay off the cloud when you choose.

  • Deep desktop integrations — OpenHuman is a native desktop assistant, not a web-only chat: memory-aware keyboard autocomplete, voice (STT listening and TTS replies), screen intelligence that understands what is on screen and feeds your local context, plus windowing and OS-level permissions—so the agent meets you on the machine, not trapped in a browser tab.

Architecture: docs/ARCHITECTURE.md. Contributor orientation: CONTRIBUTING.md. Running from source: docs/install.md.

OpenHuman vs other agents

High-level comparison (products evolve—verify against each vendor). OpenHuman is built to minimize vendor sprawl, keep workflow knowledge on-device, and ship deep desktop features—not only chat.

Claude Code/Cowork OpenClaw Hermes Agent OpenHuman
Open-source: Is the codebase open to review? 🚫 Proprietary client MIT License MIT License GNU License
Simple: Is it simple to get started? Simple Desktop App + CLI ⚠️ Terminal first and often complex ⚠️ Terminal first and often complex Simple, Clean UI/UX. Get started within minutes
Cost: How expensive is to run? ⚠️ Subscription + add-on tool/API costs ⚠️ Tied to models & hosting you choose ⚠️ Tied to models & hosting you choose Cost optimized with the option to run many things locally for free
Memory & Knowledge Base (KB): Does the agent know you and your world? Built-in memory; mostly chat/session scoped ⚠️ Has a local memory but often needs plugins for richer behavior Self-learning / task loops (typical) 🚀 Local KB + Self-learning from your activity & data (GMail, Notion etc... via skills) & prompts
API spagetti: How complex is it to hook mulitple features together? 🚫 Claude bill + often extra keys for MCP/tools 🚫 BYOK / multi-vendor common 🚫 Multiple providers common One account get access to many bundled platform APIs
Extensibility: Can you add rich features into it? MCP (different model than sandboxed skills) Plugin Architecture (SKILL.md) Plugin Architecture (SKILL.md) 🚀 Rich Skills with ability to have realtime updates, local DB & more
Desktop integrations: Can it integrate into your desktop completely? ⚠️ Desktop app & access to folders ⚠️ Often lighter native surface ⚠️ Often lighter native surface STT, TTS, screen intelligence, memory-aware autocomplete and a whole lot more

Contributors Hall of Fame

Show some love and end up in the hall of fame

OpenHuman contributors
S
Description
No description provided
Readme GPL-3.0
214 MiB
Languages
Rust 59.1%
TypeScript 37.9%
JavaScript 1.6%
Shell 1.2%
CSS 0.1%