mirror of
https://github.com/tinyhumansai/openhuman.git
synced 2026-07-27 21:08:00 +00:00
GITBOOK-39: No subject
This commit is contained in:
Steven Enamakel
Binary file not shown.
|
After Width: | Height: | Size: 8.5 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 9.2 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 9.3 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 210 KiB After Width: | Height: | Size: 1006 KiB |
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
+15
-18
@@ -1,27 +1,29 @@
|
||||
---
|
||||
description: >-
|
||||
OpenHuman is a personal AI assistant that runs on your desktop, connects to
|
||||
118+ services, and builds a local-first memory of your life from them.
|
||||
OpenHuman is a personal AI assistant that runs on your desktop, connects to
|
||||
118+ services, and builds a local-first memory of your life from them.
|
||||
icon: diamond
|
||||
---
|
||||
|
||||
# Welcome to OpenHuman
|
||||
|
||||
OpenHuman is an open-source AI assistant designed to be the **memory** and **doer** for everything you do across your tools. Built on Rust + Tauri and licensed under GNU GPL3, it closes the gap between what AI models can do and what they actually know about *you*.
|
||||
<figure><img src=".gitbook/assets/Screenshot 2026-05-08 at 9.39.39 PM.png" alt=""><figcaption></figcaption></figure>
|
||||
|
||||
OpenHuman is an open-source AI assistant designed to be the **memory** and **doer** for everything you do across your tools. Built on Rust + Tauri and licensed under GNU GPL3, it closes the gap between what AI models can do and what they actually know about _you_.
|
||||
|
||||
Every model in the world — all 200+ of them — shares the same fundamental limitation: they are stateless. You type a prompt, get a response, and the context evaporates. Even the ones with "memory" store a few bullet points. A few bullet points is a sticky note, not intelligence.
|
||||
|
||||
OpenHuman solves this with a stack that's calmly, deliberately different:
|
||||
|
||||
**A local-first [Memory Tree](features/memory-tree.md).** Every source you connect — Gmail, Slack, GitHub, Notion, your own notes — flows through a deterministic pipeline: canonical Markdown, ≤3k-token chunks, scored, folded into per-source / per-topic / per-day summary trees. Stored in SQLite on your machine. No vector-soup black box.
|
||||
**A local-first** [**Memory Tree**](features/memory-tree.md)**.** Every source you connect — Gmail, Slack, GitHub, Notion, your own notes — flows through a deterministic pipeline: canonical Markdown, ≤3k-token chunks, scored, folded into per-source / per-topic / per-day summary trees. Stored in SQLite on your machine. No vector-soup black box.
|
||||
|
||||
**An [Obsidian-style wiki](features/obsidian-wiki.md) on top of it.** The same chunks the agent reasons over land as `.md` files in a vault you can open in [Obsidian](https://obsidian.md), browse, edit, and link by hand. Inspired by [Karpathy's obsidian-wiki workflow](https://x.com/karpathy/status/2039805659525644595). You can't trust a memory you can't read.
|
||||
**An** [**Obsidian-style wiki**](features/obsidian-wiki.md) **on top of it.** The same chunks the agent reasons over land as `.md` files in a vault you can open in [Obsidian](https://obsidian.md), browse, edit, and link by hand. Inspired by [Karpathy's obsidian-wiki workflow](https://x.com/karpathy/status/2039805659525644595). You can't trust a memory you can't read.
|
||||
|
||||
**[118+ third-party integrations](features/integrations.md), with [auto-fetch](features/auto-fetch.md).** One-click OAuth into Gmail, GitHub, Slack, Notion, Stripe, Calendar, Drive, Linear, Jira and more. Every five minutes, OpenHuman pulls fresh data from every active connection and folds it into the Memory Tree without you asking.
|
||||
[**118+ third-party integrations**](features/integrations.md)**, with** [**auto-fetch**](features/auto-fetch.md)**.** One-click OAuth into Gmail, GitHub, Slack, Notion, Stripe, Calendar, Drive, Linear, Jira and more. Every five minutes, OpenHuman pulls fresh data from every active connection and folds it into the Memory Tree without you asking.
|
||||
|
||||
**An agent built for big data.** [Smart token compression (TokenJuice)](features/token-compression.md) compacts verbose tool output before it ever enters the model's context, so sweeping through your last six months of email costs single-digit dollars. [Automatic model routing](features/model-routing.md) sends each task to the right model — `hint:reasoning` to a frontier model, `hint:fast` to a cheap one, vision to vision — all under one subscription. Built-in [native tools](features/native-tools.md) cover web search, scraping, and a full filesystem/git/lint/test/grep coder loop.
|
||||
|
||||
**[Native voice (ElevenLabs)](features/voice.md).** STT in, ElevenLabs TTS out, mascot lip-sync, and a live Google Meet agent that joins meetings, transcribes them into your Memory Tree, and can speak back into the call.
|
||||
[**Native voice (ElevenLabs)**](features/voice.md)**.** STT in, ElevenLabs TTS out, mascot lip-sync, and a live Google Meet agent that joins meetings, transcribes them into your Memory Tree, and can speak back into the call.
|
||||
|
||||
Together, these turn OpenHuman into something fundamentally different from a chatbot. It is an AI agent that consumes large amounts of personal data at low cost, maintains a persistent and evolving understanding of your world, and takes proactive actions on your behalf.
|
||||
|
||||
@@ -33,17 +35,12 @@ OpenHuman is not AGI. But it is a meaningful architectural step closer, with bet
|
||||
|
||||
OpenHuman connects to your tools, pulls from them continuously, and turns the firehose into something structured any AI can act on.
|
||||
|
||||
It **fetches automatically** from every active integration every five minutes, so the agent already has tomorrow's context this morning.
|
||||
|
||||
It **compresses** millions of tokens of organizational noise into a deterministic Memory Tree of chunks, scores, entities, and summaries.
|
||||
|
||||
It **surfaces signals** that matter: decisions, action items, risks, sentiment shifts, and buried context you would otherwise miss.
|
||||
|
||||
It **routes intelligently**, picking the right model for each task and compacting tool output through TokenJuice so cost stays minimal even at scale.
|
||||
|
||||
It **speaks**, listens, and joins meetings — voice is a first-class surface, not an afterthought.
|
||||
|
||||
It **preserves privacy** by design. The Memory Tree's SQLite database and your Obsidian vault stay on your machine. Integration tokens are held by the OpenHuman backend, not on disk in plaintext on your laptop.
|
||||
* It **fetches automatically** from every active integration every five minutes, so the agent already has tomorrow's context this morning.
|
||||
* It **compresses** millions of tokens of organizational noise into a deterministic Memory Tree of chunks, scores, entities, and summaries.
|
||||
* It **surfaces signals** that matter: decisions, action items, risks, sentiment shifts, and buried context you would otherwise miss.
|
||||
* It **routes intelligently**, picking the right model for each task and compacting tool output through TokenJuice so cost stays minimal even at scale.
|
||||
* It **speaks**, listens, and joins meetings — voice is a first-class surface, not an afterthought.
|
||||
* It **preserves privacy** by design. The Memory Tree's SQLite database and your Obsidian vault stay on your machine. Integration tokens are held by the OpenHuman backend, not on disk in plaintext on your laptop.
|
||||
|
||||
## Who it's for
|
||||
|
||||
|
||||
+2
-2
@@ -73,7 +73,7 @@
|
||||
* [Memory Context Window](developing/memory-context-window.md)
|
||||
* [Memory Tree Pipeline](developing/memory-tree-pipeline.md)
|
||||
|
||||
### Frontend (app/src/)
|
||||
## Frontend (app/src/)
|
||||
|
||||
* [Overview](developing/frontend/README.md)
|
||||
* [Architecture](developing/frontend/architecture.md)
|
||||
@@ -85,7 +85,7 @@
|
||||
* [Providers](developing/frontend/providers.md)
|
||||
* [Hooks & Utilities](developing/frontend/hooks-utils.md)
|
||||
|
||||
### Tauri shell (app/src-tauri/)
|
||||
## Tauri shell (app/src-tauri/)
|
||||
|
||||
* [Overview](developing/tauri-shell/README.md)
|
||||
* [Architecture](developing/tauri-shell/architecture.md)
|
||||
|
||||
@@ -5,13 +5,13 @@ description: >-
|
||||
icon: diagram-project
|
||||
---
|
||||
|
||||
# Memory Tree Async Pipeline
|
||||
# Memory Tree Pipeline
|
||||
|
||||
The user-facing pitch of the [Memory Tree](../features/memory-tree.md) is simple: connect a source, the agent gets persistent memory of it. The pipeline that delivers on that pitch is **not** simple — it spans an HTTP-triggered ingest path, a job queue inside SQLite, a pool of background workers, three independent summary trees, and a daily UTC scheduler. This page walks through the whole thing.
|
||||
|
||||
The diagram below is the source of truth. Every box maps to code under `src/openhuman/memory/tree/`.
|
||||
|
||||
{% file src="memory-tree-pipeline.excalidraw" %}
|
||||
{% file src="../.gitbook/assets/memory-tree-pipeline (1).excalidraw" %}
|
||||
Memory Tree Async Pipeline — leaf ingestion → jobs queue → workers → source / topic / global tree building.
|
||||
{% endfile %}
|
||||
|
||||
@@ -36,9 +36,9 @@ canonicalize
|
||||
|
||||
Hot-path requirements:
|
||||
|
||||
- **Deterministic.** The `chunk_id` is a hash of `(source_kind, source_id, position, body_hash)`. Re-running ingest on identical input never produces duplicates.
|
||||
- **Fast.** No LLM calls in this lane. `score_chunks_fast` uses cheap heuristics; deeper scoring runs out of the worker pool.
|
||||
- **Bounded write.** Everything happens inside one SQLite transaction so a partial ingest can't leave dangling rows.
|
||||
* **Deterministic.** The `chunk_id` is a hash of `(source_kind, source_id, position, body_hash)`. Re-running ingest on identical input never produces duplicates.
|
||||
* **Fast.** No LLM calls in this lane. `score_chunks_fast` uses cheap heuristics; deeper scoring runs out of the worker pool.
|
||||
* **Bounded write.** Everything happens inside one SQLite transaction so a partial ingest can't leave dangling rows.
|
||||
|
||||
Code: `src/openhuman/memory/tree/ingest.rs`, `chunker.rs`, `score/fast.rs`.
|
||||
|
||||
@@ -46,26 +46,26 @@ Code: `src/openhuman/memory/tree/ingest.rs`, `chunker.rs`, `score/fast.rs`.
|
||||
|
||||
Storage: SQLite at `<workspace>/memory_tree/chunks.db`. Tables:
|
||||
|
||||
| Table | What's there |
|
||||
| --- | --- |
|
||||
| `mem_tree_chunks` | The chunks themselves — body hash, provenance, lifecycle status. |
|
||||
| `mem_tree_score` | Per-chunk score rows (fast + deep). |
|
||||
| `mem_tree_entity_index` | Entity → chunk lookup for topic-tree hotness. |
|
||||
| `mem_tree_jobs` | The job queue (see below). |
|
||||
| `mem_tree_trees` | Per-scope tree metadata (source / topic / global). |
|
||||
| `mem_tree_buffers` | L0 buffers (unsealed leaves) per tree. |
|
||||
| `mem_tree_summaries` | Sealed summaries (L1/L2/...) per tree. |
|
||||
| Table | What's there |
|
||||
| ----------------------- | ---------------------------------------------------------------- |
|
||||
| `mem_tree_chunks` | The chunks themselves — body hash, provenance, lifecycle status. |
|
||||
| `mem_tree_score` | Per-chunk score rows (fast + deep). |
|
||||
| `mem_tree_entity_index` | Entity → chunk lookup for topic-tree hotness. |
|
||||
| `mem_tree_jobs` | The job queue (see below). |
|
||||
| `mem_tree_trees` | Per-scope tree metadata (source / topic / global). |
|
||||
| `mem_tree_buffers` | L0 buffers (unsealed leaves) per tree. |
|
||||
| `mem_tree_summaries` | Sealed summaries (L1/L2/...) per tree. |
|
||||
|
||||
`mem_tree_jobs` columns that matter:
|
||||
|
||||
| Column | Purpose |
|
||||
| --- | --- |
|
||||
| `kind` | One of `extract_chunk`, `append_buffer`, `seal`, `topic_route`, `digest_daily`, `flush_stale`. |
|
||||
| `payload_json` | Job-specific args. |
|
||||
| `dedupe_key` | Coalesces duplicate enqueues — re-running an idempotent job costs one row, not N. |
|
||||
| `status` | `pending` / `running` / `done` / `failed`. |
|
||||
| `attempts` / `last_error` | Retry bookkeeping. |
|
||||
| `available_at_ms` / `locked_until_ms` | Scheduling and worker leasing. |
|
||||
| Column | Purpose |
|
||||
| ------------------------------------- | ---------------------------------------------------------------------------------------------- |
|
||||
| `kind` | One of `extract_chunk`, `append_buffer`, `seal`, `topic_route`, `digest_daily`, `flush_stale`. |
|
||||
| `payload_json` | Job-specific args. |
|
||||
| `dedupe_key` | Coalesces duplicate enqueues — re-running an idempotent job costs one row, not N. |
|
||||
| `status` | `pending` / `running` / `done` / `failed`. |
|
||||
| `attempts` / `last_error` | Retry bookkeeping. |
|
||||
| `available_at_ms` / `locked_until_ms` | Scheduling and worker leasing. |
|
||||
|
||||
Code: `src/openhuman/memory/tree/store.rs`, `jobs/queue.rs`.
|
||||
|
||||
@@ -73,21 +73,21 @@ Code: `src/openhuman/memory/tree/store.rs`, `jobs/queue.rs`.
|
||||
|
||||
Bootstrap: `jobs::start(workspace_dir)` is called once at process startup. It:
|
||||
|
||||
- Calls `recover_stale_locks()` — any job whose `locked_until_ms` is in the past becomes `pending` again. Crashes don't strand work.
|
||||
- Spawns **3 worker tasks** (configurable, but 3 is the default and what production runs).
|
||||
- Wires a `tokio::sync::Notify` so the ingest path can wake workers immediately, with a **5-second polling fallback** so a missed notify doesn't strand work.
|
||||
- Holds a shared `Semaphore(3)` for LLM-bound steps so concurrent embedding / summarization calls can't blow past the configured budget.
|
||||
* Calls `recover_stale_locks()` — any job whose `locked_until_ms` is in the past becomes `pending` again. Crashes don't strand work.
|
||||
* Spawns **3 worker tasks** (configurable, but 3 is the default and what production runs).
|
||||
* Wires a `tokio::sync::Notify` so the ingest path can wake workers immediately, with a **5-second polling fallback** so a missed notify doesn't strand work.
|
||||
* Holds a shared `Semaphore(3)` for LLM-bound steps so concurrent embedding / summarization calls can't blow past the configured budget.
|
||||
|
||||
Each worker pulls a job, runs the right handler, and updates the row. Handlers:
|
||||
|
||||
| Handler | Job kind | What it does |
|
||||
| --- | --- | --- |
|
||||
| `extract_chunk` | `extract_chunk` | Deep score + entity extraction. Decides `admitted` vs `dropped` based on the score. |
|
||||
| `append_buffer` | `append_buffer` | Adds an admitted leaf to the source (or topic) tree's L0 buffer. May trigger a seal. |
|
||||
| `seal` | `seal` | Compresses L0 buffer into an L1 summary; cascades up through L2/L3/... if the parent buffer is now full. |
|
||||
| `topic_route` | `topic_route` | Routes a leaf into per-entity topic trees, gated by the curator hotness check. |
|
||||
| `digest_daily` | `digest_daily` | Builds the global daily digest node. |
|
||||
| `flush_stale` | `flush_stale` | Force-seals buffers that have been sitting too long. |
|
||||
| Handler | Job kind | What it does |
|
||||
| --------------- | --------------- | -------------------------------------------------------------------------------------------------------- |
|
||||
| `extract_chunk` | `extract_chunk` | Deep score + entity extraction. Decides `admitted` vs `dropped` based on the score. |
|
||||
| `append_buffer` | `append_buffer` | Adds an admitted leaf to the source (or topic) tree's L0 buffer. May trigger a seal. |
|
||||
| `seal` | `seal` | Compresses L0 buffer into an L1 summary; cascades up through L2/L3/... if the parent buffer is now full. |
|
||||
| `topic_route` | `topic_route` | Routes a leaf into per-entity topic trees, gated by the curator hotness check. |
|
||||
| `digest_daily` | `digest_daily` | Builds the global daily digest node. |
|
||||
| `flush_stale` | `flush_stale` | Force-seals buffers that have been sitting too long. |
|
||||
|
||||
Code: `src/openhuman/memory/tree/jobs/{worker.rs, handlers/}`.
|
||||
|
||||
@@ -107,8 +107,8 @@ Code: `src/openhuman/memory/tree/{tree_source, tree_topic, tree_global}/`.
|
||||
|
||||
A separate scheduler loop runs independently of the ingest path:
|
||||
|
||||
- **UTC daily tick.** At 00:00 UTC each day, enqueue `digest_daily(yesterday)` and `flush_stale(today)`. Both go through the same `mem_tree_jobs` pipeline workers consume.
|
||||
- **`flush_stale`** scans every tree's buffers for ones older than the configured TTL and enqueues force-seal jobs.
|
||||
* **UTC daily tick.** At 00:00 UTC each day, enqueue `digest_daily(yesterday)` and `flush_stale(today)`. Both go through the same `mem_tree_jobs` pipeline workers consume.
|
||||
* **`flush_stale`** scans every tree's buffers for ones older than the configured TTL and enqueues force-seal jobs.
|
||||
|
||||
The scheduler **does not** run summarizers itself. Everything goes through the queue, so retries, dedupe, and stale-lock recovery all stay centralized.
|
||||
|
||||
@@ -124,10 +124,10 @@ pending_extraction ──► admitted ──► buffered ──► sealed
|
||||
──► dropped
|
||||
```
|
||||
|
||||
- `extract_chunk` decides `admitted` vs `dropped` based on the deep score.
|
||||
- `append_buffer` moves admitted leaves into a buffer — `buffered`.
|
||||
- `seal` writes the buffer's contents into a summary and marks each leaf `sealed`.
|
||||
- `dropped` leaves stop here. Their chunk row stays for provenance, but no buffer / summary references them.
|
||||
* `extract_chunk` decides `admitted` vs `dropped` based on the deep score.
|
||||
* `append_buffer` moves admitted leaves into a buffer — `buffered`.
|
||||
* `seal` writes the buffer's contents into a summary and marks each leaf `sealed`.
|
||||
* `dropped` leaves stop here. Their chunk row stays for provenance, but no buffer / summary references them.
|
||||
|
||||
This is why retrieval can show provenance without re-running the pipeline: the chunk row plus its terminal lifecycle status is enough.
|
||||
|
||||
@@ -143,13 +143,13 @@ Three reasons:
|
||||
|
||||
A few rules to keep the pipeline coherent:
|
||||
|
||||
- **All paths go through `mem_tree_jobs`.** Don't add a feature that does its own background scheduling. If you need a periodic step, enqueue a job; if you need a one-shot async step, enqueue a job.
|
||||
- **Idempotent handlers.** Every handler must be safe to run twice on the same `(kind, payload, dedupe_key)`. Workers retry on transient errors; you can't assume "called once".
|
||||
- **No LLM calls in the ingest hot path.** Anything that needs a model goes into the queue.
|
||||
- **Workspace-scoped.** Tests reset state by creating a fresh `OPENHUMAN_WORKSPACE`. Don't reach outside the workspace dir.
|
||||
* **All paths go through `mem_tree_jobs`.** Don't add a feature that does its own background scheduling. If you need a periodic step, enqueue a job; if you need a one-shot async step, enqueue a job.
|
||||
* **Idempotent handlers.** Every handler must be safe to run twice on the same `(kind, payload, dedupe_key)`. Workers retry on transient errors; you can't assume "called once".
|
||||
* **No LLM calls in the ingest hot path.** Anything that needs a model goes into the queue.
|
||||
* **Workspace-scoped.** Tests reset state by creating a fresh `OPENHUMAN_WORKSPACE`. Don't reach outside the workspace dir.
|
||||
|
||||
## See also
|
||||
|
||||
- [Memory Tree (user-facing)](../features/memory-tree.md) — the product surface this pipeline powers.
|
||||
- [Memory Context Window](memory-context-window.md) — how much of the resulting state lands in each agent turn.
|
||||
- [Local AI](../features/local-ai.md) — opt-in path for running embeddings + summarization on-device.
|
||||
* [Memory Tree (user-facing)](../features/memory-tree.md) — the product surface this pipeline powers.
|
||||
* [Memory Context Window](memory-context-window.md) — how much of the resulting state lands in each agent turn.
|
||||
* [Local AI](../features/local-ai.md) — opt-in path for running embeddings + summarization on-device.
|
||||
|
||||
@@ -1,12 +1,14 @@
|
||||
---
|
||||
description: >-
|
||||
Every memory chunk also lives as a Markdown file in an Obsidian-compatible
|
||||
vault you can open and edit. Inspired by Karpathy's obsidian-wiki workflow.
|
||||
Every memory chunk also lives as a Markdown file in an Obsidian-compatible
|
||||
vault you can open and edit. Inspired by Karpathy's obsidian-wiki workflow.
|
||||
icon: book-open
|
||||
---
|
||||
|
||||
# Obsidian Wiki
|
||||
|
||||
<figure><img src="../.gitbook/assets/image.png" alt=""><figcaption><p>A preview of the OpenHuman memory in Obsidian. Data from various sources (GMail, Slack, Whatsapp etc..) is organized as a memory tree.</p></figcaption></figure>
|
||||
|
||||
OpenHuman's memory is not a black box. The same chunks the agent reasons over are written as plain `.md` files into a vault inside your workspace. You can open it in [Obsidian](https://obsidian.md), browse it, edit it, and link notes by hand — and the agent will see your edits.
|
||||
|
||||
The design is directly inspired by [Andrej Karpathy's obsidian-wiki workflow](https://x.com/karpathy/status/2039805659525644595): a personal wiki where every interesting thing in your life ends up as a linkable note.
|
||||
@@ -35,9 +37,9 @@ Anything you put in `wiki/notes/` is fair game for ingest. The same pipeline tha
|
||||
|
||||
This means you can:
|
||||
|
||||
- Drop a meeting note in `wiki/notes/2026-05-08-board-call.md` and the agent will know the context tomorrow.
|
||||
- Maintain a file per project, per person, per ticker — the topic tree treats your manual notes as just another source.
|
||||
- Bulk-import an existing Obsidian vault: drop the `.md` files in and trigger ingest.
|
||||
* Drop a meeting note in `wiki/notes/2026-05-08-board-call.md` and the agent will know the context tomorrow.
|
||||
* Maintain a file per project, per person, per ticker — the topic tree treats your manual notes as just another source.
|
||||
* Bulk-import an existing Obsidian vault: drop the `.md` files in and trigger ingest.
|
||||
|
||||
## Why this matters
|
||||
|
||||
@@ -47,5 +49,5 @@ It's also the cleanest possible export: stop using OpenHuman tomorrow and you ke
|
||||
|
||||
## See also
|
||||
|
||||
- [Memory Tree](memory-tree.md) — the pipeline that produces the vault.
|
||||
- [Auto-fetch from Integrations](auto-fetch.md) — how the vault grows on its own.
|
||||
* [Memory Tree](memory-tree.md) — the pipeline that produces the vault.
|
||||
* [Auto-fetch from Integrations](auto-fetch.md) — how the vault grows on its own.
|
||||
|
||||
Reference in New Issue
Block a user