b9925e0bf4
## Summary - Added an on-device multilingual PII redactor covering 15 identifier types — Brazilian CPF/CNPJ (mod-11), Argentine CUIT, Mexican RFC, Japanese マイナンバー, US SSN with reserved-range filter, Luhn-validated credit cards, mod-97 IBAN, Verhoeff-validated Aadhaar, Indian PAN, UK NINO, Spanish DNI/NIE, Korean RRN, E.164 and NANP phones — and wired it into the existing `sanitize_text` pipeline that runs on every memory write. - Added a Unicode-normalization pre-pass that strips zero-width characters and folds fullwidth/Arabic-Indic digits + punctuation, defeating common regex-bypass tricks before matching while preserving non-PII bytes in the original text. - Added `has_likely_pii()` and wired it into the namespace/key boundary checks in `documents.rs` and `kv.rs` so PII-bearing inputs are rejected outright, mirroring how `has_likely_secret()` is enforced today. - Extended `SanitizationReport` with a `pii_redactions` counter and surfaced it in the existing `[memory:safety]` audit-log lines across `documents.rs`, `kv.rs`, and `fts5.rs`. ## Problem - OpenHuman ingests data from 118+ integrations into Memory Tree. The existing `memory::safety` module redacted API keys, tokens, and PEM blocks but had no coverage for personal PII — national IDs, financial identifiers, phone numbers. - Issue #2017 proposed sending raw content to a third-party HTTP endpoint (`api.trustboost.dev`) for "sanitization". That approach directly contradicts OpenHuman's privacy-first, on-device posture — it would exfiltrate the exact PII it claims to protect to an unaffiliated vendor. - A naive regex implementation would still leave two real gaps: (a) the multilingual identifier formats that motivated the original issue (LATAM, JP, IN, EU, KR) and (b) trivial bypass via fullwidth-digit or zero-width-character obfuscation, which any motivated attacker (or accidentally-pasted Japanese-locale data) will trigger. ## Solution - Built `src/openhuman/memory/safety/pii.rs` with 15 PII categories. Where checksums exist (CPF/CNPJ mod-11, CUIT, credit-card Luhn, IBAN mod-97, Aadhaar Verhoeff, Spanish DNI/NIE check-letter, SSN reserved-range), false-positives are rejected at the algorithm level — no LLM, no network. Where checksums don't exist (RFC, PAN-IN, NANP, E.164, RRN), structural format rules carry the discrimination. - Added a `NormalizedView` that strips U+200B/200C/200D/FEFF/2060/180E and folds fullwidth (`０-９`, `．－／：`) plus Arabic-Indic / Eastern Arabic-Indic digits to ASCII before matching. Match offsets are mapped back to the original byte positions so only PII bytes are replaced — surrounding text (including any intentional fullwidth glyphs) is byte-identical to input. - Patterns run in priority order (formatted before bare, IBAN before credit-card, etc.) with overlap-deduplication so a single span can't be redacted twice or partially counted as multiple types. A `RegexSet` pre-filter short-circuits PII-free text in one scan instead of ~18 per-pattern scans. - `has_likely_pii()` mirrors `has_likely_secret()` and is wired into the same boundary checks in `unified/documents.rs` (both `upsert_document` and `upsert_document_metadata_only`) and `unified/kv.rs` (both `kv_set_global` and `kv_set_namespace`). - Added 37 new tests in `pii.rs` and 2 integration tests in `safety/mod.rs`: positive + negative per pattern, checksum-failing rejection cases, Unicode/zero-width bypass attempts, `has_likely_pii` gating, and an aggressive mixed-language end-to-end test covering 13 PII types in one document. Full safety suite: 53 tests passing. Full memory module: 1007 tests passing, zero regressions. ## Submission Checklist - If a section does not apply to this change, mark the item as N/A with a one-line reason. Do not delete items. - [x] Tests added or updated (happy path + at least one failure / edge case) per Testing Strategy — 39 new tests; checksum-failing and bypass-attempt negatives included - [x] Diff coverage ≥ 80% — new file is ~100% line-covered by inline tests; integration sites have direct assertions in `safety/mod.rs::tests` - [x] All affected feature IDs from the matrix listed in ## Related — N/A no existing matrix IDs cover memory safety redaction; the new row above will be the first - [x] No new external network dependencies introduced — deliberately on-device; no new crates; `regex` and `once_cell` already in tree - [x] Manual smoke checklist updated — N/A not a release-cut UI surface - [x] Linked issue closed via Closes #NNN — see ## Related ## Impact - Runtime/platform impact: Rust core memory ingestion path only. Desktop app behavior change is two-fold for users — (1) memory writes now produce additional `[REDACTED_PII_*]` tokens in stored content when format-matching PII is present, (2) a new error return (`document/kv namespace/key cannot contain personal identifiers`) on the rare case where a caller tries to use a PII-shaped string as a namespace or key. - Performance: `RegexSet` pre-filter makes PII-free text a single-scan no-op. On text containing PII, adds one normalized-string allocation plus a handful of regex scans gated by the screen — negligible compared to the embedding/SQLite/markdown-sidecar costs already on the write path. No measurable impact on ingestion latency in local testing. - Security/migration/compatibility: no schema changes, no new dependencies. The boundary-gate rejection is a behavior change for any caller that previously stored namespace/keys *containing* identifier-shaped strings; expected impact is zero in practice because real namespace/keys are paths like `memory/global/preferences`. Privacy posture is strictly improved — every byte stays on device; no telemetry, no outbound calls. ## Related - Closes: #2017 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Personal information detection and redaction integrated into the memory system * Write operations on namespace and key fields now validate against personal identifiers * **Improvements** * Enhanced sanitization reports with additional metrics on personal identifier redactions <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/tinyhumansai/openhuman/pull/2310?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> Co-authored-by: Shanu <shanu@tinyhumans.ai> Co-authored-by: Steven Enamakel <enamakel@tinyhumans.ai>
OpenHuman
OpenHuman is your Personal AI super intelligence. Private, Simple and extremely powerful.
Discord • Reddit • X/Twitter • Docs • Follow @senamakel (Creator)
🇺🇸 English | 🇨🇳 简体中文 | 🇯🇵 日本語 | 🇰🇷 한국어 | 🇩🇪 Deutsch
Early Beta: Under active development. Expect rough edges.
To install or get started, either download from the website over at tinyhumans.ai/openhuman or run
# Download DMG, EXEs over at https://tinyhumans.ai/openhuman or run in from your terminal
# For macOS or Linux x64
curl -fsSL https://raw.githubusercontent.com/tinyhumansai/openhuman/main/scripts/install.sh | bash
# For Windows
irm https://raw.githubusercontent.com/tinyhumansai/openhuman/main/scripts/install.ps1 | iex
What is OpenHuman?
OpenHuman is an open-source agentic assistant designed to integrate with you in your daily life. Each bullet links to the deeper writeup in the docs.
-
Simple, UI-first & Human A clean desktop experience and short onboarding paths take you from install to a working agent in a few clicks — no config-first setup, no terminal required. The agent has a face: a desktop mascot that speaks, reacts to its surroundings, joins your Google Meets as a real participant, remembers you across weeks, and keeps thinking in the background even when you've stopped typing.
-
118+ third-party integrations with auto-fetch: plug into Gmail, Notion, GitHub, Slack, Stripe, Calendar, Drive, Linear, Jira and the rest of your stack with one-click OAuth. Every connection is exposed to the agent as a typed tool, and every twenty minutes the core walks each active connection and pulls fresh data into the memory tree. No prompts, no polling loops you have to write, so the agent already has tomorrow's context this morning.
Managed integrations are backend-proxied through OpenHuman's Composio connector layer. If you want to run Composio directly instead of using the managed backend path, configure direct mode with your own Composio API key; real-time trigger webhooks then need to be hosted and wired by you.
-
Memory Tree + Obsidian Wiki: a local-first knowledge base built from your data and your activity. Everything you connect is canonicalized into ≤3k-token Markdown chunks, scored, and folded into hierarchical summary trees stored in SQLite on your machine. The same chunks land as
.mdfiles in an Obsidian-compatible vault you can open, browse and edit, inspired by Karpathy's obsidian-wiki workflow. -
Batteries included: web search, a web-fetch scraper, a full coder toolset (filesystem, git, lint, test, grep), and native voice (STT in, ElevenLabs TTS out, mascot lip-sync, live Google Meet agent) are wired in by default. Model routing sends each task to the right LLM (reasoning, fast, or vision) under one subscription. No "install a plugin to read files" friction. Optional local AI via Ollama for on-device workloads.
-
Smart token compression (TokenJuice): every tool call, scrape result, email body, and search payload is run through a token compression layer before it touches any LLM Model. HTML is converted to Markdown, long URLs are shortened, and verbose tool output is deduped and summarized via a configurable rule overlay etc... CJK, emoji, and other multi-byte text are preserved grapheme-by-grapheme — never stripped. You get the same information but at a fraction of the tokens. Reducing cost & latency by up to 80%.
-
Messaging channels and privacy & security: inbound/outbound across the channels you already use, with workflow data that stays on device, encrypted locally, treated as yours.
Contributing from source
New contributor? Start with CONTRIBUTING.md for the fork/PR workflow and local validation commands, or use the copy-paste AI-agent prompt in CONTRIBUTING-BEGINNERS.md. The short path is:
- Install Git, Node.js 24+, pnpm 10.10.0, Rust 1.93.0 (
rustfmt+clippy), CMake, Ninja, ripgrep, and the platform desktop build prerequisites. - Fork and clone the repo, then run
git submodule update --init --recursivebeforepnpm installso the vendored Tauri/CEF sources are present. - Use
pnpm devfor web-only UI work,pnpm --filter openhuman-app dev:appfor the desktop shell, and focused checks such aspnpm typecheck,pnpm format:check, andcargo check -p openhuman --libbefore opening a PR.
Deeper docs: Architecture · Getting Set Up · Cloud Deploy.
Context in minutes, not weeks
OpenHuman is the first agent harness that gets to know you in minutes. Inspired by Karpathy's LLM Knowledgebase. Most agents start cold. Hermes learns by watching you work; OpenClaw waits for plugins to ferry context in. Either way, you spend days or weeks before the agent knows enough about your stack to be genuinely useful.
OpenHuman summarizes and compresses all your documents, emails & chats; and creates a memory graph that lets your agent remember everything about you.
OpenHuman skips the wait. Connect your accounts, let auto-fetch pull data locally on a 20-minute loop, and then have Memory Trees compress everything into Markdown files stored intelligently in a Karpathy-style Obsidian wiki.
In just one sync pass, the agent has full (compressed) context of your inbox, your calendar, your repos, your docs, your messages. No training period. No "give it a few weeks.". It becomes you, controlled by you.
Already self-host agentmemory across other coding agents? OpenHuman ships an optional Memory backend that proxies to it — set memory.backend = "agentmemory" in config.toml and the same durable store powers OpenHuman alongside Claude Code, Cursor, Codex, and OpenCode. See the agentmemory backend page for setup.
OpenHuman vs Other Agent Harnesses
High-level comparison (products evolve, so verify against each vendor). OpenHuman is built to minimize vendor sprawl, keep workflow knowledge on-device, and give the agent a persistent memory of your data, not only chat.
| Claude Cowork | OpenClaw | Hermes Agent | OpenHuman | |
|---|---|---|---|---|
| Open-source | 🚫 Proprietary | ✅ MIT | ✅ MIT | ✅ GNU |
| Simple to start | ✅ Desktop + CLI | ⚠️ Terminal-first | ⚠️ Terminal-first | ✅ Clean UI, minutes |
| Cost | ⚠️ Sub + add-ons | ⚠️ BYO models | ⚠️ BYO models | ✅ One sub + TokenJuice |
| Memory | ✅ Chat-scoped | ⚠️ Plugin-reliant | ✅ Self-learning | 🚀 Memory Tree + Obsidian vault, optional agentmemory backend |
| Integrations | ⚠️ Few connectors | ⚠️ BYO | ⚠️ BYO | 🚀 118+ via OAuth |
| Auto-fetch | 🚫 None | 🚫 None | 🚫 None | ✅ 20-min sync into memory |
| API sprawl | 🚫 Extra keys | 🚫 BYOK | 🚫 Multi-vendor | ✅ One account |
| Model routing | 🚫 Single model | ⚠️ Manual | ⚠️ Manual | ✅ Built-in |
| Native tools | ✅ Code-only | ✅ Code-only | ✅ Code-only | ✅ Code + search + scraper + voice |
Star us on GitHub
Building toward AGI and artificial consciousness? Star the repo and help others find the path.
Contributors Hall of Fame
Show some love and end up in the hall of fame. Contributors get free merch and special access to our Discord.