openhuman/docs/tinycortex-memory-migration-plan.md

TinyCortex Memory Migration — Plan & Audit

Status: draft plan — no code changes yet. Anchor precedent: the TinyAgents harness migration (#4249 / #4399 / #4473 and follow-ups). Target: migrate large portions of the OpenHuman memory subsystem onto the tinycortex crate, vendored as a git submodule at vendor/tinycortex (https://github.com/tinyhumansai/tinycortex).

---

0. Ground truth (as audited)

0.1 What exists today

Host memory subsystem (in-tree, all still live):

Host module	LOC (approx)	Role
src/openhuman/memory/	~20,700 (79 files)	Orchestration/policy layer: Memory trait, RPC ops/schemas, agent tools, ingest orchestrator, sync lifecycle, preferences, source-scope task-locals, redaction
src/openhuman/memory_store/	~32,800	Storage: UnifiedMemory/MemoryClient, chunks DB, content store, vectors, KV, safety
src/openhuman/memory_tree/	~25,300	Summary trees, retrieval, scoring, health
src/openhuman/memory_sync/	~30,700	Live sync: Composio, MCP, OAuth, pollers
memory_queue, memory_sources, memory_diff, memory_goals, memory_entities, memory_graph, memory_archivist, memory_tools, memory_search	~20k combined	Engine long tail

TinyCortex (vendor/tinycortex, crate tinycortex v0.1.1, MIT, single crate, ~49,500 lines under src/memory/): an already-completed port of the engine layers — store/ (content markdown + YAML, SQLite vectors, KV, entity_index, safety), chunks/, tree/, queue/, retrieval/, score/, ingest/ (canonicalize/extract/pipeline), conversations/, archivist/, diff/, sources/ (local readers only), entities/, graph/, goals/, tool_memory/. Its git history is explicitly "port X from OpenHuman" commits, and its types (MemoryEntry, MemoryCategory, MemoryTaint, RecallOpts, NamespaceSummary, the Memory trait) are wire-compatible with the host's memory::traits.

Wiring already pre-staged:

• .gitmodules has vendor/tinycortex → tinyhumansai/tinycortex.
• Root Cargo.toml [patch.crates-io] block includes tinycortex = { path = "vendor/tinycortex" } (same mechanism as tinyagents: crates.io version pin + path patch). app/src-tauri/Cargo.toml has a matching path entry.
• CI (test-reusable.yml, build-ci-image.yml, release workflows) already checks out submodules recursively. No new CI plumbing needed.

Gap: zero .rs files in src/ import tinycortex. There is no [dependencies] tinycortex = "0.1" entry activating the patch. The host still runs entirely on the in-tree engine, which has continued to evolve since the port was taken.

0.2 TinyCortex's declared boundary (the seam contract)

TinyCortex deliberately does not own (per its own module docs):

• Live sync / polling / OAuth / network source readers (Composio, GitHub, RSS, web) — only folder and conversation local readers ship.
• RPC schemas, event bus, CLI, agent-tool surfaces.
• LLM and embedding compute — abstracted behind EmbeddingBackend, score::extract::ChatProvider, tree::summarise::Summariser traits ("never makes a network call").
• A tokio worker pool — the host drives the queue via queue::run_once / drain_until_idle.
• The namespace document/graph store and the Composio identity registry.

This boundary is the migration contract: engine in TinyCortex, product policy + I/O + surfaces in OpenHuman — exactly mirroring the tinyagents split (generic runtime in the crate; prompts, security, RPC compat, UX in the host).

0.3 Submodule contribution workflow (how engine changes are made)

TinyCortex lives at vendor/tinycortex as a git submodule. Any change to engine code — whether by a human or by an LLM agent — is made inside the submodule, committed on a branch there, and raised as a PR against tinyhumansai/tinycortex. Once merged, the host repo bumps the submodule pointer in a standalone chore(vendor): bump tinycortex — <summary> (tinycortex#<PR>) commit, keeping the [dependencies] version pin in lockstep (same convention as chore(vendor): bump tinyagents commits citing tinyagents#11 etc.). Host-side PRs never contain engine source edits; they only move the submodule SHA.

---

1. Target ownership split

Moves to TinyCortex (delete from host after cutover)

Host module	TinyCortex counterpart	Notes
memory_store/ (chunks, content, vectors, kv, entity_index, safety)	store/, chunks/	Largest single win (~33k LOC)
memory_tree/ (tree, retrieval, score, summarise)	tree/, retrieval/, score/	Host keeps tree_policy.rs flavour constants
memory_queue/	queue/	Host keeps the tokio worker loop that drives it
memory/ingest_pipeline.rs internals	ingest/	Host keeps the thin entry points (ingest_chat, ingest_document_with_scope) as adapters
memory_diff/, memory_entities/, memory_graph/, memory_goals/, memory_archivist/, memory_sources/ (registry + local readers), tool_memory engine, conversations engine	same-named modules	Long tail
memory/traits.rs core types	tinycortex::memory::{Memory, MemoryEntry, MemoryCategory, MemoryTaint, RecallOpts, NamespaceSummary, …}	Host re-exports from the crate so 30+ consumer sites keep compiling unchanged

Stays in OpenHuman (product policy, I/O, surfaces)

• All RPC surfaces: memory/ops/, memory/schemas/, memory/schema/, memory/read_rpc/, rpc_models.rs (controller framework types ControllerSchema/RpcOutcome are host-only). JSON-RPC method names and payload shapes must not change.
• Agent tools: memory/tools/ and memory/query/ (Tool/ToolResult impls, SecurityPolicy gating) — they become thin wrappers over crate retrieval primitives.
• Live sync: all of memory_sync/ (Composio/MCP/OAuth/pollers), memory/sync.rs lifecycle + event-bus stage events.
• Process glue: memory/global.rs singleton + background queue worker; memory/source_scope.rs tokio task-locals; memory/chat.rs (LLM adapter over openhuman::inference); embeddings provider wiring.
• Policy/UX: preferences.rs, remember.rs, tree_policy.rs, util/redact.rs, config mapping (Config → tinycortex::MemoryConfig).
• Namespace document/graph store (until/unless deliberately upstreamed — TinyCortex explicitly excludes it today).

The adapter seam: src/openhuman/tinycortex/

New sibling module mirroring src/openhuman/tinyagents/, holding every impl of a TinyCortex trait over an OpenHuman service:

• embeddings.rs — impl tinycortex EmbeddingBackend over openhuman::embeddings (dim/model/signature from Config).
• chat.rs — impl ChatProvider + impl Summariser over memory::chat::build_chat_provider / inference::provider.
• queue_driver.rs — tokio worker loop calling queue::run_once/drain_until_idle, owned by memory/global.rs, with event-bus progress emission and Sentry hooks (host-side, since TinyCortex dropped its scheduler).
• config.rs — Config → MemoryConfig (workspace roots, EmbeddingConfig, TreeConfig, WeightProfile, SyncBudgetConfig) with tree_policy.rs flavour overlays.
• sinks.rs — TreeJobSink, SnapshotItemSource impls bridging to host state.
• bus.rs — translate engine outcomes into core::event_bus DomainEvents (host-side only; the crate stays bus-free).
• mod.rs — facade re-exports (pub use tinycortex::memory::{…}) so the rest of the host imports through one seam, plus module-doc explaining the boundary (the tinyagents seam's mod.rs header is the template).

---

2. Phase 0 — Audit & baseline (before any cutover)

This phase produces documents and upstream issues only; it is the gate for everything after.

0.1 Drift audit. The port in TinyCortex was taken at a point-in-time; the host engine has since received changes (perf waves, bug fixes). For each host module → crate module pair in the table above, diff behavior and enumerate host commits since the port SHA that must be re-applied upstream. Output: a per-module drift ledger (host commit → tinycortex PR needed / already present / obsolete). This is the highest-risk unknown in the whole migration — nothing cuts over until its module's drift ledger is closed.

0.2 API gap audit. Enumerate every host call site into memory_store/memory_tree/memory_queue/long-tail modules (the internal sibling graph: ~25 uses of memory_store::chunks, 6 of trees, etc.) and map each to a TinyCortex public API. Output: gap list, each gap becoming a tinycortex issue/PR (e.g. the known gaps: graph edge accumulation at persist time, seal-time embedding, seal_document_subtree follow-ups, tree health/doctor — memory/tools/doctor.rs wraps memory_tree::health, which the crate defers).

0.3 Data-format parity audit. Existing user workspaces must open unchanged after cutover. Verify byte/schema compatibility for: chunks.db SQLite schema + migrations, jobs table, tree tables, vector table encoding (packed f32), markdown content store paths + YAML frontmatter, entity markdown, git diff-ledger layout, deterministic chunk IDs. Output: a parity checklist with a fixture-based verification harness design (golden workspace snapshot opened by both engines, compared). Any mismatch is an upstream fix, not a host workaround.

0.4 Toolchain baseline. Add tinycortex = { version = "0.1" } under [dependencies] (activating the existing [patch.crates-io] override); align rusqlite versions between host and crate (both must link one bundled SQLite); check edition (crate is 2021), feature flags, and that both Cargo worlds (root crate and app/src-tauri) compile with the dep active; confirm GGML_NATIVE=OFF macOS builds. Verify the release workflows' submodule-init covers vendor/tinycortex (the tinyagents wave needed +5-line fixes there).

0.5 Type-unification decision. Host memory::traits types and crate types are wire-compatible twins. Decide: host re-exports crate types (preferred — one source of truth, 30+ consumer sites unchanged via pub use), vs. keeping host types + From conversions (fallback if serde/API divergence is found in 0.3). Special care: MemoryTaint is security-critical provenance (fails closed to ExternalSync, drives external-effect-tool gating) — its semantics, serde representation, and fail-closed defaults must be proven identical before re-exporting.

0.6 Spec doc. Write docs/tinycortex-migration-spec.md version-anchored to exact reviewed SHAs (host + tinycortex), with the ownership lists above, the drift/gap/parity ledgers, and a deletion ledger skeleton (every legacy file, with preconditions for deletion) — directly modeled on docs/tinyagents-migration-spec.md + 99-deletion-ledger.md.

---

3. Cutover workstreams (Phase 1–8)

Per the tinyagents rules: adapter first → prove parity → flip ownership → delete legacy, deletion mandatory and enumerated per step. Ordering follows the engine's dependency graph (storage first, surfaces last). Within every workstream, implementation lands first, tests second (see §5).

W1 — Seam scaffolding. Create src/openhuman/tinycortex/ with the adapters in §1 (config, embeddings, chat/summariser, queue driver, sinks, bus bridge, facade). No behavior flips yet; adapters are constructed and unit-verified against the crate's inert defaults. Deliverable: crate is linked, adapters compile, MemoryConfig derived from real Config.

W2 — Types & trait cutover. memory/traits.rs becomes re-exports of tinycortex types (per 0.5 decision). All 30+ external consumers (agent/harness, learning, channels/runtime, subconscious, threads, …) compile unchanged through the re-export. sqlite_conn() escape hatch on the host trait is reviewed: either upstreamed or kept as a host-side extension trait.

W3 — Store + chunks. Flip memory_store internals (UnifiedMemory/MemoryClient re-implemented over tinycortex::store + chunks), keeping the host-facing MemoryClient API stable so global.rs and all RPC ops are untouched. Delete legacy memory_store engine files. Gate: 0.3 parity harness green on a golden workspace.

W4 — Queue. memory_queue → tinycortex::queue, driven by the seam's tokio worker. Job payload/schema parity from 0.3. Delete memory_queue.

W5 — Tree + retrieval + score. memory_tree → tinycortex::{tree, retrieval, score}. Host memory/query/* tools and read_rpc/* re-point to crate primitives (query_source, drill_down, cover_window, fetch_leaves, search_entities, MMR/hybrid scoring). tree_source/registry.rs wraps the crate registry; tree_policy.rs stays host. source_scope enforcement point re-verified (retrieval must still respect the per-turn allowlist — this is a security surface). Delete memory_tree.

W6 — Ingest. memory/ingest_pipeline.rs + memory/ingestion/ re-pointed to tinycortex::ingest + score::extract (LLM extraction via the seam's ChatProvider). The namespace document/graph store path stays host-side unless deliberately upstreamed (explicit decision in this workstream). ingest_chat/ingest_document_with_scope keep their signatures — 11 call sites (learning, agent harness, archivist) unchanged.

W7 — Long tail. memory_diff→diff, memory_entities→entities, memory_graph→graph, memory_goals→goals, memory_archivist→archivist, memory_sources registry/local readers→sources, tool-memory engine→tool_memory, conversation storage→conversations. Each is small and independent; each deletes its legacy module on flip. memory_sync explicitly does not move — it keeps writing through the crate's ingest/source contracts.

W8 — Test port + parity sweep + deletion-ledger close-out. See §5. Ends with: deletion ledger fully executed, gitbooks/developing/architecture.md + a new architecture/memory.md seam doc written (the durable post-plan documentation, as agent-harness.md was for tinyagents), AGENTS.md/CLAUDE.md module tables updated, and the spec doc archived.

Sequencing note

W3–W5 are the risky core (user data on disk). W1–W2 can land quickly; W6–W7 parallelize after W5. Expect the real-world shape to match tinyagents: one or two substantial cutover-wave PRs plus a long tail of small parity-fix PRs — budget for that tail explicitly.

---

4. Git / PR / submodule workflow

• Branching: all work on feature branches off upstream/main (never on main), small focused commits, explicit git add <paths> (never -A), one workstream ≈ one PR against tinyhumansai/openhuman.
• Engine changes → submodule PRs: any modification to engine code (drift re-application, gap-filling, parity fixes) is committed inside vendor/tinycortex on a branch and PR'd to tinyhumansai/tinycortex. This applies equally to human contributors and LLM agents: change the submodule, raise the PR from there. Host PRs consume merged engine changes via chore(vendor): bump tinycortex — <what> (tinycortex#<n>) commits.
• Version lockstep: keep the [dependencies] tinycortex = "<version>" pin in lockstep with the submodule tag; publish tinycortex to crates.io at each milestone so non-vendored consumers resolve (identical to the tinyagents 1.5.0 + patch pattern).
• Interleaving: a typical workstream is a sandwich — (a) tinycortex PR(s) closing that module's drift/gap ledger, (b) host chore(vendor) bump, (c) host cutover PR (adapter flip + legacy deletion), (d) host test PR/commits. CI already handles recursive submodule checkout on all build/test lanes.

---

5. Testing strategy — implementations first, tests second

Ordering rule for this migration: within each workstream, the implementation (adapter + cutover + deletion) lands first; the test work follows as the second slice. Tests are not skipped — they are sequenced after the implementation is proven to compile and pass the existing suites. Concretely:

1. Slice A (impl): adapters + cutover + legacy deletion. Gate: cargo check both worlds, existing crate-level integration tests (tests/memory_roundtrip_e2e.rs, memory_tree_sync_deep_raw_coverage_e2e.rs, etc.) still green — these exercise the public openhuman::memory:: surface and act as the built-in parity harness for every flip.
2. Slice B (tests): port and extend the test estate for the new boundary:
   • Engine-internal tests move upstream. The host's #[path]-included sibling tests (ops_tests.rs pattern) that poke engine internals migrate into vendor/tinycortex as crate tests (via tinycortex PRs), following the crate's own *_tests.rs sibling convention.
   • Host keeps boundary tests: RPC schema/handler tests, tool gating tests, seam adapter tests (config mapping, embedding signature, taint fail-closed, source-scope enforcement), and the E2E files under tests/.
   • Test-harness re-plumbing is its own task: GLOBAL_MEMORY_TEST_LOCK, ensure_shared_memory_client, and config::TEST_ENV_LOCK are host globals the sibling tests depend on; upstreamed tests need crate-local fixtures instead.
3. Coverage gate reality check: PR CI enforces ≥80% diff coverage on changed lines, and host coverage tooling does not count tests living in the vendored crate. So "tests second" means second slice of the same PR (impl commits, then test commits, one PR) for host-side changes — not a separate later PR — or the coverage gate blocks the merge. Pure-deletion diffs and the seam re-export shims are the low-coverage-risk parts; the seam adapters need their host-side unit tests in the same PR.
4. W8 parity sweep (last): golden-workspace fixture opened by pre- and post-migration builds (read-side comparison of recall/retrieval/tree output), full pnpm test:rust + JSON-RPC E2E + the memory-related tests/*_e2e.rs matrix, plus a manual upgrade test on a real developer workspace.

---

6. Risk register

Risk	Severity	Mitigation
Engine drift since the tinycortex port (host perf/bug fixes not upstream)	High	Phase 0.1 drift ledger is a hard gate per module; no cutover with an open ledger
On-disk format divergence (SQLite schemas, chunk IDs, vault layout) breaking existing user workspaces	High	Phase 0.3 golden-workspace parity harness; upstream fixes only, never host shims
MemoryTaint / source_scope security semantics silently weakened across the boundary	High	Dedicated seam tests for fail-closed taint and scope enforcement; treated as security review items in W2/W5
Blast radius of memory::traits (30+ sites), global (25), chat (~20), redact (12)	Medium	Re-export strategy keeps import paths stable; redact/chat/global never move
rusqlite / dependency version skew (two bundled SQLites, two Cargo worlds)	Medium	Phase 0.4 alignment before any dep activation; watch app/src-tauri lockfile too
Coverage gate vs. vendored tests (host CI can't count crate-side tests)	Medium	Host-side seam/boundary tests in the same PR as impl (§5.3)
Sibling #[path] tests bound to private internals won't move cleanly	Medium	Explicit test-port slice per workstream; crate-local fixtures replace host globals
Queue driver behavior change (crate has no scheduler; host loop replaces tokio pool + Sentry hooks)	Medium	W4 keeps worker cadence/backoff/error-reporting parity; verbose [memory]-prefixed logging per repo logging rules
Namespace doc/graph store ambiguity (crate excludes it)	Low	Explicit keep-host decision in W6; revisit upstreaming later
Long parity tail (tinyagents needed ~15 follow-up PRs)	Expected	Budget the tail; isolate each parity fix as a small scoped PR, engine fixes via submodule PRs

---

7. Definition of done

• memory_store, memory_tree, memory_queue, memory_diff, memory_entities, memory_graph, memory_goals, memory_archivist, engine parts of memory_sources/memory_tools/conversations deleted from src/openhuman/; deletion ledger fully checked off.
• Host memory code = policy/surfaces only: memory/ (RPC, tools, sync lifecycle, preferences, scope, redact, global) + src/openhuman/tinycortex/ seam + memory_sync/.
• All engine logic served by tinycortex at a tagged, crates.io-published version, submodule pinned in lockstep.
• JSON-RPC method names/payloads unchanged; existing user workspaces open and recall identically (golden-workspace parity green).
• Full suites green on both CI lanes; gitbooks/AGENTS.md updated; spec + ledgers archived.